The sleeping virus can harbour mutations which, like an invisibility cloak, help it evade detection by the immune system, they said.
But there was encouraging news as well.
Lab experiments offered hope that the immune system can be trained to spot the peril and then eradicate it.
The research, published in the journal Nature, touches on the "kick-and-kill" strategy, one of the main thrusts of cure research -- the most ambitious campaign in the 33-year war on AIDS.
The human immunodeficiency virus (HIV) which causes AIDS hides away in so-called memory CD4 T cells, a component of the immune system, after the infection is rolled back by antiretroviral drugs.
Once the therapy is stopped, the virus rebounds, threatening the patient once more.
"Kick-and-kill" looks at how to kick the virus from its redoubt with powerful drugs and then kill it once it is forced into the open.
The new research explains in part why this goal seems so hard to achieve, but also offers a possible avenue, the scientists said.
"Our results suggest that luring HIV out of hiding is winning only half the battle," said Robert Siliciano, a professor of molecular biology at Johns Hopkins University School of Medicine in Baltimore, Maryland.
"We found that these pools of dormant virus carry mutations that render HIV invisible to the very immune cells capable of disarming it, so even when the virus comes out of hiding, it continues to evade immune detection."
The team analysed blood samples from 25 patients with HIV. Ten had begun therapy very soon after infection, and 15 had started the drugs only when the virus had spread to a chronic stage.
In a striking finding, the researchers discovered that those who had been early initiators of therapy had holdout virus with almost no mutations.
But those who had begun therapy later had virus that was highly altered -- it was stuffed with "escape mutations" which made it undetectable to immune sentinels.
Even so, even in this highly mutated state, the virus had retained a tiny bit of its original viral protein -- a "conserved" piece that could be an Achilles' heel.
The scientists took uninfected immune cells and exposed them to virus that was either mutated or had the "conserved," non-mutated form.
- Trained to kill -
Thus primed and ready for action, the cells were then exposed to infected cells taken from patients with the now-notorious escape mutations.
Immune cells that had been previously primed with the "conserved" virus were able to kill 61 percent of these infected cells.
But cells primed only with mutant HIV responded weakly, eliminating only 23 percent of the infected cells.
"It's as if the immune system had lost its ability to spot and destroy the virus, but priming killer T cells that recognise a different, non-mutated portion of HIV's protein reawakened that natural killer instinct," Siliciano said in a statement.