Statisticians review landmark HIV vaccine trial

May 9, 2011

Hopes ran high in 2009 when a New England Journal of Medicine article announced success in developing a vaccine against HIV, the virus that causes AIDS.

But a new Bayesian statistical analysis initiated at Duke demonstrates that the results were far from conclusive.

“The world has been waiting for an HIV for almost 30 years,” said Victor DeGruttola, a biostatistician at the Harvard School of Public Health who participated in the new analysis. "Its efficacy is an emotionally charged issue.” The HIV vaccine study, called RV 144, had been viewed as the “first evidence that suggested we were close."

New analysis of the RV144 data shows it is more likely that the vaccine actually has no effect compared to what was presented in the initial analysis. The new analysis also highlights the need to regularly use multiple statistical methods to review new medicines, said James Berger, a statistician at Duke University who spearheaded the study along with Peter Gilbert of the University of Washington and DeGruttola.

The results appear in the April 1 issue of the Journal of Infectious Diseases. The new analysis uses Bayesian statistics, which includes information from the original data and also data from previous HIV vaccine research.

The original RV144 study reported new HIV infections in 51 of the 8,197 volunteers given the vaccine versus 74 of the 8,198 participants given the placebo. These results suggested that the vaccine, a combination of two other failed vaccines, was 31.2 percent effective, with a p-value of .04. Many researchers and journalists interpreted the statistics to mean that there was only a four percent chance that the vaccine was not effective, and alternatively a 96 percent chance that it was 31-percent effective.

“But it’s not that simple,” DeGruttola said. He was dismayed with the interpretation and criticized it in the journal Science. He said the correct way to think about the numbers is to assume the vaccine does nothing. “Assume it is water. Repeating the experiment 50 times, researchers would expect that one out of 50 times they'd find equal-to or greater-than 31.2 percent vaccine efficacy,” he said, adding that these standard statistics ultimately say nothing direct about whether the vaccine works.

Berger said Bayesian analysis improves upon the standard statistical methods because it can make statements about the chance that the vaccine had some efficacy. The P-values, which were used in the original analysis, cannot. In the new analysis, the researchers explain that the p-value of 0.04 was misinterpreted and say that, using Bayesian statistics, there is no more than a 71 percent chance that the vaccine was effective at preventing HIV.

In other words, there is a 29 percent chance that the vaccine was not effective, not a 4 percent chance, he said.

Because of what is at stake in HIV vaccine research, the conversation related to RV144 has not always been as scientific as it could have been, Gilbert said. He has reviewed and devised vaccine trials since 1996 and said the new analysis provides statisticians and non-statisticians with a “golden opportunity” to have rational, scientific, and careful discussions when determining “false positive flukes” versus real effects.

“The new evidence doesn’t give us any reason for great joy, or greater gloom,” said De Gruttola, but the researchers agree that the Bayesian analysis provides a new framework to add increased rigor into future vaccine trials.

Explore further: Adaptive trial designs could accelerate HIV vaccine development

More information: "Statistical Interpretation of the RV144 HIV Vaccine Efficacy Trial in Thailand: A Case Study for Statistical Issues in Efficacy Trials," Peter B. Gilbert, et al. Journal of Infectious Disease, Vol. 203, Issue 7. doi:10.1093/infdis/jiq152

Related Stories

Insight into HIV immunity may lead to vaccine

May 6, 2011

(Medical Xpress) -- Latest insights into immunity to HIV could help to develop a vaccine to build antibodies’ defences against the disease, a University of Melbourne study has found.

Recommended for you

Targeting HIV in semen to shut down AIDS

August 18, 2015

There may be two new ways to fight AIDS—using a heat shock protein or a small molecule - to attack fibrils in semen associated with the human immunodeficiency virus (HIV) during the initial phases of infection, according ...

Vitamin D status related to immune response to HIV-1

June 15, 2015

Vitamin D plays an important part in the human immune response and deficiency can leave individuals less able to fight infections like HIV-1. Now an international team of researchers has found that high-dose vitamin D supplementation ...

HVTN 505 vaccine induced antibodies nonspecific for HIV

July 30, 2015

A study by researchers at the National Institute of Allergy and Infectious Diseases and Duke University helps explain why the candidate vaccine used in the HVTN 505 clinical trial was not protective against HIV infection ...

Why HIV's cloak has a long tail

June 2, 2015

Virologists at Emory University School of Medicine, Yerkes National Primate Research Center, and Children's Healthcare of Atlanta have uncovered a critical detail explaining how HIV assembles its infectious yet stealthy clothing.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.