Skewed results? Failure to account for clinical trial drop-outs can lead to erroneous findings in top medical journals

June 14, 2012, University at Buffalo

(Medical Xpress) -- A new University at Buffalo study of publications in the world's top five general medical journals finds that when clinical trials do not account for participants who dropped out, results are biased and may even lead to incorrect conclusions.

Published recently in the British Medical Journal, the methodological study consisted of a of 235 published in the world's top five general medical journals between 2005 and 2007 that claimed a statistically significant effect.

"We found that in up to a third of trials, the results that were reported as positive -- in other words, statistically significant -- would become negative -- not statistically significant, if the investigators had appropriately taken into consideration those participants who were lost to follow-up," says Elie A. Akl, MD, MPH, PhD, lead author, and associate professor of medicine, family medicine and social and at the University at Buffalo School of Medicine and and School of Public Health and Health Professions. He also has an appointment at McMaster University.

"In other words, one of three claims of effectiveness of interventions made in top general might be wrong," he says.

In one example, a study that compared two surgical techniques for treating found that one was superior. But in the analysis published this month, it was found that 21 percent of participants were lost to follow-up. "When we reanalyzed that study by taking into account those drop-outs, we found that the trial might have overestimated the superiority of one procedure over the other," Akl says.

According to Akl, it has always been suspected, but never proven, that loss to follow-up introduces bias into the results of clinical trials. "The methodology we developed allowed us to provide that proof," he says.

The methodology that he and his coauthors developed consists of sensitivity analyses, a statistical approach to test the robustness of the results of an analysis in the face of specific assumptions, in this case, assumptions about the outcomes of patients lost to follow-up.

"This study gives us a better understanding of the problem of loss to follow-up in clinical trials and provides us with better tools to address it," Akl says.

"This methodology will allow those who conduct the trials and those who use their results, including clinicians, other scientists, developers of clinical guidelines, policymakers and bodies like the Food and Drug Administration, to better judge the risk of bias," concludes Akl.

The studies that were analyzed had previously been published in Annals of Internal Medicine, British Medical Journal, the Journal of the American Medical Association, Lancet and the New England Journal of Medicine. To be included, the trials that were studied had to have reported a significant effect.

Akl led this major study, funded by Pfizer, which took three years to complete. His co-authors, 20 clinical epidemiologists, are from the following institutions: McMaster University; University Hospital Basel; Kaiser Permanente Northwest; Hospital for Sick Children in Toronto; Institute for Work and Health, Universitè de Sherbrooke; University Children's Hospital Tuebingen; Pontificia Universidad Catolica de Chile; Tel Aviv University; the University of Ottawa; the University of Freiburg and the University of Oxford.

Explore further: Behavior of people faced with health-care choices is not influenced by 'framing effect,' study finds

Related Stories

Behavior of people faced with health-care choices is not influenced by 'framing effect,' study finds

December 7, 2011
The behavior of consumers who are faced with making decisions about their health is not significantly influenced by the way health messages are worded or framed, according to a large, new study by researchers at the University ...

Should low molecular weight heparin be used in cancer treatment?

February 15, 2012
For decades, the blood thinner heparin has been used to prevent and treat blood clots. Could it be just as effective in treating cancer?

Recommended for you

Best of Last Year—The top Medical Xpress articles of 2017

December 20, 2017
It was a good year for medical research as a team at the German center for Neurodegenerative Diseases, Magdeburg, found that dancing can reverse the signs of aging in the brain. Any exercise helps, the team found, but dancing ...

Pickled in 'cognac', Chopin's heart gives up its secrets

November 26, 2017
The heart of Frederic Chopin, among the world's most cherished musical virtuosos, may finally have given up the cause of his untimely death.

Sugar industry withheld evidence of sucrose's health effects nearly 50 years ago

November 21, 2017
A U.S. sugar industry trade group appears to have pulled the plug on a study that was producing animal evidence linking sucrose to disease nearly 50 years ago, researchers argue in a paper publishing on November 21 in the ...

Female researchers pay more attention to sex and gender in medicine

November 7, 2017
When women participate in a medical research paper, that research is more likely to take into account the differences between the way men and women react to diseases and treatments, according to a new study by Stanford researchers.

Drug therapy from lethal bacteria could reduce kidney transplant rejection

August 3, 2017
An experimental treatment derived from a potentially deadly microorganism may provide lifesaving help for kidney transplant patients, according to an international study led by investigators at Cedars-Sinai.

Exploring the potential of human echolocation

June 25, 2017
People who are visually impaired will often use a cane to feel out their surroundings. With training and practice, people can learn to use the pitch, loudness and timbre of echoes from the cane or other sounds to navigate ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.