Clinical trials cited for ignoring previous relevant research

January 21, 2011

The vast majority of already published and relevant clinical trials of a given drug, device or procedure are routinely ignored by scientists conducting new research on the same topic, a new Johns Hopkins study suggests.

The vast majority of already published and relevant clinical trials of a given drug, device or procedure are routinely ignored by scientists conducting new research on the same topic, a new Johns Hopkins study suggests.

The authors of the findings, reported in the Jan. 4 issue of , argue that these omissions potentially skew scientific results, waste taxpayer money on redundant studies and involve patients in unnecessary research.

Conducting an analysis of published studies, the Johns Hopkins team concludes that researchers, on average, cited less than 21 percent of previously published, relevant studies in their papers. For papers with at least five prior publications available for , one-quarter cited only one previous trial, while another quarter cited no other previous trials on the topic. Those statistics stayed roughly the same even as the number of papers available for citation increased. Larger studies were no more likely to be cited than smaller ones.

“The extent of the discrepancy between the existing evidence and what was cited is pretty large and pretty striking,” says Karen Robinson, Ph.D., an assistant professor of medicine at the Johns Hopkins University School of Medicine and co-author of the research with Steven N. Goodman, M.D., M.H.S., Ph.D., a Hopkins epidemiology and biostatistics professor. “It’s like listening to one witness as opposed to the other 12 witnesses in a criminal trial and making a decision without all the evidence. Clinical trials should not be started — and cannot be interpreted — without a full accounting of the existing evidence.”

Robinson and Goodman searched the Web of Science, an Internet archive, for meta-analyses done in 2004 on groups of randomized, controlled trials on such common topics as a cancer treatment or a heart procedure. A meta-analysis is a systematic procedure for statistically combining the results of many different studies on a similar topic to determine the effectiveness of medical interventions.

The researchers ultimately looked at 227 meta-analyses comprising 1,523 separate clinical trials in 19 different fields, including oncology, neurology and pediatrics.

Of 1,101 peer-reviewed publications for which there had been at least five previous relevant papers, 46 percent acknowledged the existence of no more than one previous trial.

“Accurate representation of the prior cumulative evidence is necessary to both ethically justify a trial and to make proper inferences,” they write. Studying prior research can also lead to study designs that are more likely to fill gaps in the evidence, the team said, noting that although the presence of a citation “does not tell us how information from that trial was used, the absence of a citation almost guarantees that it was not.”

The Hopkins researchers could not say why prior trials failed to be cited or whether non-cited trials may have been taken into account in the trial design and proposal stages, such as grant requests to the National Institutes of Health.

At the very least, Robinson says, researchers often contend that their publications are so “unique” that there are no relevant studies to cite, even though someone else may have included it in a meta-analysis of like research. Others claim there just isn’t room to cite past relevant research, but Robinson says one reason for the omissions could be the self-interest of researchers trying to get ahead.

“To get published, journals are looking for novelty, uniqueness,” she says. Leaving out references to prior similar research can make findings seem more like a breakthrough, she adds. In her publications study, Robinson found several papers that claimed to be the first even when many trials on the subject preceded them.

There are no barriers to funding, conducting or publishing a clinical trial without proof that prior literature had been adequately searched and evaluated, she says. But requirements such as those have been instituted by some European funding agencies, the medical journal The Lancet, and the U.S. Centers for Medicare & Medicaid Services, which require that a covered trial not “unjustifiably duplicate existing studies,” Robinson writes.

Robinson says funders, institutional review boards and journals need to take steps to ensure that prior research is considered. To do otherwise, she says, encourages this “unethical” behavior to continue.

“Trials being done may not be justified, because researchers are not looking at or at least not reporting what is already known,” she says. “We may be wasting resources when we fund trials for which we already know the answer. And we may be coming to incorrect conclusions about what works in medicine.”

In some cases, patients who volunteer for may be getting a placebo for a medication that a previous researcher has already determined works or may be getting a treatment that another researcher has shown is of no value. In rare instances, patients have suffered severe side effects and even died in studies because researchers were not aware of previous studies documenting a treatment’s dangers.

Related Stories

Recommended for you

Pickled in 'cognac', Chopin's heart gives up its secrets

November 26, 2017
The heart of Frederic Chopin, among the world's most cherished musical virtuosos, may finally have given up the cause of his untimely death.

Sugar industry withheld evidence of sucrose's health effects nearly 50 years ago

November 21, 2017
A U.S. sugar industry trade group appears to have pulled the plug on a study that was producing animal evidence linking sucrose to disease nearly 50 years ago, researchers argue in a paper publishing on November 21 in the ...

Female researchers pay more attention to sex and gender in medicine

November 7, 2017
When women participate in a medical research paper, that research is more likely to take into account the differences between the way men and women react to diseases and treatments, according to a new study by Stanford researchers.

Drug therapy from lethal bacteria could reduce kidney transplant rejection

August 3, 2017
An experimental treatment derived from a potentially deadly microorganism may provide lifesaving help for kidney transplant patients, according to an international study led by investigators at Cedars-Sinai.

Exploring the potential of human echolocation

June 25, 2017
People who are visually impaired will often use a cane to feel out their surroundings. With training and practice, people can learn to use the pitch, loudness and timbre of echoes from the cane or other sounds to navigate ...

Team eradicates hepatitis C in 10 patients following lifesaving transplants from infected donors

April 30, 2017
Ten patients at Penn Medicine have been cured of the Hepatitis C virus (HCV) following lifesaving kidney transplants from deceased donors who were infected with the disease. The findings point to new strategies for increasing ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Jan 21, 2011

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.