Clinical trials often fall short

May 1, 2012, JAMA and Archives Journals

Clinical studies registered in between 2007-2010 are dominated by small, single-center trials and contain significant heterogeneity (different in nature, difficult to compare) in methodological approaches, including the use of randomization, blinding, and data monitoring committees, according to a study in the May 2 issue of JAMA.

"Clinical trials are the central means by which preventive, diagnostic, and are evaluated, but the U.S. clinical trials enterprise has been marked by debate regarding funding priorities for clinical research, the design and interpretation of studies, and protections for research participants," according to background information in the article. In 1997, Congress mandated the creation of the registry to assist people with serious illnesses in gaining access to trials; and in September 2004, the International Committee of Medical Journal Editors (ICMJE) announced a policy, which took effect in 2005, of requiring registration of clinical trials as a prerequisite for publication.

"Recent work highlights the inadequate evidence base of current practice, in which less than 15 percent of major guideline recommendations are based on high-quality evidence, often defined as evidence that emanates from trials with appropriate designs; sufficiently large sample sizes; and appropriate, validated outcome measures, as well as oversight by institutional review boards and data monitoring committees (DMCs) to protect participants and ensure the trial's integrity," the authors write.

Robert M. Califf, M.D., of the Duke Translational Medicine Institute, Durham, N.C., and colleagues conducted a study to examine fundamental characteristics of interventional clinical trials registered in the database, focusing on study characteristics (data elements reported in trial registration) that are desirable for generating reliable evidence from clinical trials. A data set comprising 96,346 clinical studies from was downloaded on September 27, 2010, and entered into a relational database to analyze aggregate data. Interventional trials were identified and analyses were focused on 3 clinical specialties—cardiovascular, mental health, and oncology—that together encompass the largest number of disability-adjusted life-years lost in the United States. The researchers analyzed the characteristics of registered clinical trials as reported data elements in the trial registry; how those characteristics have changed over time; differences in characteristics as a function of clinical specialty; and factors associated with use of randomization, blinding, and DMCs.

Analysis of the data indicated that the number of trials submitted for registration increased from 28,881 (October 2004 – September 2007) to 40,970 (October 2007 – September 2010). While the database was filled in more completely in the latter time period, many trials reported that they are not using data monitoring committees in the latest time period (59.4 percent). The majority of clinical trials were small in terms of numbers of participants. Overall, 96 percent of these trials had an anticipated enrollment of 1,000 or fewer participants and 62 percent had 100 or fewer participants. The median (midpoint) number of participants per trial was 58 for completed trials and 70 for trials that have been registered but not completed.

Data on funding source and number of sites were available for 37,520 of 40,970 clinical trials registered during the 2007-2010 period. The largest proportion of these trials were not funded by industry or the National Institutes of Health (NIH) (47 percent, n = 17,592) with 16,674 (44 percent) funded by industry, 3,254 (9 percent) funded by the NIH, and 757 (2.0 percent) funded by other U.S. federal agencies. The majority of trials were single site (66 percent); 34 percent were multisite trials.

"Heterogeneity in the reported methods by clinical specialty; sponsor type; and the reported use of DMCs, randomization, and blinding was evident," the authors write. "For example, reported use of DMCs was less common in industry-sponsored vs. NIH-sponsored trials, earlier-phase vs. phase 3 trials, and mental health trials vs. those in the other 2 specialties. In similar comparisons, randomization and blinding were less frequently reported in earlier-phase, oncology, and device trials."

The authors note that the finding of substantial differences in the use of randomization and blinding across specialties raises "fundamental questions about the ability to draw reliable inferences from clinical research conducted in that arena."

The researchers add that the fact that 50 percent of interventional studies registered from October 2007 to September 2010 by design include fewer than 70 participants may have important policy implications. "Small trials may be appropriate in many cases. … However, small trials are unlikely to be informative in many other settings, such as establishing the effectiveness of treatments with modest effects and comparing effective treatments to enable better decisions in practice."

"Our analysis raises questions about the best methods for generating evidence, as well as the capacity of the enterprise to supply sufficient amounts of high-quality evidence needed to ensure confidence in guideline recommendations. Given the deficit in evidence to support key decisions in clinical practice guidelines as well as concerns about insufficient numbers of volunteers for trials, the desire to provide high-quality evidence for medical decisions must include consideration of a comprehensive redesign of the clinical trial enterprise."

Explore further: Many NIH-funded clinical trials go unpublished over two years after completion

More information: JAMA. 2012;307[17]:1838-1847

Related Stories

Many NIH-funded clinical trials go unpublished over two years after completion

January 4, 2012
In a study that investigates the challenges of disseminating clinical research findings in peer-reviewed biomedical journals, Yale School of Medicine researchers have found that fewer than half of a sample of trials primarily ...

Adaptive trial designs could accelerate HIV vaccine development

April 20, 2011
In the past 12 years, four large-scale efficacy trials of HIV vaccines have been conducted in various populations. Results from the most recent trial—the RV144 trial in Thailand, which found a 31 percent reduction ...

Recommended for you

Best of Last Year—The top Medical Xpress articles of 2017

December 20, 2017
It was a good year for medical research as a team at the German center for Neurodegenerative Diseases, Magdeburg, found that dancing can reverse the signs of aging in the brain. Any exercise helps, the team found, but dancing ...

Pickled in 'cognac', Chopin's heart gives up its secrets

November 26, 2017
The heart of Frederic Chopin, among the world's most cherished musical virtuosos, may finally have given up the cause of his untimely death.

Sugar industry withheld evidence of sucrose's health effects nearly 50 years ago

November 21, 2017
A U.S. sugar industry trade group appears to have pulled the plug on a study that was producing animal evidence linking sucrose to disease nearly 50 years ago, researchers argue in a paper publishing on November 21 in the ...

Female researchers pay more attention to sex and gender in medicine

November 7, 2017
When women participate in a medical research paper, that research is more likely to take into account the differences between the way men and women react to diseases and treatments, according to a new study by Stanford researchers.

Drug therapy from lethal bacteria could reduce kidney transplant rejection

August 3, 2017
An experimental treatment derived from a potentially deadly microorganism may provide lifesaving help for kidney transplant patients, according to an international study led by investigators at Cedars-Sinai.

Exploring the potential of human echolocation

June 25, 2017
People who are visually impaired will often use a cane to feel out their surroundings. With training and practice, people can learn to use the pitch, loudness and timbre of echoes from the cane or other sounds to navigate ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.