Clinical trials often fall short

May 1, 2012

Clinical studies registered in between 2007-2010 are dominated by small, single-center trials and contain significant heterogeneity (different in nature, difficult to compare) in methodological approaches, including the use of randomization, blinding, and data monitoring committees, according to a study in the May 2 issue of JAMA.

"Clinical trials are the central means by which preventive, diagnostic, and are evaluated, but the U.S. clinical trials enterprise has been marked by debate regarding funding priorities for clinical research, the design and interpretation of studies, and protections for research participants," according to background information in the article. In 1997, Congress mandated the creation of the registry to assist people with serious illnesses in gaining access to trials; and in September 2004, the International Committee of Medical Journal Editors (ICMJE) announced a policy, which took effect in 2005, of requiring registration of clinical trials as a prerequisite for publication.

"Recent work highlights the inadequate evidence base of current practice, in which less than 15 percent of major guideline recommendations are based on high-quality evidence, often defined as evidence that emanates from trials with appropriate designs; sufficiently large sample sizes; and appropriate, validated outcome measures, as well as oversight by institutional review boards and data monitoring committees (DMCs) to protect participants and ensure the trial's integrity," the authors write.

Robert M. Califf, M.D., of the Duke Translational Medicine Institute, Durham, N.C., and colleagues conducted a study to examine fundamental characteristics of interventional clinical trials registered in the database, focusing on study characteristics (data elements reported in trial registration) that are desirable for generating reliable evidence from clinical trials. A data set comprising 96,346 clinical studies from was downloaded on September 27, 2010, and entered into a relational database to analyze aggregate data. Interventional trials were identified and analyses were focused on 3 clinical specialties—cardiovascular, mental health, and oncology—that together encompass the largest number of disability-adjusted life-years lost in the United States. The researchers analyzed the characteristics of registered clinical trials as reported data elements in the trial registry; how those characteristics have changed over time; differences in characteristics as a function of clinical specialty; and factors associated with use of randomization, blinding, and DMCs.

Analysis of the data indicated that the number of trials submitted for registration increased from 28,881 (October 2004 – September 2007) to 40,970 (October 2007 – September 2010). While the database was filled in more completely in the latter time period, many trials reported that they are not using data monitoring committees in the latest time period (59.4 percent). The majority of clinical trials were small in terms of numbers of participants. Overall, 96 percent of these trials had an anticipated enrollment of 1,000 or fewer participants and 62 percent had 100 or fewer participants. The median (midpoint) number of participants per trial was 58 for completed trials and 70 for trials that have been registered but not completed.

Data on funding source and number of sites were available for 37,520 of 40,970 clinical trials registered during the 2007-2010 period. The largest proportion of these trials were not funded by industry or the National Institutes of Health (NIH) (47 percent, n = 17,592) with 16,674 (44 percent) funded by industry, 3,254 (9 percent) funded by the NIH, and 757 (2.0 percent) funded by other U.S. federal agencies. The majority of trials were single site (66 percent); 34 percent were multisite trials.

"Heterogeneity in the reported methods by clinical specialty; sponsor type; and the reported use of DMCs, randomization, and blinding was evident," the authors write. "For example, reported use of DMCs was less common in industry-sponsored vs. NIH-sponsored trials, earlier-phase vs. phase 3 trials, and mental health trials vs. those in the other 2 specialties. In similar comparisons, randomization and blinding were less frequently reported in earlier-phase, oncology, and device trials."

The authors note that the finding of substantial differences in the use of randomization and blinding across specialties raises "fundamental questions about the ability to draw reliable inferences from clinical research conducted in that arena."

The researchers add that the fact that 50 percent of interventional studies registered from October 2007 to September 2010 by design include fewer than 70 participants may have important policy implications. "Small trials may be appropriate in many cases. … However, small trials are unlikely to be informative in many other settings, such as establishing the effectiveness of treatments with modest effects and comparing effective treatments to enable better decisions in practice."

"Our analysis raises questions about the best methods for generating evidence, as well as the capacity of the enterprise to supply sufficient amounts of high-quality evidence needed to ensure confidence in guideline recommendations. Given the deficit in evidence to support key decisions in clinical practice guidelines as well as concerns about insufficient numbers of volunteers for trials, the desire to provide high-quality evidence for medical decisions must include consideration of a comprehensive redesign of the clinical trial enterprise."

Explore further: Many NIH-funded clinical trials go unpublished over two years after completion

More information: JAMA. 2012;307[17]:1838-1847

Related Stories

Recommended for you

The ethics of tracking athletes' biometric data

January 18, 2017

(Medical Xpress)—Whether it is a FitBit or a heart rate monitor, biometric technologies have become household devices. Professional sports leagues use some of the most technologically advanced biodata tracking systems to ...

Study shows blood products unaffected by drone trips

December 7, 2016

In what is believed to be the first proof-of-concept study of its kind, Johns Hopkins researchers have determined that large bags of blood products, such as those transfused into patients every day, can maintain temperature ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.