Decade-long analysis reveals potential bottlenecks in the pathway of clinical research into medical practice

A study has characterized the factors that influence the adoption of clinical research into practice by comparing the citation rates of medical research papers between 18 different specialties, article type, geography and time. The results are published today in eLife.

The work reveals that most clinical trials are never cited by the time they reach the point-of-care—where clinical trials are embedded into regular medical practice. Furthermore, less than one in two Phase III clinical trials are ever cited across 9 out of 18 medical specialties. Phase III clinical trials typically involve a large number of participants and are designed to evaluate the safety and effectiveness of a new medical intervention across the general population.

The results represent a preliminary step to understanding why increased research funding in the U.S. has not yielded equivalent results in health outcomes. The findings could be used to monitor the efficiency of research investments and evaluate health policies influencing the translation of biomedical studies to clinical practice.

The lack of evidence to assess the effectiveness and efficiency of treatments is a contributing factor that underpins the unsustainable expenditure of the U.S. health care system. This is made worse by a limited understanding of the factors that influence the translation of clinical research into practice.

"This may partially explain why, in the U.S. health care systems, many patients do not receive the recommended services and many receive treatment that is not necessary or appropriate for them," says co-author Moustafa Abdalla, resident in the Department of Surgery, at Massachusetts General Hospital, and fellow at Harvard Medical School, both in Boston, Massachusetts, U.S.

The team used the resource tool UpToDate, which provides current evidence-based clinical information at the point of medical practice, to assess what clinical research makes it into practice. By leveraging a dataset of more than 10,000 UpToDate articles, sampled every three months between 2011–2020, they were able to trace the path of research from initial publication to medical practice, compared to the 2.4 million uncited studies published during the same time window across 18 medical specialties.

The analysis revealed substantial variation in how different medical specialties adopt research with regards to frequency of literature citations and the quality-of-evidence incorporated. Across all specialties, only 0.4% to 2.4% of published clinical research is ever cited in UpToDate. Amongst the 18 specialties included in the analysis, neurology had the highest citation rate at 2.4% (2,057 of 85,843 research articles), and pathology had the lowest rate at 0.4% (317 of 69,343 research articles).

The proportion of citations varied substantially by the type of article the research was presented in. Practice guidelines represented the most likely article type to be cited, with 9 of the 18 specialties receiving citations at a rate of over 13%. Clinical trials were the second most likely to be cited, but were also the most variable. In 9 of the 18 specialties, the team observed that less than one in ten Phase III clinical trials were ever cited at the point-of-care.

The team next sought to characterize the effects of National Institutes of Health (NIH) funding on the number of citations a paper receives, primarily focusing on the U.S. They found that, across all specialties in the past decade, average annual NIH funding correlated strongly with the total number of citations. Given this strong correlation, the team then calculated the effective cost per new citation at the point-of-care. This was used to represent the approximate indirect cost of bringing clinical research to medical practice in NIH funding dollars.

In urology—a branch of medicine that focuses on diseases of the urinary track—and nephrology—a specialty of medicine concerning the kidneys—the cost of bringing a new citation to the point-of-care was estimated at $48,086.18 per article (not including the intercept of the model which reflects set up and startup costs). On the other side of the spectrum, in pathology, it cost $874.85 for every additional citation.

While UpToDate represents the largest point-of-care resource, its relevance varies by specialty and training status. Therefore, while the study uses UpToDate as a metric of translation of research, the authors say a citation does not necessarily indicate actual changes in practice. Instead, citations represent adoption of knowledge to support current approaches, inform new changes in practice, or highlight points of controversy.

"While the findings of our study are largely descriptive and explanatory, we have provided a baseline for monitoring the efficiency of research investments. This may eventually lead to the development of strategies to quantify the effectiveness of proposed reforms to the biomedical scientific process," concludes co-author Mohamed Abdalla, Scientist, Institute for Better Health, Trillium Health Partners, Mississauga, Ontario, Canada.