Paper proposes guidelines to make systematic reviews impacting medical practice more trustworthy

The goal of a Guidance article in the Journal of Pediatric Rehabilitation Medicine is to improve how systematic reviews impacting medical practice are conducted, reported, and evaluated. Systematic review authors, and editors who publish these reviews, are encouraged to follow established guidelines and safeguards to ensure the trustworthiness of medical evidence syntheses.

Ideally, a systematic review should review and summarize empirical data from research about different clinical treatment options and possible side effects. They form the basis for clinical practice guidelines that drive specific recommendations for patient care, however, data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative.

Anyone can publish a systematic review on any topic, whether qualified to do so or not. In addition, journals do not follow uniform standards for publication of systematic reviews. Also, funding bodies might seek or commission a systematic review but not have the purest of intentions. They may exert influence on systematic review authors, and as a result, data the authors of the review select to summarize may support those desired results with little regard for how it could impact patient care.

Kat Kolaski, MD, Wake Forest University School of Medicine, Lynne Romeiser Logan, PT, Ph.D., SUNY Upstate Medical University, and world-renowned scientist John Ioannidis MD, Ph.D., Stanford University School of Medicine, share a concern about the high numbers of untrustworthy systematic reviews published in the medical literature and decided to do something about it.

"As editors and peer reviewers for various journals, we find the ongoing poor compliance of authors and journal editors with review expectations discouraging. We sought to identify practical solutions but discovered that, while guidelines for sound conduct and reporting of systematic reviews are available from many sources, they do not appear to be routinely and/or widely applied. The more we searched for comprehensive guidance in a single place, the more we realized we might have to create it," the authors explained.

The authors describe reliable methods and research practices complemented by novel pragmatic strategies to improve evidence syntheses. In addition, they organize best practice resources into a concise guide so that they can be widely adopted and adapted for routine implementation by authors and journals.

"We look forward to others' ideas and proposals for the advancement of methods for evidence syntheses. For now, we encourage dissemination and uptake of the currently accepted best tools and practices for their development and evaluation; at the same time, we stress that uptake of appraisal tools, checklists, and software programs cannot substitute for proper education in the methodology of evidence syntheses and meta-analyses," they write.

The guidance paper by Kolaski, Logan and Ioannidis was published simultaneously in seven medical journals worldwide in a unique effort to impact a wide audience.

"We at JPRM are honored to participate as one of seven prominent medical journals copublishing these critically important systematic review guidelines. These will surely have a profound impact on the reliability of published medical literature worldwide," commented Editor-in-Chief Elaine Pico, MD, UCSF Benioff Children's Hospital Oakland.