Expert panel diagnosis for diagnostic test poorly described, experts not blinded to test under study

Evaluation of diagnostic studies is often a challenge in diseases that are not defined by a specific test. Assessment of the accuracy of diagnostic tests is essential because they may be used to define who is considered to have a disease and receive treatment for it. However, measuring the accuracy of a diagnostic test requires an accurate gold standard, which defines which patients truly have and do not have the disease.

Studies of diseases not defined by a specific often rely on expert panels to establish the gold standard. In a and analysis of the diagnostic literature using expert panels to define the gold standard for a given , Loes Bertens and colleagues from University Medical Center Utrecht determined how expert panels were used in such studies and how well their process was described and reliability assessed.

The authors evaluated 81 diagnostic studies published up to May 31, 2012, including studies of diagnostic tests for (30 of 81 papers, 37%), half of which pertained to dementia, cardiovascular diseases (17 papers, 21%), and respiratory disorders (10 papers, 12%). They found that reporting was often incomplete, with 83% of studies missing at least some important information about the . In 75% of studies the panel consisted of three or fewer members, and panel members were blinded to the results of the test results being evaluated in only 31% of studies. Blinding is important because knowledge of the index text results could influence the panelists' decision as to whether the patient had the disease. Reproducibility of the decision process was assessed in only 21% of studies.

The authors state, "Complete and accurate reporting is a prerequisite for judging potential bias in a study and for allowing readers to apply the same study methods. In total, only 14 (17%) papers reported complete data on key issues such as the panel constitution, the information presented to the panel and the exact decision process to determine the final diagnosis." They also found that despite publication of reporting guidelines, the completeness of reporting did not improve over time, perhaps because the reporting guidelines do not include specific criteria for expert panel diagnoses. The authors make a number of recommendations to improve reporting of expert panel diagnosis. They conclude, "Our review revealed a large variation in applied methods as well as major deficiencies in the reporting of key features of the panel diagnosis process…The results of our review may serve as a starting point in the development of formal guidelines on methodology and reporting of panel diagnosis."

Explore further

Conflicts of interest common among panel members of guidelines that expand disease definitions

More information: Bertens LCM, Broekhuizen BDL, Naaktgeboren CA, Rutten FH, Hoes AW, et al. (2013) Use of Expert Panels to Define the Reference Standard in Diagnostic Research: A Systematic Review of Published Methods and Reporting. PLoS Med 10(10): e1001531. DOI: 10.1371/journal.pmed.1001531
Citation: Expert panel diagnosis for diagnostic test poorly described, experts not blinded to test under study (2013, October 15) retrieved 16 October 2019 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more