How reliable is prognostic research? A case study of C-reactive protein in coronary artery disease
Prognostic markers provide tools for discriminating between groups of patients who are at different risks of a particular outcome, and therefore should help clinicians to manage disease. In a comprehensive overview of studies looking at one such proposed marker, C-reactive protein (CRP) in coronary artery disease, Harry Hemingway and colleagues, from University College London, show that despite the inclusion of many tens of thousands of patients in research on this specific question, the published record is so inadequate that no clear clinical recommendations can be made.
In the study, published in PLoS Medicine, the authors carried out a detailed systematic review to identify all prospective studies reporting risk of a coronary, cardiovascular, or mortality outcome among patients in whom the CRP values had been measured. They identified 83 studies in the published scientific literature, in which data was reported for 61,684 patients, and 6,485 outcomes. A straightforward analysis of the results of these studies suggested overall evidence for a fairly strong increase in risk associated with higher CRP levels. However, the authors found strong evidence of publication bias. This occurred owing to non-publication of studies with contradictory findings to those which did get published. In an attempt to adjust for publication bias, the researchers estimated that CRP would have a much smaller strength of association with coronary or death outcomes. Finally, the authors also identified a low standard of repor ting quality for many of the published studies, and multiple types of reporting bias in the literature.
Given these biases, the authors conclude that the data "preclude firm conclusions about the magnitude and independence of the association between higher CRP levels and higher risk of subsequent death and nonfatal cardiovascular events". They propose that improvements are needed in the design and reporting of this type of research. Critically, in reporting of these studies it is unclear whether analyses are being included in a publication because the direction of the result suggests something interesting, or because the original study plan specified it; a proposal included in the paper is that study protocols for prognostic research should be registered in advance.