Treatment rankings derived from network meta-analyses have a substantial degree of imprecision
Treatment rankings derived from network meta-analyses feature a substantial degree of imprecision, according to a new study by researchers at Cochrane France and INSERM U1153 in Paris. More than half of the differences between the best-ranked treatment and the second, third, or fourth best-ranked treatments did not differ from the null beyond chance. The paper "Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials" is published in the Annals of Internal Medicine.
Network meta-analysis is a method of evidence synthesis that allows comparing multiple treatments available for the same disease, even when there are no clinical trials for some comparisons between treatments. Ranking of treatments is one of the most appealing elements of network meta-analysis.
To address the question if the rankings are reliable, the team of researchers has reanalyzed a sample of published network meta-analyses. These network meta-analyses were identified from two previous systematic reviews that involved searches of the Cochrane Library, MEDLINE, and Embase up to July 2012 for articles that included networks of at least 3 treatments. In all, 58 network meta-analyses involving 1308 randomized trials and 404 treatments were reanalyzed.
For each network, the surface under the cumulative ranking curve (SUCRA) and its 95% uncertainty interval was estimated for each treatment. Based on these SUCRA values, the treatments were then rank-ordered between 0% (worst) and 100% (best) in each network.
According to the researchers, the findings suggest a considerable degree of uncertainty. The median width of the 95% uncertainty intervals of the SUCRA was 65%. In about a third of networks, there was a 50% or greater probability that the best-ranked treatment was actually not the best. There was no evidence of difference between the best-ranked intervention and the second and third best-ranked interventions in 90% and 71% of comparisons, respectively. These findings must be balanced with other factors that were not considered, such as the risk of bias within trials or small-study effects, and that may affect the reliability of rankings.
The researchers recommend that readers of network meta-analyses interpret with caution the ranking of treatments, especially if uncertainty is not reported, and that they should be careful in the use of rankings to guide practice.