Poor research practice suggests true impact of homeopathy may be 'substantially' overestimated

Poor research practice suggests that the true impact of homeopathy may be substantially overestimated, finds an analysis of the current body of evidence on the effectiveness of this type of complementary medicine, published online in BMJ Evidence Based Medicine.

Many clinical trials haven't been registered, with the main outcome changed in a quarter of those that have been. And many remain unpublished. All this indicates "a concerning lack of scientific and ethical standards in the field of homeopathy and a high risk for reporting bias," say the researchers.

Homeopathy was developed almost 200 years ago, based on the principle of similarity ('like cures like'). It remains a popular alternative to conventional medicine in many developed countries, despite its effectiveness being the subject of fierce debate.

The study authors wanted to find out if the published clinical trials might not represent all the scientific studies on homeopathy, but a select few reporting only positive results–a phenomenon known as 'reporting bias'.

Public clinical trial registries were set up to try and reduce this risk, and since 2008, registration and publication of clinical trial results have been regarded as an ethical, although not mandatory, obligation for researchers.

The study authors therefore set out to: find out how many registered trials assessing homeopathy remain unpublished; whether the primary outcomes of registered trials reflect those actually published; as well as the number of homeopathy trials that had been both registered and published.

They also wanted to assess the impact of any reporting bias on the pooled data analysis of homeopathy trial results, a research method designed to strengthen the evidence base.

They searched major international registries for clinical trials registered up to April 2019, and research databases to track publication of these trials up to April 2021.

They found that since 2002, nearly 38% of registered homeopathy trials remain unpublished, while over half (53%) of published randomized controlled trials haven't been registered. In all, nearly a third (30%) of randomized controlled trials published during the past 5 years haven't been registered.

They also found that homeopathy trials were more likely to be registered after they had started (retrospectively) than before they had started (prospective registration). What's more, a quarter (25%) of published primary outcomes weren't the same as those originally registered.

The study authors then assessed the potential impact on clinical practice by separately pooling the data from unregistered and registered homeopathy trials. This revealed that unregistered trials tended to report larger treatment effects.

The study authors accept that their searches covered 17 trial registries, so it's highly likely that they missed records not covered by these registries. And they pooled the data from homeopathic treatments that weren't tailored to individual needs, so the findings might not be applicable to personalized treatment.

Nevertheless, the findings "suggest a concerning lack of scientific and ethical standards in the field of homeopathy and a high risk for reporting bias," they write.

And they "also indicate that journals publishing homeopathy trials do not adhere to policies by the [International Committee of Medical Journal Editors], which demand that only registered [randomized controlled trials] should be published," they add.

The poor research practice they found "likely affects the validity of the body of evidence of homeopathic literature and may substantially overestimate the true treatment effect of homeopathic remedies," they conclude.