Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis

Abstract

ObjectivesTo assess the magnitude of reporting bias in trials assessing homeopathic treatments and its impact on evidence syntheses.DesignA cross-sectional study and meta-analysis. Two persons independently searched Clinicaltrials.gov, the EU Clinical Trials Register and the International Clinical Trials Registry Platform up to April 2019 to identify registered homeopathy trials. To determine whether registered trials were published and to detect published but unregistered trials, two persons independently searched PubMed, Allied and Complementary Medicine Database, Embase and Google Scholar up to April 2021. For meta-analyses, we used random effects models to determine the impact of unregistered studies on meta-analytic results.Main outcomes and measuresWe report the proportion of registered but unpublished trials and the proportion of published but unregistered trials. We also assessed whether primary outcomes were consistent between registration and publication. For meta-analyses, we used standardised mean differences (SMDs).ResultsSince 2002, almost 38% of registered homeopathy trials have remained unpublished, and 53% of published randomised controlled trials (RCTs) have not been registered. Retrospective registration was more common than prospective registration. Furthermore, 25% of primary outcomes were altered or changed compared with the registry. Although we could detect a statistically significant trend toward an increase of registrations of homeopathy trials (p=0.001), almost 30% of RCTs published during the past 5 years had not been registered.A meta-analysis stratified by registration status of RCTs revealed substantially larger treatment effects of unregistered RCTs (SMD: −0.53, 95% CI −0.87 to −0.20) than registered RCTs (SMD: −0.14, 95% CI −0.35 to 0.07).ConclusionsRegistration of published trials was infrequent, many registered trials were not published and primary outcomes were often altered or changed. This likely affects the validity of the body of evidence of homeopathic literature and may overestimate the true treatment effect of homeopathic remedies.

Why homoeopathy is pseudoscience

“…4.4 and 4.5, that homoeopaths mischaracterise and cherry-pick the available evidence. Moreover, they do this against the background of an already distorted evidence base because they also selectively report their findings, as Gartlehner et al (2022) found in a recent cross-sectional study and meta-analysis. The authors systematically investigated the extent of reporting bias in trials on homoeopathy.…”

Seven days in medicine: 9-15 March 2022

“…Poor research practice may mean that the true impact of homeopathy is substantially overestimated, said an evidence review published in BMJ Evidence Based Medicine 4. Many clinical trials have not been registered, 38% remain unpublished, and the main outcome was changed in a quarter of those published.…”

Efficacy of homoeopathic treatment: Systematic review of meta-analyses of randomised placebo-controlled homoeopathy trials for any indication

“…However, the between-group difference of effect estimates was not significant (meta-regression: SMD 0.39, 95% CI −0.09 to +0.87). 33 It is not clear why the trial #A93 of the Mathie 2017 MA (Lewith 2002, listed in Gartlehner 2022, Supplement Table 3 as 'not registered') was not included in these analyses.…”

“…The 16 reports consisted in 6 primary publications of one [6][7][8]10,11 or two 9 MA, respectively; 2 published MA protocols 27,28 ; 7 publications of additional analyses 3,[29][30][31][32][33] and 1 error correction 34 (Table 3).…”

“…After literature searches and data collection for this SR had been completed, an additional subgroup analysis of the Mathie 2017 MA was published, which we decided to include, as it concerned an item which had not been analysed for any of the MA: trial registration (Gartlehner 2022). 33 The 54 trials of the Mathie 2017 MA had been published in the period 1976-2014, thereof 20 trials published 2002-2014. Of this group, Gartlehner et al analysed 19 trials, stratified according to clinical trial registration, which had been available at ClinicalTrials.gov since 2000.…”