Comparison of evidence of treatment effects in randomized and nonrandomized studies on allergen immunotherapy

Abstract

Nonrandomized studies (NRS) on allergen immunotherapy (AIT) particularly lend themselves to evaluate outcomes that are insufficiently addressed in randomized controlled studies (RCTs). However, NRS are prone to several sources of bias, which limit their validity. We aimed at comparing AIT effects between RCTs and NRS and evaluate the reasons for discrepancies in study results. In this analysis, we compared NRS on AIT (including subcutaneous and sublingual immunotherapy, SCIT and SLIT, respectively) with SLIT and SCIT RCTs from published meta‐analyses, assessing the Risk of Bias (RoB) for each study and the certainty of evidence from NRS and RCTs using the GRADE approach. We found: (1) very serious RoB in the 7 NRS included in the meta‐analysis showing a large difference between AIT and controls (standardized mean difference [SMD] for symptom score [SS], −1.77; 95% CI, −2.30, −1.24; p < .001; I2 = 95%) with very low certainty evidence; (2) serious RoB in the 13 SCIT‐RCTs reporting a moderate‐to‐high difference between SCIT and controls (SMD for SS, −0.81; 95% CI, −1.12, −0.49; p < .001; I2 = 88%) with moderate certainty evidence; (3) low RoB in the 13 SLIT‐RCTs reporting a small benefit (SMD for SS, −0.28; 95% CI, −0.37, −0.19; p < .001; I2 = 54.2%) with high certainty evidence. Similar results were reported for medication score. Our evidence is sufficient to conclude that the magnitude of effect estimates of NRS and RCTs directly correlate with the degree of RoB and inversely with the overall evidence certainty. NRS, which are more affected than RCTs by bias resulting in low certainty evidence, showed the largest effect size. Sound NRS are needed to complement RCTs.

Local Tumour Control Following Microwave Ablation: Protocol for the Prospective Observational CIEMAR Study

“…While the level of evidence of observational data is considered low compared to randomised controlled trials (RCTs), several comparative studies have shown that outcomes of RCTs and observational studies are comparable in terms of treatment effect and quality [ 22 – 24 ]. Nevertheless, a recent meta-analysis showed that the risk of bias was directly correlated with the effect estimate [ 25 ]. Therefore, the choice of a robust primary outcome measure, inclusion criteria, and prospective data collection are essential in reducing the risk of bias [ 26 – 28 ].…”

Mechanism and clinical evidence of immunotherapy in allergic rhinitis

“…However, many meta-analysis and reviews outlined that major limitation in evaluating the clinical outcome of AIT are the broad diversity in composition of products, the study designs, the heterogeneity of the environmental conditions, the age of the population involved and the use of different primary and secondary study endpoints ( 88 , 94 , 95 ). Moreover, different outcomes have been reported between SLIT and SCIT, as well as between SLIT delivered by tablets or by drops ( 88 ).…”

Clinical outcomes of AIT in the elderly population

“…Allergen immunotherapy (AIT) remains one of the most important treatment methodologies in contemporary allergology [1 & ,2,3]. Its efficacy in treating allergic rhinitis, some forms of asthma, and allergies to Hymenoptera venoms and selected foods has been confirmed in many clinical randomized and nonrandomized trials, reports, and meta-analyses, [4][5][6][7][8][9][10][11]. Qualification criteria, contraindications, routes of administration, and the assessment of safety and efficacy depending on the desensitized allergens are included in many international consensuses, most importantly, in the European Academy of Allergy and Clinical Immunology (EAACI) recommendations [6,10,12,13].…”