Exact inference for the random‐effect model for meta‐analyses with rare events

Abstract

Meta-analysis aggregates information across related studies to provide more reliable statistical inference and has been a vital tool for assessing the safety and efficacy of many high profile pharmaceutical products. A key challenge in conducting a metaanalysis is that the number of related studies is typically small. Applying classical methods that are asymptotic in the number of studies can compromise the validity of inference, particularly when heterogeneity across studies is present. Moreover, serious adverse events are often rare and can result in one or more studies with no events in at least one study arm. While it is common to use arbitrary continuity corrections or remove zero-event studies to stabilize or define effect estimates in such settings, these practices can invalidate subsequent inference. To address these significant practical issues, we introduce an exact inference method for comparing event rates in two treatment arms under a random effects framework, which we coin "XRRmeta". In contrast to existing methods, the coverage of the confidence interval from XRRmeta is guaranteed to be at or above the nominal level (up to Monte Carlo error) when the event rates, number of studies, and/or the within-study sample sizes are small. XRRmeta is also justified in its treatment of zero-event studies through a conditional inference argument. Importantly, our extensive numerical studies indicate that XR-Rmeta does not yield overly conservative inference. We apply our proposed method to reanalyze the occurrence of major adverse cardiovascular events among type II diabetics treated with rosiglitazone and in a more recent example examining the utility of face masks in preventing person-to-person transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and coronavirus disease 2019 (COVID-19) 1,2 . We also introduce an R package implementing XRRmeta to encourage use by practitioners.

Comments on “Exact inference for the random‐effect model for meta‐analyses with rare events”

“…The length of the exact interval is 0.339, which is wider than the simulation based exact interval proposed by Gronsbell et al, 1 0.295 [the 95% interval (0.072,0.367)]. It can be seen that both the exact lower limit and the exact upper limit are larger than those by Gronsbell et al 1 When sample sizes are medium to large and the number of studies is large, it becomes computationally challenge due to big data. The size of the complete sample space is too big for a stand-alone computer.…”

“…3,4 To overcome the computational intensity, they showed through numerical studies that the maximum of the profile p value occurs at the boundary of for a given . 1 The search space is reduced to a line instead of a two-dimensional space, and then it is computationally easier to calculate exact confidence intervals. 5,6 The acute myeloid leukemia (AML) example was used to illustrate the application of their proposed exact confidence intervals based on simulations.…”

“…5,6 The acute myeloid leukemia (AML) example was used to illustrate the application of their proposed exact confidence intervals based on simulations. 1 Both the number of studies and sample sizes in each study are small. Their Table 3 presented rates of veno-occlusive disease for patients treated by Mylotarg, a drug for patients with CD33-positive AML.…”

“…Gronsbell et al developed a new simulation‐based confidence interval for the location parameter of the beta‐binominal model in meta‐analysis with rare events. They made several important contributions to the literature of meta‐analysis.…”

Response to the letter by Guogen Shan, Hua Zhang, and Tao Jiang

“…The resulting confidence interval (CI) is (0.106, 0.445). This CI is wider than the interval from our proposed simulation‐based approach, (0.072, 0.367) 6 …”