The results were inconclusive in the clinical management of root resorption, but there is evidence to support the use of light forces, especially with incisor intrusion.

We examine the use of fixed-effects and random-effects moment-based meta-analytic methods for analysis of binary adverse event data. Special attention is paid to the case of rare adverse events which are commonly encountered in routine practice. We study estimation of model parameters and between-study heterogeneity. In addition, we examine traditional approaches to hypothesis testing of the average treatment effect and detection of the heterogeneity of treatment effect across studies. We derive three new methods, simple (unweighted) average treatment effect estimator, a new heterogeneity estimator, and a parametric bootstrapping test for heterogeneity. We then study the statistical properties of both the traditional and new methods via simulation. We find that in general, moment-based estimators of combined treatment effects and heterogeneity are biased and the degree of bias is proportional to the rarity of the event under study. The new methods eliminate much, but not all of this bias. The various estimators and hypothesis testing methods are then compared and contrasted using an example dataset on treatment of stable coronary artery disease.

No abstract

Randomized controlled trials (RCTs) are the traditional gold standard evidence for medical decision-making. However, protocols that limit enrollment eligibility introduce selection error that severely limits a RCT's applicability to a wide range of patients. Conversely, high quality observational data can be representative of entire populations, but freedom to choose treatment can bias estimators based on this data. Cross design synthesis (CDS) is an approach to combining both RCT and observational data in a single analysis that capitalizes on the RCT's strong internal validity and the observational study's strong external validity. We proposed and assessed a simple estimator of effect size based on the CDS approach. We evaluated its properties within a formal framework of causal estimation and compared our estimator with more traditional estimators based on single sources of evidence. We show that under ideal conditions the simple CDS estimator is unbiased whenever the observational data-based estimators' treatment selection error is constant across those who are and are not eligible for RCT participation. Whereas this assumption may not often hold in practice, assumptions required for the unbiasedness of usual single-source estimators may also be implausible. We show that, under some reasonable data assumptions, our simple CDS estimator has smaller bias and better coverage than commonly used estimates based on randomized or observational studies alone.

For the results of randomized controlled clinical trials (RCTs) and related meta-analyses to be useful in practice, they must be relevant to a definable group of patients in a particular clinical setting. To the extent this is so, we say that the trial is generalizable or externally valid. Although concern about the generalizability of the results of RCTs is often discussed, there are few examples of methods for assessing the generalizability of clinical trial data. In this paper, we describe and illustrate an approach for making what we call generalizability judgments and illustrate the approach in the context of a case study of the risk of suicidality among pediatric antidepressant users.

The results provide some evidence of a positive benefit from cognitive interventions, but cannot be regarded as robust given the overall very low quality of the evidence.

While randomized controlled trials (RCT) are considered the “gold standard” for clinical studies, the use of exclusion criteria may impact the external validity of the results. It is unknown whether estimators of effect size are biased by excluding a portion of the target population from enrollment. We propose to use observational data to estimate the bias due to enrollment restrictions, which we term generalizability bias. In this paper we introduce a class of estimators for the generalizability bias and use simulation to study its properties in the presence of non-constant treatment effects. We find the surprising result that our estimators can be unbiased for the true generalizability bias even when all potentially confounding variables are not measured. In addition, our proposed doubly robust estimator performs well even for mis-specified models.

Because of model specification and interpretation issues raised in this paper, we conclude that the evidence supporting a causal link between antidepressant use and suicidality in children is weak. The use of Bayesian hierarchical models for meta-analysis has facilitated the incorporation of potentially important sources of variability and the use of sensitivity analysis to assess the consequences of model specifications and their impact on important regulatory decisions.