<scp>Meta‐regression</scp> methods to characterize evidence strength using <scp>meaningful‐effect</scp> percentages conditional on study characteristics

Mathur, Maya B.; VanderWeele, Tyler J.

doi:10.1002/jrsm.1504

Cited by 6 publications

(3 citation statements)

References 58 publications

(182 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 63 Meta-regression techniques for the assessment of heterologous within-effect variability can identify sources of variation within an effect estimate from a meta-analysis. 64 , 65 These and other forms of quantitative bias analysis may provide enhanced confidence to committees considering observational data to make and justify policy decisions. Both RCTs and RWE provide complementary sources of data for inclusion within evidence appraisal.…”

Section: Discussionmentioning

confidence: 99%

The economic rationale for cell-based influenza vaccines in children and adults: A review of cost-effectiveness analyses

Fisman,

Giglio,

Levin

et al. 2024

Human Vaccines & Immunotherapeutics

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

The economic rationale for cell-based influenza vaccines in children and adults: A review of cost-effectiveness analyses

Fisman,

Giglio,

Levin

et al. 2024

Human Vaccines & Immunotherapeutics

View full text Add to dashboard Cite

“…When summarizing results across scenarios, we report medians because the metrics were often skewed across scenarios. To help characterize variability in results across scenarios, we also report on each method's performance in the 10% of scenarios in which the method performed the worst 78 . For each respective performance metric, we defined “worse” performance as larger values of

∣ bias ∣

, MAE, RMSE,

∣ coverage - 0.95 ∣

, confidence interval width, and

(1 - convergence)

.…”

Section: Simulation Studymentioning

confidence: 99%

Sensitivity analysis for the interactive effects of internal bias and publication bias in meta‐analyses

Mathur

2023

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

Meta‐analyses can be compromised by studies' internal biases (e.g., confounding in nonrandomized studies) as well as publication bias. These biases often operate nonadditively: publication bias that favors significant, positive results selects indirectly for studies with more internal bias. We propose sensitivity analyses that address two questions: (1) “For a given severity of internal bias across studies and of publication bias, how much could the results change?”; and (2) “For a given severity of publication bias, how severe would internal bias have to be, hypothetically, to attenuate the results to the null or by a given amount?” These methods consider the average internal bias across studies, obviating specifying the bias in each study individually. The analyst can assume that internal bias affects all studies, or alternatively that it only affects a known subset (e.g., nonrandomized studies). The internal bias can be of unknown origin or, for certain types of bias in causal estimates, can be bounded analytically. The analyst can specify the severity of publication bias or, alternatively, consider a “worst‐case” form of publication bias. Robust estimation methods accommodate non‐normal effects, small meta‐analyses, and clustered estimates. As we illustrate by re‐analyzing published meta‐analyses, the methods can provide insights that are not captured by simply considering each bias in turn. An R package implementing the methods is available (multibiasmeta).

show abstract

“…2. Our preregistration specified that we would estimate the percentages of positive effects and of effects stronger than SM D = 0.20 for each source by using a single metaregression model (Mathur & VanderWeele, 2021). However, the heterogeneity estimate in such a model is an average over the two sources, and it became apparent during data analysis that the MA showed considerably more heterogeneity than the MLR.…”

Section: Changes and Additions To Preregistered Protocolmentioning

confidence: 99%

Evidence for infant-directed speech preference is consistent across large-scale, multi-site replication and meta-analysis

Zettersten,

Cox,

Bergmann

et al. 2023

Preprint

View full text Add to dashboard Cite

There is substantial evidence that infants prefer infant-directed speech (IDS) to adult-directed speech (ADS). The strongest evidence for this claim has come from two large-scale investigations: i) a community-augmented meta-analysis of published behavioral studies and ii) a large-scale multi-lab replication study. In this paper, we aim to improve our understanding of the IDS preference and its boundary conditions by combining and comparing these two data sources across key population and design characteristics of the underlying studies. Our analyses reveal that both the meta-analysis and multi-lab replication show moderate effect sizes (d = 0.35 for both estimates) and that both of these effects persist when relevant study-level moderators are added to the models (i.e., experimental methods, infant ages, and native languages). However, while the overall effect size estimates were similar, the two sources diverged in the effects of key moderators: both infant age and experimental method predicted IDS preference in the multi-lab replication study, but showed no effect in the meta-analysis. These results demonstrate that the IDS preference generalizes across a variety of experimental conditions and sampling characteristics, while simultaneously identifying key differences in the empirical picture offered by each source individually and pinpointing areas where substantial uncertainty remains about the influence of theoretically central moderators on IDS preference. Overall, our results show how meta-analyses and multi-lab replications can be used in tandem to understand the robustness and generalizability of developmental phenomena.

show abstract

Meta‐regression methods to characterize evidence strength using meaningful‐effect percentages conditional on study characteristics

Abstract: Citation: Mathur MB & VanderWeele TJ (in press). Meta-regression methods to characterize evidence strength using meaningful-effect percentages conditional on study characteristics. Research Synthesis Methods.

Cited by 6 publications

References 58 publications

The economic rationale for cell-based influenza vaccines in children and adults: A review of cost-effectiveness analyses

The economic rationale for cell-based influenza vaccines in children and adults: A review of cost-effectiveness analyses

Sensitivity analysis for the interactive effects of internal bias and publication bias in meta‐analyses

Evidence for infant-directed speech preference is consistent across large-scale, multi-site replication and meta-analysis

Contact Info

Product

Resources

About