GORIC Evidence Aggregation: Combining Statistical Evidence for a Central Theory from Diverse Studies using an AIC-type Criterion

Abstract

In social and behavioral science, the gold standard for scientific evidence is finding results that are consistent across independent studies. To summarize results from multiple studies, parameter estimates are conventionally aggregated with meta-analysis. However, this method is limited to studies that share the same context and design, which often means that a wealth of information remains unexploited. This paper proposes evidence aggregation using GORIC(A) weights: an alternative and/or complementary statistical tool for the aggregation of evidence across studies. Rather than aggregating parameter estimates to come to an overall estimate, GORIC(A) evidence aggregation combines support for a shared central theory and quantifies the overall support. It does so using GORIC(A), an information criterion that can evaluate both equality and inequality/order restrictions. GORIC(A) can be applied to a single study, and this GORIC(A) evidence can be aggregated over multiple studies, irrespective of context or design. The method is validated with a simulation study that shows that GORIC(A) evidence aggregation is not affected by study heterogeneity and can be used for evidence synthesis. This implies that GORIC(A) evidence aggregation can successfully combine evidence for a central theory over a widely diverse set of studies. This increases the available information to investigate a theory. Furthermore, GORIC(A) evidence aggregation aids in robustness and confidence of results because it can take into account the results of all type of studies that examine the central theory.