More Than Numbers: The Power of Graphs in Meta-Analysis

Abstract

In meta-analysis, the assessment of graphs is widely used in an attempt to identify or rule out heterogeneity and publication bias. A variety of graphs are available for this purpose. To date, however, there has been no comparative evaluation of the performance of these graphs. With the objective of assessing the reproducibility and validity of graph ratings, the authors simulated 100 meta-analyses from 4 scenarios that covered situations with and without heterogeneity and publication bias. From each meta-analysis, the authors produced 11 types of graphs (box plot, weighted box plot, standardized residual histogram, normal quantile plot, forest plot, 3 kinds of funnel plots, trim-and-fill plot, Galbraith plot, and L'Abbé plot), and 3 reviewers assessed the resulting 1,100 plots. The intraclass correlation coefficients (ICCs) for reproducibility of the graph ratings ranged from poor (ICC = 0.34) to high (ICC = 0.91). Ratings of the forest plot and the standardized residual histogram were best associated with parameter heterogeneity. Association between graph ratings and publication bias (censorship of studies) was poor. Meta-analysts should be selective in the graphs they choose for the exploration of their data.

Graphing Effects as Fuzzy Numbers in Meta-Analysis

“…Graphical displays of data in meta-analysis are intrinsic to answering such questions. As meta-analysis has evolved as a science, several graphical approaches have been developed (for overviews and usage suggestions, see Anzures-Cabrera & Higgins, 2010;Bax et al, 2009). The study and introduction of new graphical methods remains active today (e.g., Schild & Voracek, 2015).…”

DNA methylation and leukemia susceptibility in China: Evidence from an updated meta-analysis

“…Heterogeneity of the included studies was assessed using I 2 statistics (19). When there was significant heterogeneity (I 2 >50%), the random-effects model was used to calculate the overall OR and 95% CI; otherwise, the fixed-effects model was applied (20).…”

Association between TGFB1 915G/C polymorphism and susceptibility to chronic hepatitis C virus infection: A meta-analysis

“…Heterogeneity was considered significant when P<0.10 for Cochran's Q statistic (26) or I 2 >50% for I 2 statistic (27) and the random effects model was adopted as the pooling method; otherwise, the fixed effects model was used (P>0.10 and I 2 <50%). When heterogeneity was observed, the Galbraith plot was used to detect the possible sources of heterogeneity (28). A combined analysis was performed by excluding the studies that possibly caused the heterogeneity to confirm the robustness of the pooled OR.…”