Meta‐mining: The political economy of meta‐analysis

Abstract

Meta‐analysis studies the literature reporting estimates of one parameter, which at present is assumed positive. The purpose of the analysis is to find the best meta‐average, which corrects the mean of the estimates for bias. The two main biases are: (i) Publication bias, where the correction nearly always makes the average smaller. (ii) Omitted variable bias, where the correction typically makes the average larger. Consequently, the bias is likely to increase if the correction is for the wrong bias. This allows a game of meta‐mining to be played. A case study demonstrates the scope for meta‐mining, and that it has been done. The game of meta‐mining is surely against the purpose of meta‐analysis.

A Systematic Literature Review and Meta-Analysis of the Effects of Lockdowns on COVID-19 Mortality II

“…However, the influenza vanished at the exact same time in Norway and Sweden, as illustrated in Figure 8, suggesting that if lockdowns spurred significant voluntary behavioral changes, the Swedish 500person limit on public gatherings effective by March 12, 2020, may have been sufficient to spur these changes. 35 Wilder-Smith and Osman (2020) state that "six events were declared PHEIC between 2007H1N1 influenza pandemic, Ebola (West African outbreak 2013-2015, outbreak in Democratic Republic of Congo 2018, poliomyelitis (2014 to present), Zika (2016) and COVID-19 (2020 to present)." 36 There's approximately a two-to-four-week lag between infection and deaths.…”

“…Mueller-Langer et al (2019) find that only 0.1% of publications in the top 50 economics journals were replication studies. However, as described by Paldam (2022), the same question is often analyzed in many studies, which use different data samples, estimation models, control variables, etc. The theories of the studies may differ, but the estimation models are often similar.…”

“…Following Paldam (2015) and Stanley and Doucouliagos (2010), we also convert standard errors (SE) 51 and use the precision of each estimate (defined as 1/SE) to calculate the precisionweighted average of all estimates. The precision-weighted average (PWA) is our primary indicator of the efficacy of lockdowns, but we also report arithmetic averages and medians in the meta-analysis.…”

“…Wilder-Smith and Osman (2020) state that "six events were declaredPHEIC between 2007 H1N1 influenza pandemic, Ebola (West African outbreak 2013-2015, outbreak in Democratic Republic of Congo 2018, poliomyelitis (2014 to present), Zika (2016) and COVID-19 (2020 to present)." 36 There's approximately a two-to-four-week lag between infection and deaths.…”