Applying meta-analysis to research on bilingualism: An introduction

Abstract

Meta-analysis overcomes a number of the limitations of traditional literature reviews (Norris & Ortega, 2006). Consequently, the use of meta-analysis as a synthetic technique has been applied across a range of scientific disciplines in recent decades. This paper seeks to formally introduce the potential of meta-analysis to the field of bilingualism. In doing so, we first describe a number of advantages to the meta-analytic approach such as greater systematicity, objectivity, and transparency relative to narrative reviews. We also outline the major stages in conducting a meta-analysis, highlighting critical considerations encountered at each stage. These include (a) domain definition, (b) coding scheme development and implementation, (c) analysis, and (d) interpretation. The focus, however, is on providing a conceptual introduction rather than a full-length tutorial. Meta-analyses in bilingualism and nearby fields are referred to throughout in order to illustrate the points being made.

The benefits of preregistration for hypothesis-driven bilingualism research

“…The Type I error issue might be particularly prevalent in L2 studies because many of them use frequentist NHST and only report binary decisions about the presence or absence of an effect without also reporting effect estimates (Marsden et al, 2018c). One unfortunate consequence is that other researchers cannot gain knowledge about the magnitude of an effect across studies, or conduct meta-analyses due to the lack of information from previous studies (Plonsky, 2013;Larson-Hall & Plonsky, 2015;Plonsky & Oswald, 2014;Al-Hoorie & Vitta, 2019; for an introduction to meta-analyses in bilingualism research, see Plonsky & Oswald, 2015;Plonsky, Sudina & Hu, 2020).…”

P-CURVING AS A SAFEGUARD AGAINST P-HACKING IN SLA RESEARCH

“…In bilingualism research a conspicuous exception is the study by Dick et al (2019) that was summarized in a previous section. Also with respect to bilingualism research, Plonsky et al (2021) referred to use of p-curving for the purpose of estimating effect sizes in a meta-analytic context. As to SLA research, Vitta and Al-Hoorie (2020) used the p-curve app (Simonsohn et al, 2017) to carry out a p-curve analysis as one strand of their standard random effects meta-analysis of empirical studies of the "flipped classroom."…”

The benefits of preregistration for hypothesis-driven bilingualism research

“…The Type I error issue might be particularly prevalent in L2 studies because many of them use frequentist NHST and only report binary decisions about the presence or absence of an effect without also reporting effect estimates (Marsden, Thompson & Plonsky, 2018). One unfortunate consequence is that other researchers cannot gain knowledge about the magnitude of an effect across studies, or conduct meta-analyses due to the lack of information from previous studies (Plonsky, 2013;Larson-Hall & Plonsky, 2015;Plonsky & Oswald, 2014;Al-Hoorie & Vitta, 2019; for an introduction to meta-analyses in bilingualism research, see Plonsky & Oswald, 2015;Plonsky, Sudina & Hu, 2020).…”