Sampling and Generalizability in Lx Research: A Second-Order Synthesis

Abstract

As in many other social sciences, second/additional language (Lx) researchers are often interested in generalizing their findings beyond the samples they collect data from. However, very little is known about the range of learner backgrounds and settings found in Lx research. Moreover, the few papers that have addressed the range of settings and demographics sampled in Lx research paint a disappointing picture). The current study examines the extent to which concerns expressed over this issue are merited and worthy of further attention. Toward that end, sample-related features such as L1, Lx/target language, age, proficiency, and educational setting (or lack thereof) were extracted from a sample of 308 systematic reviews of Lx research. The data from this “meta-synthetic” sample are then used to estimate the extent to which Lx research has sampled—and might or might not be able to generalize to—different populations and contexts including those pertinent to migrant populations, the focus of this special issue. The results reveal an incredibly disproportionate interest in participants with English as a first or target language and as well as university students in a narrow range of countries. The findings are used to call out the applied linguistics community on this gross oversight while also seeking to inform future research and contribute to the ongoing methodological reform movement in applied linguistics.

Uncovering Sampling Biases, Advancing Inclusivity, and Rethinking Theoretical Accounts in Second Language Acquisition: Introduction to the Special Issue SLA for All?

“…Similar concerns can also motivate expansion to other languages and cultures in (applied) linguistics research. On the basis of an analysis of secondary research studies, Plonsky (2023) recently estimated that 41% of all participants involved in second or additional language acquisition studies had English as a first language, while 62% were learning English as a second language. The need for a better representation of all language learners in research is therefore quite clear.…”

“…University students are not representative of the human population at large (Andringa, 2014). They are frequent outliers (Henrich et al., 2010b), and contemporary American university students, who comprise the bulk of all university students sampled (Plonsky, 2023), are outliers within an outlier population (Henrich et al., 2010b). Therefore, there are important questions that we in the field need to ask ourselves, such as: To what extent has the reliance on convenience samples in SLA skewed theoretical accounts of second language (L2) learning and teaching?…”

“…Adults other than college students accounted for only 12% of the research with adult language learners. Plonsky (2023) found that at least 39% of applied linguistics research included in synthetic research drew on university samples, at least 31% used young learners, and only 6% involved research participants from outside university settings. The percentage of samples drawn from university populations might be higher than 39%, and could be as high as 58%, because 19% of studies in the metasynthetic review by Plonsky involved research participants from multiple educational settings, one of which was likely university-level.…”

“…The percentage of samples drawn from university populations might be higher than 39%, and could be as high as 58%, because 19% of studies in the metasynthetic review by Plonsky involved research participants from multiple educational settings, one of which was likely university-level. Not only do research participants tend to come from universities (Andringa & Godfroid, 2020a;Plonsky, 2023), they tend to come from universities in the Global North, especially North America and Western Europe, which pales in comparison to the Global South in terms of its linguistic diversity and degree of multilingualism (Bylund et al, 2023).…”

Introduction to the special issue on working memory and L2 writing: Implications for SLA individual differences research

“…Populations certainly need to be expanded because samples in WM studies come too often from English majors with writing expertise who are completing or have completed academic writing courses (see field-wide evidence of such concerns in Bylund et al, 2023, and in Plonsky, 2023). This almost exclusive focus on English L2 justifies Kormos’s claim to focus on WM effects in learners of other L2s.…”