Missing Data Management Practices in L2 Research: The Good, The Bad and The Ugly

Abstract

Missing data are one of the frequently encountered problems in quantitative research. When neglected or handled improperly, this problem can have adverse impact on research results. However, the issue of missing data in quantitative second language (L2) research has largely been ignored when compared to the other sister disciplines such as education and psychology. The purpose of this methodological synthesis was, therefore, to investigate the issue of missing data in L2 research, with a particular focus on L2 researchers' current missing data management practices. A total of 143 studies published in six leading L2 journals were reviewed in this synthesis. The results indicated that missing data were indeed quite common in L2 research in that 41% of the studies indicated evidence of missing data, but L2 researchers' management and reporting of missing data was often less than optimal. In light of the results, several directed suggestions were made to improve the rigor and quality of L2 research.

Improving Listening Skills with Extensive Listening Using Podcasts and Vodcasts

“…A missing value analysis was also performed to deal with missing data. The proportion of missing values was found to be less than 5%, and thus conventional missing data management methods (i.e., list-wise and pair-wise deletion methods) were applied when necessary (Gonulal, 2019c;Schafer & Graham, 2002). Then, frequencies, percentages, descriptive statistics, confidence intervals, and the Wilcoxon signed-rank test were calculated, and parallel coordinate plots were created.…”

The Use of Instagram as a Mobile-Assisted Language Learning Tool

“…In looking at the rate of missing values, 7% (n = 226) was missing out of the total values (n = 2,878) on the survey. Considering that the proportion of missing data was larger than the suggested cut-off level (i.e., 5%; see Gonulal, 2019 andSchafer &Graham, 2002), multiple imputation method was used to remedy the missing data issue. In the multiple imputation method, the data were imputed 5 times and then the returned estimates were averaged.…”

Study and Scale Quality in Second Language Survey Research, 2009–2019: The Case of Anxiety and Motivation

“…Finally, concerning reporting practices, primary researchers are strongly encouraged to provide information regarding (a) survey response and/or completion rate: the higher the response rate, the lower the nonresponse bias, and the better the generalizability of the findings; (b) missing data handling techniques: the commonly used deletion techniques that operate by discarding incomplete cases are only accurate if the data are missing completely at random (MCAR; see the Discussion section for more information regarding this issue); (c) instrument origins, development, reliability, and availability; and (d) full descriptive statistics associated with the variables of interest (as recommended by Baraldi & Enders, 2010;Brown, 2001;Derrick, 2016;Dörnyei & Csizér, 2012;Fincham, 2008;Flake et al, 2017;Fowler, 2013;Gönülal, 2019;Kim, 2009;Plonsky, 2013).…”

“…A coding scheme was designed to extract all relevant data needed to address the research questions. This instrument was based on (a) recommendations by prominent synthetic researchers in the field (e.g., Oswald & Plonsky, 2010;Plonsky & Oswald, 2015), (b) previous systematic reviews related to the topic (e.g., Derrick, 2016;Flake et al, 2017;Gönülal, 2019;Kim, 2009;Lang & Little, 2018;Mendoza & Phung, 2019;Plonsky, 2013Plonsky, , 2014Plonsky & Derrick, 2016;Plonsky & Gönülal, 2015;Teimouri et al, 2019), and (c) relevant literature on questionnaire and scale development from within and beyond applied linguistics (e.g., Baraldi & Enders, 2010;Brown, 2001;DeVellis, 2017; Dewaele, 2018;Dörnyei & Csizér, 2012;Dörnyei, 2010;Fowler, 2013;Menold & Bogner, 2016).…”