Handling missing data: analysis of a challenging data set using multiple imputation

Abstract

Missing data is endemic in much educational research. However, practices such as step-wise regression common in the educational research literature have been shown to be dangerous when significant data are missing, and multiple imputation (MI) is generally recommended by statisticians. In this paper, we provide a review of these advances and their implications for educational research. We illustrate the issues with an educational, longitudinal survey in which missing data was significant, but for which we were able to collect much of these missing data through subsequent data collection. We thus compare methods, that is, step-wise regression (basically ignoring the missing data) and MI models, with the model from the actual enhanced sample. The value of MI is discussed and the risks involved in ignoring missing data are considered. Implications for research practice are discussed.

Validating constructs of learners’ academic self-efficacy for measuring learning gain

“…In this section, we briefly outline previous literature on these areas with emphasis on how they interplay in our study. Sander and Sanders (2003), building upon work by Bandura (1986Bandura ( , 1993 and Pajares (2000), consider that a student's confidence may be a significant determining factor as to the level of commitment they are prepared to invest into academic and social activities. Furthermore, as Webb-Williams (2006) state, confidence, and in particular self-efficacy (i.e.…”

“…drawing inferences from the fact that one student has a higher measure than another). Second, longitudinal data in education is highly prone to attrition and so missingness: inferences from data-sets that have high levels of 'missingness' will require careful analysis (Pampaka, Hutcheson, & Williams, 2014).…”

Tournament Satisfaction Scale (TOSS)

“…First, descriptive statistic was used for scanning data and missing variables. Expectation of maximization was conducted in order to complete missing parts (Pampaka, Hutcheson & Williams, 2016). Exploratory Factor Analysis with Maximum Likelihood extraction method (Fabrigar, Wegener, MacCallum, & Strahan, 1999) and oblique rotation (direct oblimin) (Preacher & MacCallum, 2003) was carried out by using the data obtained from 278 athletes in various branches.…”

What to do instead of significance testing? Calculating the ‘number of counterfactual cases needed to disturb a finding’

“…This then 'permits' them to use their favoured statistical approaches with over 60% of the relevant data missing (e.g. see Pampaka, Hutcheson, & Williams, 2014). The distinction is a false one.…”