Measurement invariance techniques to enhance measurement sensitivity

Abstract

Rigorous evidence supporting the effectiveness of interventions is needed to inform teaching practice and improve educational outcomes. In many instances, gathering such evidence includes cluster randomised control trials estimating the effectiveness of educational treatments. Such studies often require the collection of data from large samples in order to accurately detect 80 B.F. French et al.intervention effects. A failure to detect these effects could be due to the inability of the intervention to produce effects or due to a lack of measurement sensitivity to the intervention itself. The current study outlines a two-stage method for evaluating measurement sensitivity by first conducting content analysis to align items with hypothesised intervention effects, followed by the use of differential item functioning analyses to detect intervention effects more precisely, and thereby test for measurement sensitivity. Increasing measurement sensitivity could lead to increased effect sizes, increased statistical power, reduced sample sizes and reduced costs.

Absolute and Relative Measures of Instructional Sensitivity

“…Studies taking on a time points perspective have also regularly applied relative item sensitivity measures. Actually, relative item sensitivity between time points is better known as item parameter drift due to the teaching students were exposed to and is commonly investigated using methods of DIF detection (e.g., DeMars, 2004;French, Finch, Randel, Hand, & Gotch, 2016). Methods of DIF detection calculate item sensitivity measures conditional on test sensitivity.…”