The quality of outcome measure reporting in early numeracy intervention studies

Abstract

There is an increased demand for useful measures that capture students' math learning during intervention. Similarly, there is an awareness of the importance of researchers observing guidelines for study quality in publishing intervention results, including information related to outcome measures. We investigated the characteristics of outcome measures to assist researchers and practitioners in selecting appropriate outcome measures for early numeracy interventions. We also explored the level of quality regarding how studies reported outcome measure information. To do this, we analyzed 94 outcome measures of math achievement across 25 kindergarten early numeracy intervention studies. Overall, studies met 84% of the quality indicators related to outcome measures. Fewer studies met the recommendation to include multiple measures (i.e., proximal and distal measures; 60%) or provide validity information about measures (39%). Ultimately, the results of this study provide researchers with valuable information for developing and selecting outcome measures to determine the effectiveness of early numeracy interventions.

A Meta-Analysis and Quality Review of Mathematics Interventions Conducted in Informal Learning Environments with Caregivers and Children

“…Other desirable indicators that researchers may report include reliability and validity information of the outcome measures. Our quality review results indicate that reporting reliability and validity of measures is an area of improvement, similar to previous reviews of school-based math interventions (Park & Nelson, 2022). It is important for readers to have this information because tests that are used to determine the effectiveness of an intervention that also have low reliability or validity represent a threat to the internal validity of the study.…”