Using Meta-Analysis to Assess Affective Outcomes in a Multi-Course QR Module Intervention

Abstract

When quantitative reasoning(QR) interventions share a common hypothesis or goal, a promising approach for evaluation involves integrating separate analyses through the use of meta-analysis. This paper reports an assessment of a module-based QR intervention distributed across 20 courses at a single institution. Topics and participating courses were diverse, including arts & humanities, quantitative behavioral sciences, and natural sciences & mathematics groupings, but all addressed the shared affective goals of reducing student QR self-doubt and increasing appreciation for QR value and utility. With a local framework to guide module development, we assess these outcomes using reliable selfreport measures in a pre-post design for each course. Random effects meta-analysis for self-doubt outcomes reveals significant moderation by course grouping, with significant but modest-sized reductions for arts & humanities (M d =-0.27, CI 95% [-0.45,-0.08]) and quantitative behavioral sciences (M d =-0.24, CI 95% [-0.47,-0.01]) but not for natural sciences & mathematics (M d = 0.13, CI 95% [-0.06, 0.32]). Analysis of perceived utility outcomes reveals a significant overall increase without moderation, but again with a pattern of significant change for the arts & humanities (M d = 0.47, CI 95% [0.11, 0.84]) and quantitative behavioral sciences (M d = 0.29, CI 95% [0.02, 0.55]) but not for natural sciences & mathematics (M d = 0.12, CI 95% [-0.18, 0.42]). Overall, the meta-analyses reveals expected patterns that would have gone undetected in the underpowered (small N) individual course implementations. We discuss strengths and limitations of meta-analytic approaches to QR assessment, along with the potential value of such aggregated information for researchers, individual instructors, and institutions.