Measuring Motivation in Low‐Stakes Assessments

Abstract

There is a growing concern that when scores from low‐stakes assessments are reported without considering student motivation as a construct of interest, biased conclusions about how much students know will result. Low motivation is a problem particularly relevant to low‐stakes testing scenarios, which may be low stakes for the test taker but have considerable consequences for teachers, school districts, or educational and governmental institutions. The current review addresses the impact of motivation on assessment scores and research that have identified methods for minimizing the error introduced by unmotivated test takers. A comprehensive review of the test conditions that have been shown to influence motivation is provided. In addition, the review identifies a number of circumstances under which motivation can be enhanced or diminished. The benefits and limitations of the various measurement techniques that have been used to mitigate the negative impact of low‐motivation test takers on score interpretation are discussed.

Conditioning factors of test-taking engagement in PIAAC: an exploratory IRT modelling approach considering person and item characteristics

“…This section outlines how disengaged responses can occur for various reasons at the person level, the item level, or the interaction of both (Finn 2015). The term test-taking engagement already suggests that performance on a test depends not only on ability but also on motivational and emotional aspects (Asseburg and Frey 2013).…”

“…In a recent review, Finn (2015) discussed several person-level predictors of low testtaking motivation. Test-takers who were less compliant, that is, less motivated, to take the test tended to show higher levels of reactance (Brown and Finney 2011).…”

“…The negative consequences of this can include, inter alia, the underestimation of respondents' true proficiency levels and the introduction of construct-irrelevant variance (Finn 2015;Haladyna and Downing 2004;Kong et al 2007;Wise 2015).…”

“…Another option is to give a monetary reward (Braun et al 2011). However, empirical findings on whether incentives increase test-taking engagement seem to be heterogeneous, dependent on various factors, and also raise ethical issues (Finn 2015).…”

Use of Response Time for Measuring Cognitive Ability

“…These include assessing examinee motivation levels, particularly for low stakes tests [132][133][134], evaluating strategy use, for example by differences in response times for subgroups employing different problem solving strategies [130,135], or even the same individual employing different strategies at different points in the test [136], and evaluating cultural differences in pacing and time management during test taking [137]. Other applications include detecting cheating [97], assembling parallel forms [138], and item selection in adaptive testing [139,140], which may be particularly important for ensuring score comparability 16 across sets of items that might be similar in difficulty but differ in their time intensity [142,143].…”