The successful test taker: exploring test-taking behavior profiles through cluster analysis

Abstract

To be successful in a high-stakes testing situation is desirable for any test taker. It has been found that, beside content knowledge, test-taking behavior, such as risktaking strategies, motivation, and test anxiety, is important for test performance. The purposes of the present study were to identify and group test takers with similar patterns of test-taking behavior and to explore how these groups differ in terms of background characteristics and test performance in a high-stakes achievement test context. A sample of the Swedish Scholastic Assessment Test test takers (N = 1891) completed a questionnaire measuring their motivation, test anxiety, and risk-taking behavior during the test, as well as background characteristics. A two-step cluster analysis revealed three clusters of test takers with significantly different test-taking behavior profiles: a moderate (n = 741), a calm risk taker (n = 637), and a test anxious risk averse (n = 513) profile. Group difference analyses showed that the calm risk taker profile (i.e., a high degree of risktaking together with relatively low levels of test anxiety and motivation during the test) was the most successful profile from a test performance perspective, while the test anxious risk averse profile (i.e., a low degree of risk-taking together with high levels of test anxiety and motivation) was the least successful. Informing prospective test takers about these insights can potentially lead to more valid interpretations and inferences based on the test scores. Eur J Psychol Educ (2018) shown that for a test taker to be successful when taking a test, it is important to be able to reduce anxiety and sustain motivation (Dodeen et al. 2014;Naylor 1997;Sternberg 1998), as well as using effective test-taking strategies, such as willingness to take risks (Bicak 2013;Bond and Harman 1994;Dodeen 2008). Studies in this area often compare groups of test takers and have for example found that high achievers tend to report using more effective testtaking strategies when compared to low achievers (Stenlund et al. 2017;Ellis and Ryan 2003;Hong et al. 2006;Kim and Goetz 1993), that males are more prone to taking risks when answering test items (see, e.g., Baldiga 2014), and that females and low achievers seem to experience higher levels of test anxiety than males and high achievers (Stenlund et al. 2017;Cassady and Johnson 2002;Naylor 1997). Findings like these can help understand and to some extent possibly explain performance differences that are often observed between manifest groups in achievement tests. Still, considering the consequences of successful test-taking behavior in high-stakes test situations, and assuming that test takers adopt different test-taking behavior, exploring differences across groups, in terms of profiles, and identifying patterns that seem associated with sucessful and less successful test-taking, respectively, might add important knowledge to this area. With this study, we therefore aim at identifying subgroups of test takers with similar patter...

Effects of Response Format on Psychometric Properties and Fairness of a Matrices Test: Multiple Choice vs. Free Response

“…Regarding effects of personality variables on cognitive test results, neither effects of test anxiety nor of risk propensity revealed significant differences, even though interactions between these characteristics and response format have been suggested by earlier research (e.g., Rowley, 1974;Benjamin et al, 1981;Crocker and Schmitt, 1987;Alnabhan, 2002;Rubio et al, 2010). As previous studies revealed effects of subjects' test anxiety and risk propensity on test scores particularly within high-stakes assessments (Segool et al, 2013;Knekta, 2017;Stenlund et al, 2018), aspects of the test situation (low-stakes) could have contributed to this finding. Given the small personal relevance of the test outcome for the pupils in the current study, test anxiety and risk propensity may not have influenced the subjects substantially in this test situation.…”

“…Numerous studies have shown a higher tendency for test takers possessing higher levels of risk propensity to guess in MC tests when they were unsure about items, while people with low risk propensity preferred to skip those questions (Alnabhan, 2002;Rubio et al, 2010;Baldiga, 2013). As the probability of guessing correctly in a MC test with eight response options is 12.5%, or even more if some distractors can be excluded, high risk-taking individuals have a significantly higher likelihood of achieving more points in MC tests than low risk-taking subjects (Rowley, 1974;Alnabhan, 2002;Rubio et al, 2010;Stenlund et al, 2018). Research showed that even when test takers were required to answer all items in a test based on MC format, persons with higher scores in the willingness to risk a penalty in a test situation outperformed those with lower risk scores (Rowley, 1974).…”

“…The second personality characteristic that we addressed with reference to response format was test anxiety. Test anxiety, defined as the tendency to extreme fear of being rated negatively in upcoming tests (Zeidner, 1998), has been acknowledged as a major disruptive factor for academic careers: severe test anxious pupils and students often failed in tests and achieved lower mean grades than less test anxious ones (Hembree, 1988;Zeidner, 1998;Von der Embse et al, 2013;Stenlund et al, 2018). Several studies reported sex differences in test anxiety (e.g., Putwain and Daly, 2014;Núñez-Peña et al, 2016;Lowe, 2019) and it has been identified as a source of bias for underperformance in standardized testing (as reviewed by Zeidner, 1998;McDonald, 2001).…”

The correlation between vocational school students’ test motivation and the performance in a standardized test of economic knowledge: using direct and indirect indicators of test motivation

“…The assessment of selfreported motivation can take place either prior to, during, or after the performance test (the economic knowledge test in the context of this paper). In the literature no clear suggestions have been made as to when test motivation should be assessed as a control variable: In some studies test motivation has been assessed immediately upon completion of a knowledge test (Knekta 2017;Ortner et al 2014); in others, an intermediate questionnaire has been completed at various points throughout the test (Stenlund et al 2018). Repeated assessment of test motivation certainly would make sense from a theoretical point of view because motivation may change during the test.…”

Group Differences in the Value of Subscores: A Fairness Issue

“…It is interesting to examine subscore value for males and females because these groups have for a long time been in the focus when discussing equity and fairness in society in large, and we know that they differ in several test-taking aspects, such as overall test performance, test anxiety and use of test-taking strategies (Stenlund et al, 2017(Stenlund et al, , 2018, as well as performance on constructed-response vs. multiple-choice items (Livingston and Rupp, 2004). Such group differences will show as gender bias only when these differences is unrelated to the intended test construct but still has an impact on the test result.…”