A Quantitative Framework for the Analysis of Two-Stage Exams

Abstract

Two-stage exams have gained traction in education as a means of creating collaborative active-learning experiences in the classroom in a manner that advances learning, positively increases student engagement, and reduces test anxiety. Published analyses have focused almost exclusively on the increase in student scores from the first individual stage to the second collaboration stage and have shown clear positive effects on gains in student scores. Missing from these analyses is a comprehensive evaluation of the effects of individual preparation, the characteristics of questions, and small group composition on the outcomes two-stage exams. I developed a simple quantitative framework that provides a flexible approach for estimating and evaluating the effects of individuals, questions, and groups on student performance. Additionally, the framework yields statistics appropriate for making inferences about productive collaboration, consensus-building, and counter-productive interaction that happens within small groups. Analyses of 12 exams across two courses and 2 years using the quantitative framework revealed considerable variation for all three of these effects within and among exams. Overall, the results highlight the value of quantitative estimation of two-stage exams for gaining perspective on the effects of individuals, questions, and groups on student performance, and facilitates data-driven revision of assessments, curricula, and teaching strategies towards achieving gains in students' collaborative skills.

Collaborative Two-Stage Exams Benefit Students in a Biology Laboratory Course

“…We theorized that the group exam consequently was a higher-difficulty exam than the individual exam, but we did not we did not track variability among items or across exams. Recently, Martin ( 32 ) provided a quantitative framework for researchers interested in exploring the interactions between individual knowledge, group dynamics, and question difficulty. Applying this framework to future analyses would allow us to explore variability among questions.…”

Two-stage Examinations in STEM: A Narrative Literature Review

“…Wieman et al (2014) suggested using the same questions in the group component as the individual component (perhaps with added questions), because it helped in providing targeted feedback on performance, and provided a starting point for discussion among the students (because they had all thought about the questions and had committed to an answer). Martin (2018) found a particularly interesting result in a study where identical questions were presented in the group stage as in the individual stage. In this study, where all students in a group got a particular question wrong, the group collectively got that question right in the group stage 57% of the time.…”

“…Students should not be time-stressed during the group stage of the examination, so as to maximize the benefit of the group discussion (Bentley et al, 2021). The approach of Martin (2018) to give students as much time as they need was applied with a large cohort; this is a useful model and is compatible with an asynchronous assessment format.…”

Defining Active Learning: A Restricted Systemic Review

“…Treagust et al (2020) provide more specificity with connecting social constructivism and active learning by identifying process-oriented guided inquiry learning (POGIL) as an example of small group active learning, based in Vygotsky's (1978) notion that the origin of knowledge construction is the interaction between people that involves sharing, comparing, and debating. Martin (2018), as well, focuses on the social nature of small group activities, stating that "active learning typically involves having students construct their understanding often as a consequence of working in small groups of peers on authentic problems aligned with relevant goals" (49). In sum, Holec and Marynowski (2020) state that "active learning is a way to engage students in the social construction of knowledge" (141).…”