Using Potential Performance Theory to Assess How to Increase Student Consistency in Taking Exams

Abstract

<p>It has been a concern among educators and academics that U.S. students suffer from a lack of knowledge about the world around them. This is reflected in low history scores, particularly in world history. The common explanation for this is that there is some systematic deficiency in American students, in that they either do not know the material or have poor testing strategies. We offer a different way of looking at this problem using Potential Performance Theory (PPT). With PPT, we assessed the consistency with which students answered test questions and show how much performance would improve if a student were perfectly consistent. Furthermore, we show how much improvement there is in consistency over multiple sessions. Participants were given a short world history test six times in a row. The results were interesting. Consistency did improve with practice, but the systematic factors that students employed (e.g. strategies) were poor enough to counter-act the improvement due to rising consistency levels.</p

Consistency of Aviation Students When Taking the Private Pilot Knowledge Exam

“…Consistency has been shown to be valuable in optimizing student performance when completing assessments (Rice, Geels, Trafimow, & Hackett, 2011). A unique aspect to the current study was the use of a real-world exam, whereas previous studies have been conducted in lab settings (Hunt, Rice, Trafimow, & Sandry, in press;Rice, Geels, Hackett, et al, 2012;Rice, Trafimow, & Hunt, 2010;Rice, Trafimow, & Kraemer, 2012;Sandry et al, in press). The current study utilized questions from the Federal Aviation Administration's Private Pilot Knowledge Exam and was completed in a real-world setting where participants were studying to complete the official examination for pilot certification.…”

“…Based on participant responses, a withinpersons correlation is completed, and this creates a quantifiable gauge of a participant's consistency. Prior research on human-automation performance, counting, educational testtaking, memory, time pressure, visual search, perception, aerial reconnaissance, and morality have used within-persons consistency coefficients (Hunt, Rice, Trafimow, & Sandry, in press;Rice, Geels, Hackett, et al, 2012;Rice, Geels, Trafimow, & Hackett, 2011;Rice & Trafimow, 2012a, 2012bRice, Trafimow, & Hunt, 2010;Rice, Trafimow, Keller, Hunt, & Geels, 2011;Rice, Trafimow & Kraemer, 2012;Trafimow, MacDonald, & Rice, 2012;Trafimow & Rice, 2008. One common trend throughout these studies was participants' rather low consistency, particularly when the task was difficult.…”

“…Originally, there was an assumption that students' separation in scores was a result of knowledge acquired on the specific topic. Previous studies revealed the potential performance of students on assessment completion when adjusting for consistency (Rice, Geels, Trafimow, & Hackett, 2011;Rice, Trafimow, & Kraemer, 2012). These previous studies were missing two key components: a) having students self-reflect on assessment consistency and, b) completing exams in a real-world environment (a non-lab setting).…”

The Case for Adding Randomness to the Nature-Nurture Debate

“…Mean randomness figures were 39% for astronomy and 86% for spatial memory. Rice, Trafimow and Kraemer (2012) had participants take a world history test spanning thousands of years and various countries. The authors reported a mean consistency coefficient of .69 for the test, with the mean randomness being 52%.…”