A Framework for Policies and Practices to Improve Test Security Programs: Prevention, Detection, Investigation, and Resolution (PDIR)

Abstract

Test security is not an end in itself; it is important because we want to be able to make valid interpretations from test scores. In this article, I propose a framework for comprehensive test security systems: prevention, detection, investigation, and resolution. The article discusses threats to test security, roles and responsibilities, rigorous training for everyone involved in testing, and an evaluation of current practices in test security. I call on everyone responsible for testing programs—the Association of Test Publishers, Council of Chief State School Officers, National Council on Measurement in Education, U.S. Department of Education, and state assessment program managers, their vendors, and the research community—to collaborate on guidelines and practices for security violation prevention, detection, investigation, and resolution.

Cheating Detection in Tests: A Systematic Review

“…The primary objective of this study is to explore the literature landscape on cheating detection methods with a systematic review. As Ferrara (2017) suggests, research on methods to detect cheating behavior is still inorganized. Despite having several literatures that compare the existing indices (See Gaertner and McBride (2017), Wang, Liu, Robin, andGuo (2019), andWollack (2003)), no existing literature provides an exhaustive summary of cheating detection methods.…”

“…Given the situation, test security is considered as an essential aspect of the test to uphold confidence of the public sector that decisions made from the test score is valid and supported (Ferrara, 2017). Numerous cheating detection methods were invented in response to the widespread of cheating in tests, from the traditional answer comparing program to the employment of machine learning and biometric scan (Hartman, 1986;Man, Harring, & Sinharay, 2019;Weinstein, 2017).…”

“…Numerous cheating detection methods were invented in response to the widespread of cheating in tests, from the traditional answer comparing program to the employment of machine learning and biometric scan (Hartman, 1986;Man, Harring, & Sinharay, 2019;Weinstein, 2017). Nevertheless, literature on the topic of cheating detection is still in its emerging phase and scattered due to its requirement of a sizable number of examinees to establish a standard for the test (Ferrara, 2017). More method is continually invented without any milestone to keep track of the literature accumulation.…”

Machine Learning Literacy for Measurement Professionals: A Practical Tutorial

“…Even before the pandemic, the ML revolution has already impacted most, if not all, disciplines. For example, in educational measurement, researchers and practitioners have successfully applied ML in various areas, including content validity (Anderson et al., 2020), item/test development (Rafatbakhsh et al., 2021), test security (Ferrara, 2017), marking/scoring (Ercikan & McCaffrey, 2022), and crisis prediction (Burkhardt et al., 2020), to name a few. Meanwhile, various new ML‐based educational disciplines have emerged (e.g., educational data mining, learning analytics, and computational psychometrics; von Davier et al., 2021).…”

Hierarchical Agglomerative Clustering to Detect Test Collusion on Computer‐Based Tests

“…Examples of test collusion include cases where test items are exposed by brain‐dumping via website or social media; inappropriate test preparation is provided; and secure material is taught by instructors, teachers, and trainers. It also extends to proxy testing, using cheat sheets, and security breaches at the testing centers (Ferrara, 2017; Maynes, 2017; Wollack & Maynes, 2017). Studies have shown that compromised test content is among the most frequently observed abnormalities before test administrations.…”