Use of Data Mining Methods to Detect Test Fraud

Abstract

Data mining methods have drawn considerable attention across diverse scientific fields. However, few applications could be found in the areas of psychological and educational measurement, and particularly pertinent to this article, in test security research. In this study, various data mining methods for detecting cheating behaviors on large-scale assessments are explored as an alternative to the traditional methods including person-fit statistics and similarity analysis. A common data set from the Handbook of Quantitative Methods for Detecting Cheating on Tests (Cizek & Wollack) was used for comparing the performance of the different methods. The results indicated that the use of data mining methods may combine multiple sources of information about test takers' performance, which may lead to higher detection rate over traditional item response and response time methods. Several recommendations, all based on our findings, are provided to practitioners.

Cognitive Diagnostic Models for Random Guessing Behaviors

“…In addition to RG, RT-GDINA-RG can be easily extended to adapt diverse test-taking behaviors and various tests’ requirements. For example, we can extend CDMs to include other test-taking behaviors such as prior knowledge/pre-knowledge ( Wang et al, 2018 ; Man et al, 2019 ) or nonresponses ( Ulitzsch et al, 2019 ) if and only if the probabilities of a correct response from different latent indicators (or classes) can be clearly defined. In a high-stakes test, individuals often use pre-knowledge to correctly answer items with extremely short RT (unlike solution attempts with relatively long RT and unlike RGs with often wrong answers and short RT).…”

Using Natural Language Processing to Predict Item Response Times and Improve Test Construction

“…We experimented with a number of the machine learning algorithms implemented in the WEKA toolkit (Hall et al., 2009) using our full set of features. (An introduction to data mining in the context of educational measurement can be found in Sinharay, 2016; a recent application can be found in Man, Harring, & Sinharay, 2019.) All models were evaluated using a 10‐fold cross validation, where data are split into 10 folds or parts.…”

Cómo asegurar evaluaciones válidas y detectar falseamiento en pruebas a distancia síncronas

“…Recientemente, este tipo de modelización ha sido utilizado como un camino alternativo para la detección de falseamiento (Qian, Staniewska, Reckase & Woo, 2016, Sinharay & Johnson, 2019y Kasli Zopluoglu & Toton, 2020. Otra herramienta por considerar es partir del supuesto de que los estudiantes que falsean los resultados en las pruebas tengan un patrón de respuestas similar, por lo tanto, diversos métodos de clasificación automática pueden arrojar información acerca de estos grupos (Zopluoglu, 2019a y Man, Harring & Sinharay, 2019). Pueden ser implementados diferentes algoritmos, por ejemplo k-medias, SVM, bosques aleatorios, redes neuronales y pueden compararse sus resultados con los obtenidos a través de los índices clásicos (Zopluoglu, 2019b).…”