Mining Process Data to Detect Aberrant Test Takers

Liao, Manqian; Patton, Jeffrey; Yan, Ray; Jiao, Hong

doi:10.1080/15366367.2020.1827203

Cited by 7 publications

(6 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data mining steps can interact with users or knowledge bases, provide interesting patterns to users, or store them in knowledge bases as new knowledge [8]. Data mining is used to discover hidden patterns, which is the most important step in the whole process of knowledge discovery.…”

Section: Research On Data Mining Modelmentioning

confidence: 99%

Intelligent Application of Data Mining Model in Chinese International Education

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In order to improve the effect of intelligentization of Chinese international education, this paper puts forward the application method of the data mining model. Through the collection, transformation, processing, mining, and analysis of a large number of random data generated in education and teaching, this method provides learning suggestions for learners, realizes evaluation and prediction, presents knowledge information for educators, realizes educational decision-making, and realizes the continuous optimization of the teaching process. The experimental results show that the relationship between teachers’ teaching and students’ scores is obtained through the analysis of association rules in data mining technology. The support and confidence of students’ scores taught by teacher B are 28.57% and 72%, respectively, and the support and confidence of students’ scores taught by teacher C are 19.05% and 70.59%, respectively. The two rules are established to a great extent through data association. Data mining technology can effectively optimize the effect of intelligentization of Chinese international education.

show abstract

Section: Research On Data Mining Modelmentioning

confidence: 99%

Intelligent Application of Data Mining Model in Chinese International Education

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…For example, Liao et al. (2021) performed clustering of process data with multiple initial values. Similarly, Chen et al.…”

Section: Resultsmentioning

confidence: 99%

“…Among articles that addressed/mentioned this risk, cluster stability (e.g., in cluster analysis, latent class analysis, and topic model) and the sensitivity of clustering algorithms to initial values were discussed in various applications of clustering. For example, Liao et al (2021) performed clustering of process data with multiple initial values. Similarly, Chen et al ( 2017) adjusted the value of K in the K-nearest-neighbor algorithm to monitor cluster stability.…”

Section: Risks Associated With Involving ML In Measurementmentioning

confidence: 99%

MxML (Exploring the Relationship between Measurement and Machine Learning): Current State of the Field

Zheng,

Nydick,

Huang

et al. 2024

Educational Measurement

View full text Add to dashboard Cite

The recent surge of machine learning (ML) has impacted many disciplines, including educational and psychological measurement (hereafter shortened as measurement). The measurement literature has seen rapid growth in applications of ML to solve measurement problems. However, as we emphasize in this article, it is imperative to critically examine the potential risks associated with involving ML in measurement. The MxML project aims to explore the relationship between measurement and ML, so as to identify and address the risks and better harness the power of ML to serve measurement missions. This paper describes the first study of the MxML project, in which we summarize the state of the field of applications, extensions, and discussions about ML in measurement contexts with a systematic review of the recent 10 years’ literature. We provide a snapshot of the literature in (1) areas of measurement where ML is discussed, (2) types of articles (e.g., applications, conceptual, etc.), (3) ML methods discussed, and (4) potential risks associated with involving ML in measurement, which result from the differences between what measurement tasks need versus what ML techniques can provide.

show abstract

“…The information obtained from log files has been used for addressing a wide range of research questions, such as identifying problem-solving strategies (Stadler et al, 2019), exploring different patterns of behaviors (Hahnel et al, 2022; Zhu et al, 2019), and measuring test-takers’ ability, engagement, and motivation levels (Nagy et al, 2022; Xiao et al, 2021). However, only a few studies have utilized process data for anomaly detection (e.g., Gorgun & Bulut, 2022; Liao et al, 2021). The term anomaly (also referred to as aberrance or outlier in the literature) describes rare events, items, or observations that differ significantly from standard (i.e., norm) patterns.…”

Section: Theoretical Frameworkmentioning

confidence: 99%

Unsupervised Anomaly Detection in Sequential Process Data

Bulut,

Gorgun,

2024

Zeitschrift für Psychologie

View full text Add to dashboard Cite

In this study, we present three types of unsupervised anomaly detection to identify anomalous test-takers based on their action sequences in problem-solving tasks. The first method relies on the use of the Isolation Forest algorithm to detect anomalous test-takers based on raw action sequences extracted from process data. The second method transforms raw action sequences into contextual embeddings using the Bidirectional Encoder Representations from Transformers (BERT) model and then applies the Isolation Forest algorithm to detect anomalous test-takers. The third method follows the same procedure as the second method, but it includes an intermediary step of dimensionality reduction for the contextual embeddings before applying the Isolation Forest algorithm for detecting anomalous cases. To compare the outcomes of the three methods, we analyze the log files from test-takers in the US sample ( n = 2,021) who completed the problem-solving in technology-rich environments (PSTRE) section of the Programme for the International Assessment of Adult Competencies (PIAAC) 2012 assessment. The results indicated that different groups of test-takers were flagged as anomalous depending on the representation (raw action sequences vs. contextual embeddings) and dimensionality of action sequences. Also, when the contextual embeddings were used, a larger number of test-takers were flagged by the Isolation Forest algorithm, indicating the sensitivity of this algorithm to the dimensionality of input data.

show abstract

Mining Process Data to Detect Aberrant Test Takers

Cited by 7 publications

References 14 publications

Intelligent Application of Data Mining Model in Chinese International Education

Intelligent Application of Data Mining Model in Chinese International Education

MxML (Exploring the Relationship between Measurement and Machine Learning): Current State of the Field

Unsupervised Anomaly Detection in Sequential Process Data

Contact Info

Product

Resources

About