Using Machine Learning to Detect ‘Multiple-Account’ Cheating and Analyze the Influence of Student and Problem Features

Ruipérez‐Valiente, José A.; Muñoz‐Merino, Pedro J.; Alexandron, Giora; Pritchard, David E.

doi:10.1109/tlt.2017.2784420

Cited by 51 publications

(30 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another direction to take from this research is to develop detection methods that can generalize across platforms and course designs, e.g. by using ML (Ruipérez-Valiente et al, 2017b) or anomaly detection techniques (Alexandron et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be!

Alexandron

Yoo

Ruipérez‐Valiente

et al. 2019

Int J Artif Intell Educ

Self Cite

View full text Add to dashboard Cite

The rich data that Massive Open Online Courses (MOOCs) platforms collect on the behavior of millions of users provide a unique opportunity to study human learning and to develop data-driven methods that can address the needs of individual learners. This type of research falls into the emerging field of learning analytics. However, learning analytics research tends to ignore the issue of the reliability of results that are based on MOOCs data, which is typically noisy and generated by a largely anonymous crowd of learners. This paper provides evidence that learning analytics in MOOCs can be significantly biased by users who abuse the anonymity and open-nature of MOOCs, for example by setting up multiple accounts, due to their amount and aberrant behavior. We identify these users, denoted fake learners, using dedicated algorithms. The methodology for measuring the bias caused by fake learners' activity combines the ideas of Replication Research and Sensitivity Analysis. We replicate two highlycited learning analytics studies with and without fake learners data, and compare the results. While in one study, the results were relatively stable against fake learners, in the other, removing the fake learners' data significantly changed the results. These findings raise concerns regarding the reliability of learning analytics in MOOCs, and highlight the need to develop more robust, generalizable and verifiable research methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be!

Alexandron

Yoo

Ruipérez‐Valiente

et al. 2019

Int J Artif Intell Educ

Self Cite

View full text Add to dashboard Cite

show abstract

“…This is an effective means of identifying false learners in online learning and a prerequisite for behavioral intervention. Ruiperez-Valiente et al (2019) applied machine learning algorithms to successfully detect "multi-account" cheating behavior, and analyzed the relationship between student personality characteristics and cheating behavior [23]. In addition, some researchers have tried to use learners' biometrics and behavior patterns to identify students, thereby preventing them from learning dishonesty [24] [25].…”

Section: Rq3: What Are the Methods To Identify College Students' Learmentioning

confidence: 99%

Online Academic Dishonesty of College Students: A Review

Chen¹,

Long²,

Liu³

et al. 2020

Proceedings of the 2020 International Conference on Advanced Education, Management and Social Science (AEMSS2020)

View full text Add to dashboard Cite

e-Learning provides students with an open, free, and self-supervised scene, which has found a feasible path for lifelong education. The quality of online learning and the certification of achievements must be based on the real learning process. According to the existing literature, research on online academic dishonesty (OAD) is relatively rare. We need to advance future research based on the past experience on online academic dishonesty, and we also need to systematically sort out the research conclusions and possible development directions of this topic. In this paper, the author systematically combed the relevant literature on online academic dishonesty, covering most of the relevant papers as of the end of 2019. Most research on online academic fraud has the following characteristics: using questionnaires to obtain exploration conclusions, focusing on learners in a certain discipline or situation, lack of research on big data learning behavior sample data, and the formation mechanism of online academic dishonesty behavior needs to be further studied and elaboration. The research results show that online academic dishonesty is indeed widespread, and the factors that cause the dishonesty behavior come from many aspects such as personality, cognition, teaching, system, etc. In future research, it is necessary to identify online academic dishonesty in time and take comprehensive intervention measures to make integrity and moral culture in online environments.

show abstract

“…In the work of Ruiperez-Valiente et al (21), the same CAMEO technique is analyzed through a machine learning approach, using a Random Forest model and 15 features to detect submissions as fraudulent. The model achieved a sensitivity level of 0.966 and a specificity level of 0.996.…”

Section: Related Workmentioning

confidence: 99%

“…They use features that rely on the student's behaviour that is affected or associated with cheating, such as the amount of interaction with the course resources, the time to answer, the student's ability and two parameters from the Item Response Theory, designed by the psychometrics research community, which are used to classify unusual response patterns, including cheating: the Guttman error and the standard error from ability estimates. By using a probabilistic classifier (logistic regression) with four features, they consider only students who received a certificate and classify them into those who used multiple accounts and those who did not, with an AUC of 0.826 on the same dataset as the one used in the work of Ruiperez-Valiente et al (21).…”

Section: Related Workmentioning

confidence: 99%

Identifying Cheating Users in Online Courses

Sangalli

Martínez-Muñoz

Cañabate

2020

2020 IEEE Global Engineering Education Conference (EDUCON)

View full text Add to dashboard Cite

Resumen Los estudiantes interactúan con los cursos en línea principalmente de dos maneras: revisando los materiales del curso y resolviendo ejercicios. Sin embargo, hay casos en los que el comportamiento de los estudiantes difiere y tiende a centrarse más en resolver ejercicios sin mirar los materiales del curso. Este tipo de interacción podría ser indicativo de un comportamiento poco ético, como los estudiantes que colaboran compartiendo respuestas entre ellos o cuentas falsas que usan los estudiantes para obtener las respuestas correctas para los ejercicios. En este trabajo, proponemos varias métricas para identificar estos dos tipos de trampas basadas en eventos concurrentes y medidas de interacción con el curso. Del conjunto de cuentas en el curso, los pares de cuentas que resuelven ejercicios muy cercanos en el tiempo se consideran cuentas potencialmente colaboradoras. Las métricas propuestas se calculan para estos pares de cuentas y la agrupación de K-means se usa para separar pares de estudiantes reales que colaboran con respecto a los estudiantes que usan cuentas falsas para obtener las respuestas correctas a los ejercicios. Se logra una precisión de generalización superior al 95 % para clasificar estos tipos de trampas mediante el uso de una máquina de vectores de soporte (SVM)

show abstract

Using Machine Learning to Detect ‘Multiple-Account’ Cheating and Analyze the Influence of Student and Problem Features

Cited by 51 publications

References 17 publications

Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be!

Are MOOC Learning Analytics Results Trustworthy? With Fake Learners, They Might Not Be!

Online Academic Dishonesty of College Students: A Review

Identifying Cheating Users in Online Courses

Contact Info

Product

Resources

About