Cheat me not: automated proctoring of digital exams on bring-your-own-device

Migut, Gosia; Koelma, D.C.; Snoek, Cees G. M.; Brouwer, Natasa

doi:10.1145/3197091.3205813

Cited by 11 publications

(10 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, the exam was performed within a restricted time span; therefore, students did not have time to cheat. Consequently, students were probably very reluctant to cheat, and this might have increased their stress levels [ 10 , 13 ].…”

Section: Discussionmentioning

confidence: 99%

“…Prior to the exam, we thoroughly instructed the candidates on the registration of suspicious events. This single intervention might have been decisive in stopping candidates from committing fraud during this high-stakes exam [ 2 , 4 , 10 ]. A simple recording of sounds and images combined with a safe web browser that blocked all other webpages might have been just as efficient as the supervisor app [ 2 , 4 ].…”

Section: Discussionmentioning

confidence: 99%

“…We decided to prioritize the format of this high-stakes test and organize-in addition to the original on-campus exam-a proctored off-campus exam. Web-based exams (or remote exams) for high-stakes assessments are considered user-friendly and cost-friendly and are flexible in terms of planning, but their validity is questionable if fraud cannot be ruled out [3,[8][9][10]. The development and administration processes of proctored exams are still under construction and have been boosted during the COVID-19 pandemic [5].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Schoenmakers¹,

Wens²

2021

JMIR Form Res

View full text Add to dashboard Cite

Background The COVID-19 pandemic has affected education and assessment programs and has resulted in complex planning. Therefore, we organized the proficiency test for admission to the Family Medicine program as a proctored exam. To prevent fraud, we developed a web-based supervisor app for tracking and tracing candidates’ behaviors. Objective We aimed to assess the efficiency and usability of the proctored exam procedure and to analyze the procedure’s impact on exam scores. Methods The application operated on the following three levels to register events: the recording of actions, analyses of behavior, and live supervision. Each suspicious event was given a score. To assess efficiency, we logged the technical issues and the interventions. To test usability, we counted the number of suspicious students and behaviors. To analyze the impact that the supervisor app had on students’ exam outcomes, we compared the scores of the proctored group and those of the on-campus group. Candidates were free to register for off-campus participation or on-campus participation. Results Of the 593 candidates who subscribed to the exam, 472 (79.6%) used the supervisor app and 121 (20.4%) were on campus. The test results of both groups were comparable. We registered 15 technical issues that occurred off campus. Further, 2 candidates experienced a negative impact on their exams due to technical issues. The application detected 22 candidates with a suspicion rating of >1. Suspicion ratings mainly increased due to background noise. All events occurred without fraudulent intent. Conclusions This pilot observational study demonstrated that a supervisor app that records and registers behavior was able to detect suspicious events without having an impact on exams. Background noise was the most critical event. There was no fraud detected. A supervisor app that registers and records behavior to prevent fraud during exams was efficient and did not affect exam outcomes. In future research, a controlled study design should be used to compare the cost-benefit balance between the complex interventions of the supervisor app and candidates’ awareness of being monitored via a safe browser plug-in for exams.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Schoenmakers¹,

Wens²

2021

JMIR Form Res

View full text Add to dashboard Cite

show abstract

“…Online human proctoring means that there will be remote proctors watching students during the whole online exam. It is a very common method used by many online testing solution providers (e.g., Kryterion, Loyalist Exam Services) and some universities (e.g., University of Amsterdam) [23]. However, it is very labor-intensive and the cost will be high when a large number of students attend an online exam.…”

Section: Online Proctoringmentioning

confidence: 99%

“…However, it is often difficult for the existing fully-automated proctoring methods to achieve a very high accuracy and the validation of the results becomes an issue. To mitigate this issue, some recent online exam proctoring approaches combine the detection by machine learning approaches and further manual confirmation by proctors [20,23]. But their manual confirmation relies on manually checking the original videos backwards and forwards, which is still inconvenient and time-consuming for proctors.…”

Section: Introductionmentioning

confidence: 99%

A Visual Analytics Approach to Facilitate the Proctoring of Online Exams

Wang

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Online exams have become widely used to evaluate students' performance in mastering knowledge in recent years, especially during the pandemic of COVID-19. However, it is challenging to conduct proctoring for online exams due to the lack of face-to-face interaction. Also, prior research has shown that online exams are more vulnerable to various cheating behaviors, which can damage their credibility. This paper presents a novel visual analytics approach to facilitate the proctoring of online exams by analyzing the exam video records and mouse movement data of each student. Specifically, we detect and visualize suspected head and mouse movements of students in three levels of detail, which provides course instructors and teachers with convenient, efficient and reliable proctoring for online exams. Our extensive evaluations, including usage scenarios, a carefully-designed user study and expert interviews, demonstrate the effectiveness and usability of our approach. CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI); Visual analytics; • Applied computing → E-learning; Learning management systems.

show abstract