Computer Adaptive Testing Using Upper-Confidence Bound Algorithm for Formative Assessment

Abstract

There is strong support for formative assessment inclusion in learning processes, with the main emphasis on corrective feedback for students. However, traditional testing and Computer Adaptive Testing can be problematic to implement in the classroom. Paper based tests are logistically inconvenient and are hard to personalize, and thus must be longer to accurately assess every student in the classroom. Computer Adaptive Testing can mitigate these problems by making use of Multi-Dimensional Item Response Theory at cost of introducing several new problems, most problematic of which are the greater test creation complexity, because of the necessity of question pool calibration, and the debatable premise that different questions measure one common latent trait. In this paper a new approach of modelling formative assessment as a Multi-Armed bandit problem is proposed and solved using Upper-Confidence Bound algorithm. The method in combination with e-learning paradigm has the potential to mitigate such problems as question item calibration and lengthy tests, while providing accurate formative assessment feedback for students. A number of simulation and empirical data experiments (with 104 students) are carried out to explore and measure the potential of this application with positive results.

Time Saving Students’ Formative Assessment: Algorithm to Balance Number of Tasks and Result Reliability

“…There are other approaches to model students and knowledge acquisition [38], these include performance factor analysis [39,40], knowledge tracing [41][42][43], and multi-arm bandit (MAB) solutions [44][45][46]. Multi-armed bandit algorithms are named after a gambler's problem, where they have to choose which slot machine, one armed bandit, to play in order to maximize their reward based on previous pay-off observations [47][48][49].…”

Smart Learning

“…Guerrero-Higueras et al [48] show a methodology that uses several machine learning models' performances to select the appropriate predicting model for assessing the students' achievements based on their interactions with a version control system in computer science subjects. Melesko and Novickij [49] tackle the formative assessment defining an approach based on a Multi-Armed bandit problem that uses the Upper-Confidence Bound algorithm in adaptive tests.…”

Gamifying Massive Online Courses: Effects on the Social Networks and Course Completion Rates

“…Social networking presents a unique opportunity to develop peer support, thereby filling this instructional void [11]. E-learning can also take advantage of personalized and adaptive methods such as testing for measuring student performance [12], which can be used to tailor content and feedback. However, the effects of gamification in the MOOC context have not been thoroughly explored yet because it is being implemented in similar ways to those in small-scale contexts [13].…”