Defining and measuring completion and assessment biases with respect to English language and development status: not all MOOCs are equal

Abstract

The emergence of Massive Open Online Courses (MOOCs) broadened the educational landscape by providing free access to quality learning materials for anyone with a device connected to the Internet. However, open access does not guarantee equals opportunities to learn, and research has repetitively reported that learners from affluent countries benefit the most from MOOCs. In this work, we delve into this gap by defining and measuring completion and assessment biases with respect to learners’ language and development status. We do so by performing a large-scale analysis across 158 MITx MOOC runs from 120 different courses offered on edX between 2013 and 2018, with 2.8 million enrollments. We see that learners from developing countries are less likely to complete MOOCs successfully, but we do not find evidence regarding a negative effect of not being English-native. Our findings point out that not only the specific population of learners is responsible for this bias, but also that the course itself has a similar impact. Independent of and less frequent than completion bias, we found assessment bias, that is when the mean ability gained by learners from developing countries is lower than that of learners from developed countries. The ability is inferred from the responses of the learners to the course-assessment using item response theory (IRT). Finally, we applied differential item functioning (DIF) methods with the objective of detecting items that might be causing the assessment bias, obtaining weak, yet positive results with respect to the magnitude of the bias reduction. Our results provide statistical evidence on the role that course design might have on these biases, with a call for action so that the future generation of MOOCs focus on strengthening their inclusive design approaches.

The Impact of Gamification With an Emphasis on External Motivation in Moocs for Students in Emerging Regions From an Underprivileged Group: a Case Study of the Level Up Course by Gamelab Kbtu and Unicef in Kazakhstan

“…Despite the positive results of gamification in MOOCs, the paper also identifies several research gaps, particularly the focus on developed countries [2,21,22,23,24] and the lack of evidence regarding gamification's effectiveness in developing country contexts. This oversight becomes even more critical considering that learners from developing countries tend to have lower MOOC completion rates [7,8], highlighting an urgent need for research and targeted interventions to bridge these educational disparities. Additionally, further research is essential to understand how external motivation can be effectively integrated within gamified learning environments.…”

Portraying the Performance of Indonesian’s Massive Open Online Course Facilitators

“…MOOCs were initially intended to be massive, that is, cater to vast numbers of students. MOOCs have become a strategic educational medium as they are broadly affordable to all social levels of people interested in learning (Czerniewicz et al, 2017; Gershon et al, 2021; Yu et al, 2019). It was online, that is, provided over the internet.…”