Predicting the decrease of engagement indicators in a MOOC

User Model User-Adap Inter

2018

149

Predictive models of student success in Massive Open Online Courses (MOOCs) are a critical component of effective content personalization and adaptive interventions. In this article we review the state of the art in predictive models of student success in MOOCs and present a categorization of MOOC research according to the predictors (features), prediction (outcomes), and underlying theoretical model. We critically survey work across each category, providing data on the raw data source, feature engineering, statistical model, evaluation method, prediction architecture, and other aspects of these experiments. Such a review is particularly useful given the rapid expansion of predictive modeling research in MOOCs since the emergence of major MOOC platforms in 2012. This survey reveals several key methodological gaps, which include extensive filtering of experimental subpopulations, ineffective student model evaluation, and the use of experimental data which would be unavailable for real-world student success prediction and intervention, which is the ultimate goal of such models. Finally, we highlight opportunities for future research, which include temporal modeling, research bridging predictive and explanatory student models, work which contributes to learning theory, and evaluating long-term learner success in MOOCs.

Section: Activity-based Modelsmentioning

confidence: 99%

Student success prediction in MOOCs

User Model User-Adap Inter

2018

149

“…These samples represent the hypothetical differences in performance on a future unseen dataset [9]; generating N = 50, 000 samples on a typical laptop computer takes only a few seconds, and conducting this comparison for all 4560 pairwise comparisons in the experiment below takes less than 10 minutes using the BayesianTestsML Python library. 3 The MCMC samples are used to estimate θ by simply counting the proportion of samples for which θ i has the highest posterior probability. The results of this sampling can be visualized by projecting the (θ X>Y , θ ROP E , θ X<Y ) triplets onto barycentric coordinates to produce a posterior plot, shown in Figure 3.…”

Section: Bayesian Model Evaluationmentioning

confidence: 99%

“…average grade) as well as more complex features (e.g. number of submissions relative to the highest number of submissions by any student that week) [3,30,43]. Where courses used no assignments, models using this method defaulted to majority-class prediction.…”

Section: Feature Extraction/data Sourcementioning

confidence: 99%

Evaluating Predictive Models of Student Success: Closing the Methodological Gap

2018

Learning Analytics

Model evaluation -the process of making inferences about the performance of predictive models -is a critical component of predictive modeling research in learning analytics. We survey the state of the practice with respect to model evaluation in learning analytics, which overwhelmingly uses only naïve methods for model evaluation or statistical tests which are not appropriate for predictive model evaluation. We conduct a critical comparison of both null hypothesis significance testing (NHST) and a preferred Bayesian method for model evaluation. Finally, we apply three methods -the naïve average commonly used in learning analytics, NHST, and Bayesian -to a predictive modeling experiment on a large set of MOOC data. We compare 96 different predictive models, including different feature sets, statistical modeling algorithms, and tuning hyperparameters for each, using this case study to demonstrate the different experimental conclusions these evaluation techniques provide.

“…To date, MOOC research has addressed a diverse array of research questions from psychometrics and social psychology to predictive modeling and machine learn-ing. For example, several works have explored prediction of various student outcomes using behavioral, linguistic, and assignment data from MOOCs to evaluate and predict various student outcomes including course completion [1], [2], [3], assignment grades [4], Correct on First Attempt (CFA) submissions [5], student confusion [6], and changes in behavior over time [7]. A key area of research has been methods for feature engineering, or extracting structured information from raw data (i.e.…”

Section: A Educational Big Data In the Mooc Eramentioning

confidence: 99%

MORF: A Framework for Predictive Modeling and Replication At Scale With Privacy-Restricted MOOC Data

2018 IEEE International Conference on Big Data (Big Data)

Andrés

et al. 2018

Big data repositories from online learning platforms such as Massive Open Online Courses (MOOCs) represent an unprecedented opportunity to advance research on education at scale and impact a global population of learners. To date, such research has been hindered by poor reproducibility and a lack of replication, largely due to three types of barriers: experimental, inferential, and data. We present a novel system for large-scale computational research, the MOOC Replication Framework (MORF), to jointly address these barriers. We discuss MORF's architecture, an opensource platform-as-a-service (PaaS) which includes a simple, flexible software API providing for multiple modes of research (predictive modeling or production rule analysis) integrated with a high-performance computing environment. All experiments conducted on MORF use executable Docker containers which ensure complete reproducibility while allowing for the use of any software or language which can be installed in the linux-based Docker container. Each experimental artifact is assigned a DOI and made publicly available. MORF has the potential to accelerate and democratize research on its massive data repository, which currently includes over 200 MOOCs, as demonstrated by initial research conducted on the platform. We also highlight ways in which MORF represents a solution template to a more general class of problems faced by computational researchers in other domains.