Online Students’ Learning Behaviors and Academic Success: An Analysis of LMS Log Data From Flipped Classrooms via Regularization

Abstract

The main purpose of this study was to demonstrate the uses of regularization, a machine learning technique, in exploring important predictors for online student success. We analyzed student and learning behavioral variables from undergraduate fully-online flipped classrooms. In particular, students' instructional video watching behaviors at an instructional unit level were extracted from LMS (learning management system) log data, and Enet (elastic net) and Mnet were employed among regularization. As results, regularization not only showed comparable prediction performance to random forest, a nonlinear method well-known for its prediction capabilities, but also produced interpretable prediction models as a linear method. Enet and Mnet selected 17 and 19 important predictors out of 159, respectively, and could identify potential low-performers as early as the first instructional week of the course. Important variables rarely recognized in previous studies included complete viewings of the first video before class and repeated complete viewings of challenging contents after in-class meetings. Unlike previous studies, aggregate measures of video lecture views were not important predictors. Variables less frequently studies in previous studies were the number of non-mandatory quiz-taking and mobile lecture watching frequencies. Variables in line with previous research were student attitudes towards the course, gender, grade level, and clicks on learning materials postings. Many students turned out not to watch lecture videos completely before class. Further research on regularization and exploration of these variables with other potentially important predictors can provide more insight into students' online learning from a comprehensive perspective.

The Comparing the Impact of Asynchronous Online Quizzes on Student Learning Outcomes in a Computer Communication and Networking Course

“…Based on recent work by other researchers, other than gender, grade level and mindsets, factors such as not viewing instructional videos multiple times before and after class, mobile learning and non-mandatory quiz may contribute to lower scores (Yoo et al, 2022). Their findings found that students who failed to complete watching the videos had lower grades due to procrastination and lack of persistence, significantly influencing achievement.…”

Exploration of Variables Predicting Sense of School Belonging Using the Machine Learning Method—Group Mnet

“…First, penalized regression is a linear method with strength in interpretation compared to nonlinear models, such as random forest or deep learning. Although nonlinear models are considered to show higher prediction than linear models, recent studies that analyze social science large-scale data (Yoo & Rho, 2022) or learning analytics data (Beemer et al, 2018; Yoo et al, 2022) have reported that nonlinear methods such as random forest did not outperform penalized regression in terms of prediction. Second, penalized regression relies on the sparsity assumption (Hastie et al, 2015).…”

Machine learning procedures for predictor variable selection for schoolwork-related anxiety: evidence from PISA 2015 mathematics, reading, and science assessments

“…In the context of social science large-scale data analysis, penalized regression has several key advantages. First, penalized regression assumes linearity, and linear models are easier to interpret than nonlinear models (Yoo & Rho, 2021;Yoo et al, 2022). Although linear methods are known to produce less predictive models, in a recent study to analyze large-scale survey data, penalized regression outperformed random forest (RF), a nonlinear ML-based approach, in terms of prediction (Yoo & Rho, 2021).…”

“…While RF also has tuning parameters, researchers typically adopt what Breiman suggested, which are default settings in various statistical packages, including R. Specifically for a continuous response variable, Breiman (2001) suggested the number of predictors divided by 3 as the number of predictor candidates considered at each split. In a recent study, empirical findings found that efforts to tune RF parameters did not outperform Breiman's suggestion (Yoo et al, 2022). In this study, we employed the default values of the randomForest library in R (Liaw, 2022).…”