Design Analytics for Mobile Learning

Abstract

This research was triggered by the identified need in literature for large-scale studies about the kinds of designs that teachers create for mobile learning (m-learning). These studies require analyses of large datasets of learning designs. The common approach followed by researchers when analyzing designs has been to manually classify them following high-level pedagogically guided coding strategies, which demands extensive work. Therefore, the first goal of this paper is to explore the use of supervised machine learning (SML) to automatically classify the textual content of m-learning designs using pedagogically relevant classifications, such as the cognitive level demanded by students to carry out specific designed tasks, the phases of inquiry learning represented in the designs, or the role that the situated environment has in the designs. Because not all SML models are transparent, but researchers often need to understand their behaviour, the second goal of this paper is to consider the trade-off between models’ performance and interpretability in the context of design analytics for m-learning. To achieve these goals, we compiled a dataset of designs deployed using two tools, Avastusrada and Smartzoos. With this dataset, we trained and compared different models and feature extraction techniques. We further optimized and compared the best performing and most interpretable algorithms (EstBERT and Logistic Regression) to consider the second goal with an illustrative case. We found that SML can reliably classify designs with accuracy > 0.86 and Cohen’s kappa > 0.69.

Model-Based Learning Analytics for a Partnership of Teachers and Intelligent Systems: A Bibliometric Systematic Review

“…Therefore, the future systems that we are advocating for should focus not only on training AI systems based on diverse and representative data, but also on the implementation of interpretable algorithms whenever possible. Recent advancements in machine learning that help to create black-box models more interpretable might also be useful in this context [45]. Moreover, it is necessary to consider distortion in the decisions made by the system based on a possible bias found in the dataset used to train it.…”

“…Moreover, training is one of the common approaches used to support teachers' professional development. While teacher training could promote the adoption of MbLA, it is expensive and its effects are limited in time [45]. Therefore, future research should also focus on systems that include sufficient intelligent guidance that would not only require less training but that will also be perceived as useful by teachers.…”

Behavioral Involvement, Technology Acceptance, and Failure in Mobile Learning: A Systematic Review

“…As stated in (Akour et al, 2021), the J48 classifier outperformed others in predicting people's intention to use mobile learning platforms during the COVID-19 pandemic. Moreover, in (Pishtari et al, 2022), SML was applied to classify m-learning designs, with EstBERT and Logistic Regression emerging as the bestperforming and most interpretable algorithms. In (Sultan et al, 2022), the superior performance of the fast learning network, with an accuracy of 91.6% in predicting student satisfaction in M-learning, was demonstrated.…”

Single-case learning analytics: Feasibility of a human-centred analytics approach to support doctoral education

“…This step enables the SCLA4DE system to include in its analyses/modeling new factors and variables that the students have noticed, and which were not in the original data collection design (again, giving them a voice and agency in the LA process). Moreover, once enough unstructured data has been coded by humans, computational elements can automate this coding process for continued operation (either based on latest advances in ML/NLP [Pishtari, 2021] or other means like regular expressions [Cai et al, 2019]).…”