Using Machine Learning to Train a Wearable Device for Measuring Students’ Cognitive Load during Problem-Solving Activities Based on Electrodermal Activity, Body Temperature, and Heart Rate: Development of a Cognitive Load Tracker for Both Personal and Classroom Use

Abstract

Automated tracking of physical fitness has sparked a health revolution by allowing individuals to track their own physical activity and health in real time. This concept is beginning to be applied to tracking of cognitive load. It is well known that activity in the brain can be measured through changes in the body’s physiology, but current real-time measures tend to be unimodal and invasive. We therefore propose the concept of a wearable educational fitness (EduFit) tracker. We use machine learning with physiological data to understand how to develop a wearable device that tracks cognitive load accurately in real time. In an initial study, we found that body temperature, skin conductance, and heart rate were able to distinguish between (i) a problem solving activity (high cognitive load), (ii) a leisure activity (moderate cognitive load), and (iii) daydreaming (low cognitive load) with high accuracy in the test dataset. In a second study, we found that these physiological features can be used to predict accurately user-reported mental focus in the test dataset, even when relatively small numbers of training data were used. We explain how these findings inform the development and implementation of a wearable device for temporal tracking and logging a user’s learning activities and cognitive load.

Toward Mental Effort Measurement Using Electrodermal Activity Features

“…While the term mental effort is often associated with cognitive load theory, in other areas of the literature it is common to encounter the term mental workload. These terms describe similar constructs [ 6 ] and in fields outside of education, mental workload is often measured through physiological means [ 21 ] since the autonomic nervous system responds to changes in mental workload, and these changes happen without the individual knowing the changes are occurring [ 22 ]. In relation to grain size of measurement, it is important to note that these measures are viewed as an index of workload [ 23 ], and perhaps should not be viewed as the exact amount of workload or mental effort.…”

“…To date, research around the use of wearable sensors for predicting learning-relevant outcomes has produced promising results. For example, researchers have found success using multi-modal physiological measures to differentiate between different activities and predict mental focus [ 6 ], as well as predicting perceived learning [ 7 ].…”

Intelligent Biofeedback Augmented Content Comprehension (TellBack)

“…• This work addresses the limitations of a single modality using data fusion of non-intrusive biosensors and real-time biomarkers from HRV and pupillometry. Unlike other recent studies [2,3,12] that exploit the multimodality of biosensors, we used simple features (in terms of computation), but discriminant enough to detect the increase of the user's cognitive load/mental effort in real-time. • The use of machine learning and AI techniques to optimize the prediction of "when and where" the user is encountering difficulty on screen.…”

“…Detecting cognitive load in real-time using multimodal sources of biosensors has recently emerged as one of the promising approaches in adaptive and cognition-aware elearning [2,3,12,13]. Those very recent studies were mainly focusing on providing feedback on learners' engagement and cognitive load changes.…”

Assessing learning effort with hand motion tracking methods

“…On the other hand, psychophysiological measures such as EEG (Antonenko et al, 2010) and eye‐tracking (Paas et al, 2003) data offer measures of cognitive load with higher spatial and temporal resolutions. There are also measures of heart rate, skin conductance, and body temperature, the sensors of which are wearable, and the data of which are predictive of cognitive load when used with predictive analytics (Romine et al, 2020). Psychophysiological measures can be tracked in real‐time, but they require obtrusive and often expensive devices for the measurements.…”