Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Spikol, Daniel; Ruffaldi, Emanuele; Dabisias, Giacomo; Cukurova, Mutlu

doi:10.1111/jcal.12263

Cited by 136 publications

(123 citation statements)

References 38 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…Extending previous results (eg, Fairclough et al ., ; Marshall, ), and coinciding with more recent findings (eg, Di Mitri et al , ; Junokas et al , ; Spikol et al , ), our findings suggest that although individual modalities can be a good proxy for performance and effort, fusing features from different modalities has the potential to further increase prediction accuracy. In other words, and in line with Giannakos et al (), it is confirmed that data fusion produces more consistent and accurate predictions than those from individual data sources.…”

Section: Discussionsupporting

confidence: 91%

“…Previous studies revealed significant findings in terms of what physiological data are appropriate for explaining students' behavior (Lane & D'Mello, 2019) and modeling and predicting their emotions, engagement with the tasks and performance, in diverse learning settings (D'Mello et al, 2009;Di Mitri et al, 2018;Fairclough et al, 2009;Marshall, 2002;Spikol et al, 2018). In these settings, the exploitation of ML techniques was proposed to reduce human workload during the analysis of learners' interactions, and to select appropriate multimodal data for capturing learners' behaviors, (Andrade et al, 2016;Ochoa et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

“…Taking advantage of technological advancements in “big data” capturing and processing, many studies carried out in educational settings have used multimodal data and focused either on measuring learner engagement in individual/collaborative conditions (eg, Andrade, Delandshere, & Danish, ; Worsley & Blikstein, ), or on predicting individual learner performance in more diverse set‐ups (eg, Spikol, Ruffaldi, Dabisias, & Cukurova, ).…”

Section: Related Work: Utilizing Multimodal Data To Predict Learning mentioning

confidence: 99%

“…Building pipelines for educational data 3009 measuring learner engagement in individual/collaborative conditions (eg,Andrade, Delandshere, & Danish, 2016;Worsley & Blikstein, 2014), or on predicting individual learner performance in more diverse set-ups (eg,Spikol, Ruffaldi, Dabisias, & Cukurova, 2018).…”

mentioning

confidence: 99%

See 3 more Smart Citations

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Sharma¹,

Papamitsiou²,

Giannakos³

2019

Brit J Educational Tech

View full text Add to dashboard Cite

Students' on‐task engagement during adaptive learning activities has a significant effect on their performance, and at the same time, how these activities influence students' behavior is reflected in their effort exertion. Capturing and explaining effortful (or effortless) behavior and aligning it with learning performance within contemporary adaptive learning environments, holds the promise to timely provide proactive and actionable feedback to students. Using sophisticated machine learning (ML) algorithms and rich learner data, facilitates inference‐making about several behavioral aspects (including effortful behavior) and about predicting learning performance, in any learning context. Researchers have been using ML methods in a “black‐box” approach, ie, as a tool where the input data is the learner data and the output is a given class from the chosen construct. This work proposes a methodological shift from the “black‐box” approach to a “grey‐box” approach that bridges the hypothesis/literature‐driven (feature extraction) “white‐box” approach with the computation/data‐driven (feature fusion) “black‐box” approach. This will allow us to utilize data features that are educationally and contextually meaningful. This paper aims to extend current methodological paradigms, and puts into practice the proposed approach in an adaptive self‐assessment case study taking advantage of new, cutting‐edge, interdisciplinary work on building pipelines for educational data, using innovative tools and techniques. What is already known about this topic Capturing and measuring learners' engagement and behavior using physiological data has been explored during the last years and exhibits great potential. Effortless behavioral patterns commonly exhibited by learners, such as “cheating,” “guessing” or “gaming the system” counterfeit the learning outcome. Multimodal data can accurately predict learning engagement, performance and processes. What this paper adds Generalizes a methodology for building machine learning pipelines for multimodal educational data, using a modularized approach, namely the “grey‐box” approach. Showcases that fusion of eye‐tracking, facial expressions and arousal data provide the best prediction of effort and performance in adaptive learning settings. Highlights the importance of fusing data from different channels to obtain the most suited combinations from the different multimodal data streams, to predict and explain effort and performance in terms of pervasiveness, mobility and ubiquity. Implications for practice and/or policy Learning analytics researchers shall be able to use an innovative methodological approach, namely the “grey‐box,” to build machine learning pipelines from multimodal data, taking advantage of artificial intelligence capabilities in any educational context. Learning design professionals shall have the opportunity to fuse specific features of the multimodal data to drive the interpretation of learning outcomes in terms of physiological learner states. The constraints from th...

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Discussionmentioning

confidence: 99%

Section: Related Work: Utilizing Multimodal Data To Predict Learning mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Sharma¹,

Papamitsiou²,

Giannakos³

2019

Brit J Educational Tech

View full text Add to dashboard Cite

show abstract

“…Similarly, Di Mitri et al (2017) investigated the potential of multimodal data from Fitbit wristband and computer logs to predict learners' self-regulation performance. In the context of project-based learning, Cukurova, Luckin, Millán, and Mavrikis (2018), and Spikol, Ruffaldi, Dabisias, and Cukurova (2018) collected data from learners' hand movements and head direction to predict their success in open-ended design tasks. In the area of professional development, Echeverria, Martinez-Maldonado, Power, Hayes, and Shum (2018) used sensor data to capture trainee nurses' interactions during healthcare training and created visualisations of their interaction and movements for effective reflections.…”

mentioning

confidence: 99%

Artificial intelligence and multimodal data in the service of human decision‐making: A case study in debate tutoring

Cukurova¹,

Kent²,

Luckin³

2019

Brit J Educational Tech

Self Cite

View full text Add to dashboard Cite

The question: “What is an appropriate role for AI?” is the subject of much discussion and interest. Arguments about whether AI should be a human replacing technology or a human assisting technology frequently take centre stage. Education is no exception when it comes to questions about the role that AI should play, and as with many other professional areas, the exact role of AI in education is not easy to predict. Here, we argue that one potential role for AI in education is to provide opportunities for human intelligence augmentation, with AI supporting us in decision‐making processes, rather than replacing us through automation. To provide empirical evidence to support our argument, we present a case study in the context of debate tutoring, in which we use prediction and classification models to increase the transparency of the intuitive decision‐making processes of expert tutors for advanced reflections and feedback. Furthermore, we compare the accuracy of unimodal and multimodal classification models of expert human tutors' decisions about the social and emotional aspects of tutoring while evaluating trainees. Our results show that multimodal data leads to more accurate classification models in the context we studied.

show abstract

The digitalization of science education: Déjà vu all over again?

Neumann

Waight

2020

J Res Sci Teach

View full text Add to dashboard Cite

The digitalization of science education: Déjà vu all over again? 1 | INTRODUCTION This special issue set out to provide a platform for reporting on empirical research that examines the use and impact of 21st century cutting-edge digital technologies and ecologies on science teaching, learning, and assessment. For decades technologies, and more recently digital technologies, have been said to revolutionize education, STEM education and, more specifically, science education. This movement started in the 1980s, when personal computers began to become available in classrooms. Personal computers, together with the respective software opened up a wealth of new possibilities in teaching and learning. Simulations provided dynamic visualizations of complex content in order to better support students in mastering understanding of these contents (Marks, 1982), and interactive learning programs based on videos allowed students to work through the curriculum at their own pace enabling a more individualized learning experience (Leonard, 1985). The former quickly developed into a vast amount of software tools designed to support students' science learning-from tools designed to model authentic phenomena (Doerr, 1996) to simulations of laboratory environments to engage students in authentic inquiry (Niesink et al., 1997). The latter, partially fueled by the advent of the internet and related technologies such as the hypertext protocol, developed into a variety of computer-based learning environments-from online (e)learning environments provided for remote, self-determined learning (Ajadi, Salawu, & Adeoye, 2007) to intelligent tutoring systems designed to automatically monitor and support students' learning (Graesser, Conley & Olney, 2012). More recent developments are driven by substantial increases in computing power on the one hand and research findings suggesting that (digital) technologies, such as simulations or modeling tools, alone are not necessarily helping students' learning, but instead need to be embedded in meaningful curriculum (e.g., Zhang, 2012; for an overview see Krajcik & Mun, 2014) on the other. Many new developments in the field engage students in authentic learning experiences through simulations of the real world (Barab et al., 2009), and integrate multiple individual technologies, such as simulations, modeling, or data analysis tools into a carefully sequenced curriculum activities (Gerard, Spitulnik & Linn, 2010). Most recent developments even integrate an automated tracking of students' learning and respective supports either through the learning environment itself or through the teacher (Gobert et al., 2013; Gobert & Sao Pedro, 2017). And the future is envisioned even brighter: Augmented Reality devices are expected to create authentic learning experiences, and Artificial Intelligence to allow for more open, exploratory (e)learning environments that automatically guide students in their learning based on their specific needs. All these technologies are said to be soon delivered through low cost person...

show abstract

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Cited by 136 publications

References 38 publications

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Artificial intelligence and multimodal data in the service of human decision‐making: A case study in debate tutoring

The digitalization of science education: Déjà vu all over again?

Contact Info

Product

Resources

About