Data collection and processing for a multimodal learning analytic system

Ruffaldi, Emanuele; Dabisias, Giacomo; Landolfi, Lorenzo

doi:10.1109/sai.2016.7556079

Cited by 8 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The data on the server are further processed for extracting LA and statistics. For details about the architecture please refer to Ruffaldi, Dabisias, Landolfi, and Spikol (2016).…”

Section: Pelars Lasmentioning

confidence: 99%

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Spikol

Ruffaldi

Dabisias

et al. 2018

Computer Assisted Learning

Self Cite

136

107

View full text Add to dashboard Cite

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates the use of diverse sensors, including computer vision, user‐generated content, and data from the learning objects (physical computing components), to record high‐fidelity synchronised multimodal recordings of small groups of learners interacting. We processed and extracted different aspects of the students' interactions to answer the following question: Which features of student group work are good predictors of team success in open‐ended tasks with physical computing? To answer this question, we have explored different supervised machine learning approaches (traditional and deep learning techniques) to analyse the data coming from multiple sources. The results illustrate that state‐of‐the‐art computational techniques can be used to generate insights into the "black box" of learning in students' project‐based activities. The features identified from the analysis show that distance between learners' hands and faces is a strong predictor of students' artefact quality, which can indicate the value of student collaboration. Our research shows that new and promising approaches such as neural networks, and more traditional regression approaches can both be used to classify multimodal learning analytics data, and both have advantages and disadvantages depending on the research questions and contexts being investigated. The work presented here is a significant contribution towards developing techniques to automatically identify the key aspects of students success in project‐based learning environments, and to ultimately help teachers provide appropriate and timely support to students in these fundamental aspects.

show abstract

“…The data on the server are further processed for extracting LA and statistics. For details about the architecture please refer to Ruffaldi, Dabisias, Landolfi, and Spikol (2016).…”

Section: Pelars Lasmentioning

confidence: 99%

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Spikol

Ruffaldi

Dabisias

et al. 2018

Computer Assisted Learning

Self Cite

136

107

View full text Add to dashboard Cite

show abstract

“…Besides the toolkit, the learners and observers used mobile devices to capture multimedia data (text, images and video) to self-document the learning activities. Overall, the PELARS project has developed an intelligent system for collecting activity data for diverse learning analytics (with data-mining, reasoning and visualisations) and active user-generated material and digital content (mobile tools) from practice-based activities [28]. The Talkoo toolkit was used and evaluated in combination with the PELARS LAS.…”

Section: Discussionmentioning

confidence: 99%

Physical computing with plug-and-play toolkits:Key recommendations for collaborative learning implementations

Katterfeldt

Cukurova

Cuartielles

2018

International Journal of Child-Computer Interaction

Self Cite

View full text Add to dashboard Cite

Physical computing toolkits have long been used in educational contexts to learn about computational concepts by engaging in the making of interactive projects. This paper presents a comprehensive toolkit that can help educators teach programming with an emphasis on collaboration, and provides suggestions for its effective pedagogical implementation. The toolkit comprises the Talkoo kit with physical computing plug-and-play modules and a visual programming environment. The key suggestions are inspired by the results of the evaluation studies which show that children (aged 14 to 18 in a sample group of 34 students) are well motivated when working with the toolkit but lack confidence in the kit's support for collaborative learning. If the intention is to move beyond tools and code in computer education to community and context, thus encouraging computational participation, collaboration should be considered as a key aspect of physical computing activities. Our approach expands the field of programming with physical computing for teenage children with a focus on empowering teachers and students with not only a kit but also its appropriate classroom implementation for collaborative learning.

show abstract

“…We reviewed 18 data infrastructures in MMLA (nine from our last review [Di Mitri et al, 2017, Muñoz-Cristóbal et al, 2018, Fiaidhi, 2014, Domínguez and Chiluiza, 2016, Ruffaldi et al, 2016, Harrer, 2013, Berg et al, 2016, Wagner et al, 2011, Segal et al, 2017 and nine proposed after our review , Munoz et al, 2018, Tamura et al, 2019, Ciordas-Hertel et al, 2019, Huertas Celdrán et al, 2020, Camacho et al, 2020, Domínguez et al, 2021, Serrano Iglesias et al, 2021, Slupczynski and Klamma, 2021). We started our review by finding out how many learning tools were supported in the proposals.…”

Section: Existing Mmla Infrastructuresmentioning

confidence: 99%

CIMLA: A Modular and Modifiable Data Preparation, Organization, and Fusion Infrastructure to Partially Support the Development of Context-aware MMLA Solutions

Shankar¹,

Ruiz‐Calleja²,

Prieto³

et al. 2023

jucs

View full text Add to dashboard Cite

Multimodal Learning Analytics (MMLA) solutions aim to provide a more holistic picture of a learning situation by processing multimodal educational data. Considering contextual information of a learning situation is known to help in providing more relevant outputs to educational stakeholders. However, most of the MMLA solutions are still in prototyping phase and dealing with different dimensions of an authentic MMLA situation that involve multiple cross-disciplinary stakeholders like teachers, researchers, and developers. One of the reasons behind still being in prototyping phase of the development lifecycle is related to the challenges that software developers face at different levels in developing context-aware MMLA solutions. In this paper, we identify the requirements and propose a data infrastructure called CIMLA. It includes different data processing components following a standard data processing pipeline and considers contextual information following a data structure. It has been evaluated in three authentic MMLA scenarios involving different cross-disciplinary stakeholders following the Software Architecture Analysis Method. Its fitness was analyzed in each of the three scenarios and developers were interviewed to assess whether it meets functional and non-functional requirements. Results showed that CIMLA supports modularity in developing context-aware MMLA solutions and each of its modules can be reused with required modifications in the development of other solutions. In the future, the current involvement of a developer in customizing the configuration file to consider contextual information can be investigated.

show abstract

Data collection and processing for a multimodal learning analytic system

Cited by 8 publications

References 13 publications

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Supervised machine learning in multimodal learning analytics for estimating success in project‐based learning

Physical computing with plug-and-play toolkits:Key recommendations for collaborative learning implementations

CIMLA: A Modular and Modifiable Data Preparation, Organization, and Fusion Infrastructure to Partially Support the Development of Context-aware MMLA Solutions

Contact Info

Product

Resources

About