Analysing Verbal Communication in Embodied Team Learning Using Multimodal Data and Ordered Network Analysis

Zhao, Linxuan; Tan, Yuanru; Gašević, Dragan; Shaffer, David Williamson; Yan, Lixiang; Alfredo, Riordan; Li, Xinyu; Martínez-Maldonado, Roberto

doi:10.1007/978-3-031-36272-9_20

Cited by 3 publications

(1 citation statement)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the unique affordances of unifying descriptions and quantitative representation, QE has a rising interest in learning analytics. For example, Epistemic Network Analysis models behaviors captured in learning environments connected via temporal co-occurrence, which supports interpretations of learner strategy differences and their relationship to outcomes [12,15,48]. For example, Fougt et al [15] distinguish student proficiency levels in higher education writing assessment to support grading via keywords.…”

Section: Quantitative Ethnography Methods In Learning Analyticsmentioning

confidence: 96%

Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis

Borchers,

Wang,

Karumbaiah

et al. 2024

Proceedings of the 14th Learning Analytics and Knowledge Conference

View full text Add to dashboard Cite

Learning analytics research increasingly studies classroom learning with AI-based systems through rich contextual data from outside these systems, especially student-teacher interactions. One key challenge in leveraging such data is generating meaningful insights into effective teacher practices. Quantitative ethnography bears the potential to close this gap by combining multimodal data streams into networks of co-occurring behavior that drive insight into favorable learning conditions. The present study uses transmodal ordered network analysis to understand effective teacher practices in relationship to traditional metrics of in-system learning in a mathematics classroom working with AI tutors. Incorporating teacher practices captured by position tracking and human observation codes into modeling significantly improved the inference of how efficiently students improved in the AI tutor beyond a model with tutor log data features only. Comparing teacher practices by student learning rates, we find that students with low learning rates exhibited more hint use after monitoring. However, after an extended visit, students with low learning rates showed learning behavior similar to their high learning rate peers, achieving repeated correct attempts in the tutor. Observation notes suggest conceptual and procedural support differences can help explain visit effectiveness. Taken together, offering early conceptual support to students with low learning rates could make classroom practice with AI tutors more effective. This study advances the scientific understanding of effective teacher practice in classrooms learning with AI tutors and methodologies to make such practices visible.

show abstract

Section: Quantitative Ethnography Methods In Learning Analyticsmentioning

confidence: 96%

Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis

Borchers,

Wang,

Karumbaiah

et al. 2024

Proceedings of the 14th Learning Analytics and Knowledge Conference

View full text Add to dashboard Cite

show abstract

Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learning

Yan,

Echeverria,

Jin

et al. 2024

Brit J Educational Tech

View full text Add to dashboard Cite

Multimodal learning analytics (MMLA) offers the potential to provide evidence‐based insights into complex learning phenomena such as collaborative learning. Yet, few MMLA applications have closed the learning analytics loop by being evaluated in real‐world educational settings. This study evaluates the effectiveness of an MMLA solution in enhancing feedback and reflection within a complex and highly dynamic collaborative learning environment. A two‐year longitudinal study was conducted with 399 students and 17 teachers, utilising an MMLA system in reflective debriefings in the context of healthcare education. We analysed the survey data of 74 students and 11 teachers regarding their perceptions of the MMLA system. We applied the Evaluation Framework for Learning Analytics, augmented by complexity, accuracy and trust measures, to assess both teachers' and students' perspectives. The findings illustrated that teachers and students both had generally positive perceptions of the MMLA solution. Teachers found the MMLA solution helpful in facilitating feedback provision and reflection during debriefing sessions. Similarly, students found the MMLA solution effective in providing clarity on the data collected, stimulating reflection on their learning behaviours, and prompting considerations for adaptation in their learning behaviours. However, the complexity of the MMLA solution and the need for qualitative measures of communication emerged as areas for improvement. Additionally, the study highlighted the importance of data accuracy, transparency, and privacy protection to maintain user trust. The findings provide valuable contributions to advancing our understanding of the use of MMLA in supporting feedback and reflection practices in intricate collaborative learning while identifying avenues for further research and improvement. We also provided several insights and practical recommendations for successful MMLA implementation in authentic learning contexts. Practitioner notesWhat is currently known about this topic Multimodal learning analytics (MMLA) seeks to generate data‐informed insights about learners' metacognitive and emotional states as well as their learning behaviours, by utilising intricate physical and physiological signals. MMLA has not only pioneered novel data analytic methods but also aspired to complete the learning analytics loop by crafting innovative, tangible solutions that relay these insights to the concerned stakeholders. A prominent direction within MMLA research has been the formulation of tools to support feedback and reflection in collaborative learning scenarios, given MMLA's capacity to discern intricate and dynamic learning behaviours. What this paper adds Teachers' and students' positive perceptions of an MMLA implementation in stimulating considerations of adaptations in their pedagogical practices and learning behaviours, respectively. Empirical evidence supporting the potential of MMLA in assisting teachers to facilitate students' reflective practices during intricate collaborative learning scenarios. The importance of addressing issues related to design complexity, interpretability for users with disabilities, aggregated data representation, and concerns related to trust for building a practical MMLA solution in real learning settings. Implications for practice and/or policy The MMLA solution can provide teachers with a comprehensive view of student performance, illuminate areas for improvement, and confirm learning scenario outcomes. The MMLA solution can stimulate students' reflections on their learning behaviours and promote considerations of adaptation in their learning behaviours. Providing clear explanations and guidance on how to interpret analytics, as well as addressing concerns related to data completeness and representation, are essential to maximising utility.

show abstract

Data Storytelling Editor: A Teacher-Centred Tool for Customising Learning Analytics Dashboard Narratives

Fernandez-Nieto,

Martinez-Maldonado,

Echeverria

et al. 2024

Proceedings of the 14th Learning Analytics and Knowledge Conference

View full text Add to dashboard Cite

Analysing Verbal Communication in Embodied Team Learning Using Multimodal Data and Ordered Network Analysis

Cited by 3 publications

References 26 publications

Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis

Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis

Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learning

Data Storytelling Editor: A Teacher-Centred Tool for Customising Learning Analytics Dashboard Narratives

Contact Info

Product

Resources

About