Using the DiCoT framework for integrated multimodal analysis in mixed-reality training environments

Abstract

Simulation-based training (SBT) programs are commonly employed by organizations to train individuals and teams for effective workplace cognitive and psychomotor skills in a broad range of applications. Distributed cognition has become a popular cognitive framework for the design and evaluation of these SBT environments, with structured methodologies such as Distributed Cognition for Teamwork (DiCoT) used for analysis. However, the analysis and evaluations generated by such distributed cognition frameworks require extensive domain-knowledge and manual coding and interpretation, and the analysis is primarily qualitative. In this work, we propose and develop the application of multimodal learning analysis techniques to SBT scenarios. Using these analysis methods, we can use the rich multimodal data collected in SBT environments to generate more automated interpretations of trainee performance that supplement and extend traditional DiCoT analysis. To demonstrate the use of these methods, we present a case study of nurses training in a mixed-reality manikin-based (MRMB) training environment. We show how the combined analysis of the video, speech, and eye-tracking data collected as the nurses train in the MRMB environment supports and enhances traditional qualitative DiCoT analysis. By applying such quantitative data-driven analysis methods, we can better analyze trainee activities online in SBT and MRMB environments. With continued development, these analysis methods could be used to provide targeted feedback to learners, a detailed review of training performance to the instructors, and data-driven evidence for improving the environment to simulation designers.

Application of eye‐tracking in nursing research: A scoping review

“…Twelve studies applied eye‐tracking to explore nursing students’ visual attention among different tasks. Eye‐tracking was used to assess the effect of communication training (Anbro et al., 2020), clinical bioethics decision‐making of nursing students (Fernandes et al., 2017), students’ visual behaviours and cognition engagement in simulated/virtual reality teaching (Amster et al., 2015; Dubovi, 2022; Groepel et al., 2022), factors influencing visual processing of non‐verbal communication (Matsumoto et al., 2010), communication skills training (Kobayashi et al., 2022), predict students’ self‐confidence (Vatral et al., 2023), students’ management of and response to interruptions (Vital & Nathanson, 2023), feasibility of eye‐tracking in simulated teaching, and the role of different visual interfaces in virtual teaching (Pence et al., 2014) as a method of feedback during students’ learning (Browning et al., 2016; O'Meara et al., 2015).…”