Virtual Reality Adaptation Using Electrodermal Activity to Support the User Experience

Welsch, Robin; Villa, Steeven; Chuang, Lewis L.; Mayer, Sven

doi:10.3390/bdcc6020055

Cited by 23 publications

(2 citation statements)

References 99 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wearable sensors (functional near-infrared spectroscopy (fNIRS) brain monitoring device and Shimmer's wristband) were used to collect three types of neurophysiological signals: brain activity (fNIRS), heart activity (ECG), and electrodermal response from the skin (EDA) [16,17]. A paired t-test examined any significant changes between the baseline and post-test scores.…”

Section: Data Collection and Analysismentioning

confidence: 99%

How Does Human-Centred Extended Reality Support Healthcare Students’ Learning in Clinical Conditions?

Mikkonen,

Ferdinando,

Sobocinski

et al. 2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

Healthcare education needs to be reformed to sustain quality, faster response to crises and ensure a rapid and efficient graduation path for future healthcare professionals. In this study, our multidisciplinary team has developed and tested a Human-centred extended reality (XR) to solve challenges in healthcare by connecting humans to technology in a human-centred, ethical way and by empowering end users through social innovation. In our study, we aimed to develop an intuitive XR virtual simulation environment with realistic scenarios and metahuman avatars, enabling team interaction to test and analyse participants’ real-time adaptation through a combination of neurophysiological and behavioural data collected by wearable sensors. This novel research offers a solution to complement clinical placements of nursing and medical students and ensure that students achieve the required competencies even if unexpected situations or crises threaten to interrupt the practice of competencies in real-life environments. Furthermore, by utilising the neurophysiological data, we can assess the learning event based on analysis of the recorded signals. The XR solutions can reduce nursing and medical students’ stress levels and enhance their resilience to work effectively in collaborative interprofessional teams.

show abstract

Section: Data Collection and Analysismentioning

confidence: 99%

How Does Human-Centred Extended Reality Support Healthcare Students’ Learning in Clinical Conditions?

Mikkonen,

Ferdinando,

Sobocinski

et al. 2024

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…While there has been significant research exploring the potential of adaptive VR environments in healthcare, emergency training, user experience and cognitive training (Chiossi et al, 2022; Finseth et al, 2022; Kritikos et al, 2021; Reidy et al, 2020), there is a gap in the literature focusing on developing and examining VR environments that adapt to learners' unique self‐regulatory processes captured through multimodal data. Specifically within this paper, we capture and measure learners' SRL processes using multimodal data to understand how learners use metacognitive monitoring (captured via think‐aloud data) in response to the cognitive load (measured via heart rate variability) experienced by the learner and how this may then trigger behavioural changes as related to the physical movements learners make (captured via birds‐eye‐view video).…”

Section: Introductionmentioning

confidence: 99%

Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal data

Sobocinski,

Dever,

Wiedbusch

et al. 2023

Brit J Educational Tech

View full text Add to dashboard Cite

This study examines the embodied ways in which learners monitor their cognition while learning about exponential functions in an immersive virtual reality (VR) based game, Pandemic by Prisms of Reality. Traditionally, metacognitive monitoring has been assessed through behavioural traces and verbalised instances. When learning in VR, learners are fully immersed in the learning environment, actively manipulating it based on affordances designed to support learning, offering insights into the relationship between physical interaction and metacognition. The study collected multimodal data from 15 participants, including think‐aloud audio, bird's‐eye view video recordings and physiological data. Metacognitive monitoring was analysed through qualitative coding of the think‐aloud protocol, while movement was measured via optical flow analysis and cognitive load was assessed through heart rate variability analysis. The results revealed embodied metacognition by aligning the data to identify learners' physical states alongside their verbalised metacognition. The findings demonstrated a temporal interplay among cognitive load, metacognitive monitoring, and motion during VR‐based learning. Specifically, cognitive load, indicated by the low‐ and high‐frequency heart rate variability index, predicted instances of metacognitive monitoring, and monitoring predicted learners' motion while interacting with the VR environment. This study further provides future directions in understanding self‐regulated learning processes during VR learning by utilizing multimodal data to inform real‐time adaptive personalised support within these environments. Practitioner notesWhat is already known about this topic Immersive virtual reality (VR) environments have the potential to offer personalised support based on users' individual needs and characteristics. Self‐regulated learning (SRL) involves learners monitoring their progress and strategically regulating their learning when needed. Multimodal data captured during VR learning, such as birds‐eye‐view video, screen recordings, physiological changes and verbalisations, can provide insights into learners' SRL processes and support needs. What this paper adds Provides insights into the embodied aspects of learners' metacognitive monitoring during learning in an immersive VR environment. Demonstrates how SRL processes can be captured via the collection and analysis of multimodal data, including think‐aloud audio, bird's‐eye view video recordings and physiological data, to capture metacognitive monitoring and movement during VR‐based learning. Contributes to the understanding of the interplay between cognitive load, metacognitive monitoring, and motion in immersive VR learning. Implications for practice and/or policy Researchers and practitioners can use the causal relationships identified in this study to identify instances of SRL in an immersive VR setting. Educational technology developers can consider the integration of online measures, such as cognitive load and physiological arousal, into adaptive VR environments to enable real‐time personalised support for learners based on their self‐regulatory needs.

show abstract

Design and Implementation of an Interactive Photoplethysmography and Galvanic Skin Response Based Gamepad

Sánchez-Sánchez

Orozco-del-Castillo

Atoche

2022

Communications in Computer and Information Science

View full text Add to dashboard Cite

Virtual Reality Adaptation Using Electrodermal Activity to Support the User Experience

Cited by 23 publications

References 99 publications

How Does Human-Centred Extended Reality Support Healthcare Students’ Learning in Clinical Conditions?

How Does Human-Centred Extended Reality Support Healthcare Students’ Learning in Clinical Conditions?

Capturing self‐regulated learning processes in virtual reality: Causal sequencing of multimodal data

Design and Implementation of an Interactive Photoplethysmography and Galvanic Skin Response Based Gamepad

Contact Info

Product

Resources

About