Brain computer interface to distinguish between self and other related errors in human agent collaboration

Dimova-Edeleva, Viktorija; Ehrlich, Stefan K.; Cheng, Gordon

doi:10.1038/s41598-022-24899-8

Cited by 5 publications

(5 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The current work as well as the aforementioned studies showing EEG/ERP correlates of other types of VR interface errors (Dimova-Edeleva et al, 2022;Gehrke et al, 2019Gehrke et al, , 2022Si-Mohammed et al, 2020;Yazmir & Reiner, 2022) all have in common that they are examining brain activity elicited by physically different events (i.e., immediate vs. delayed sound presentation, matched vs. mismatched timing of multisensory information, correct vs. erroneous VR response to user input). This is promising for identifying system glitches in contexts in which the VR itself cannot be as precisely controlled as in these research settings.…”

Section: Discussionmentioning

confidence: 98%

“…Other studies have pursued similar machine-learning approaches, with ERP correlates of movement error attribution to oneself or another VR agent yielding a single-trial classification accuracy of 73% (Dimova-Edeleva et al, 2022), and ERP correlates of visuospatial tracking errors reaching a singletrial classification accuracy of 85% (Si-Mohammed et al, 2020). In the study by Si-Mohammed and colleagues (2020), two other types of VR errors (erroneous feedback and visual anomalies in the background) could not reliably be detected at the level of the users' EEG.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Feder¹,

Miksch²,

Grimm³

et al. 2023

Preprint

View full text Add to dashboard Cite

Actions in the real world have immediate sensory consequences. Mimicking these in digital environments is within reach, but technical constraints usually impose a certain latency (delay) between user actions and system responses. It is important to assess the impact of this latency on the users, ideally with measurement techniques that do not interfere with their digital experience. One such unobtrusive technique is electroencephalography (EEG), which can capture the users’ brain activity associated with motor responses and sensory events by extracting event-related potentials (ERPs) from the continuous EEG recording. Here we exploit the fact that the amplitude of sensory ERP components (specifically, N1 and P2) reflects the degree to which the sensory event was perceived as an expected consequence of an own action (self-generation effect). Participants (N=24) elicit auditory events in a virtual-reality (VR) setting by entering codes on virtual keypads to open doors. In a within-participant design, the delay between user input and sound presentation is manipulated across blocks. Occasionally, the virtual keypad is operated by a simulated robot instead, yielding a control condition with externally generated sounds. Results show that N1 (but not P2) amplitude is reduced for self-generated relative to externally generated sounds, and P2 (but not N1) amplitude is modulated by delay of sound presentation in a graded manner. This dissociation between N1 and P2 effects maps back to basic research on self-generation of sounds. We suggest P2 amplitude as a candidate read-out to assess the quality and immersiveness of digital environments with respect to system latency.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Discussionmentioning

confidence: 99%

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Feder¹,

Miksch²,

Grimm³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The current work as well as the aforementioned studies showing EEG/ERP correlates of other types of VR interface errors (Gehrke et al, 2019(Gehrke et al, , 2022Si-Mohammed et al, 2020;Alsuradi et al, 2021;Dimova-Edeleva et al, 2022;Yazmir and Reiner, 2022) all have in common that they are examining brain activity elicited by physically different events (i.e., immediate vs. delayed sound presentation, matched vs. mismatched timing of multisensory information, correct vs. erroneous VR response to user input). This is promising for identifying system glitches in contexts in which the VR itself cannot be as precisely controlled as in these research settings.…”

Section: Discussionmentioning

confidence: 98%

“…Other studies have pursued machine-learning approaches similar to Gehrke et al ( 2022 ), with ERP correlates of movement error attribution to oneself or another VR agent yielding a single-trial classification accuracy of 73% (Dimova-Edeleva et al, 2022 ), and ERP correlates of visuospatial tracking errors reaching a single-trial classification accuracy of 85% (Si-Mohammed et al, 2020 ). In the study by Si-Mohammed et al ( 2020 ), two other types of VR errors (erroneous feedback and visual anomalies in the background) could not reliably be detected at the level of the users' EEG.…”

Section: Discussionmentioning

confidence: 99%

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Feder

Miksch

Grimm

et al. 2023

Front. Neuroergonomics

View full text Add to dashboard Cite

Actions in the real world have immediate sensory consequences. Mimicking these in digital environments is within reach, but technical constraints usually impose a certain latency (delay) between user actions and system responses. It is important to assess the impact of this latency on the users, ideally with measurement techniques that do not interfere with their digital experience. One such unobtrusive technique is electroencephalography (EEG), which can capture the users' brain activity associated with motor responses and sensory events by extracting event-related potentials (ERPs) from the continuous EEG recording. Here we exploit the fact that the amplitude of sensory ERP components (specifically, N1 and P2) reflects the degree to which the sensory event was perceived as an expected consequence of an own action (self-generation effect). Participants (N = 24) elicit auditory events in a virtual-reality (VR) setting by entering codes on virtual keypads to open doors. In a within-participant design, the delay between user input and sound presentation is manipulated across blocks. Occasionally, the virtual keypad is operated by a simulated robot instead, yielding a control condition with externally generated sounds. Results show that N1 (but not P2) amplitude is reduced for self-generated relative to externally generated sounds, and P2 (but not N1) amplitude is modulated by delay of sound presentation in a graded manner. This dissociation between N1 and P2 effects maps back to basic research on self-generation of sounds. We suggest P2 amplitude as a candidate read-out to assess the quality and immersiveness of digital environments with respect to system latency.

show abstract

“…The experimental task comprised the interaction between a human and a robot in completing a trajectory-following task in a 7x7 grid-world environment ( Fig 1 ) which is an adaptation from our previous work [ 65 ]. To successfully complete the trajectory, human and robot were tasked with moving the robot end-effector to each grid tile within the trajectory in sequential order.…”

Section: Experimental Hri Studymentioning

confidence: 99%

Human-robot collaborative task planning using anticipatory brain responses

et al. 2023

Self Cite

View full text Add to dashboard Cite

Human-robot interaction (HRI) describes scenarios in which both human and robot work as partners, sharing the same environment or complementing each other on a joint task. HRI is characterized by the need for high adaptability and flexibility of robotic systems toward their human interaction partners. One of the major challenges in HRI is task planning with dynamic subtask assignment, which is particularly challenging when subtask choices of the human are not readily accessible by the robot. In the present work, we explore the feasibility of using electroencephalogram (EEG) based neuro-cognitive measures for online robot learning of dynamic subtask assignment. To this end, we demonstrate in an experimental human subject study, featuring a joint HRI task with a UR10 robotic manipulator, the presence of EEG measures indicative of a human partner anticipating a takeover situation from human to robot or vice-versa. The present work further proposes a reinforcement learning based algorithm employing these measures as a neuronal feedback signal from the human to the robot for dynamic learning of subtask-assignment. The efficacy of this algorithm is validated in a simulation-based study. The simulation results reveal that even with relatively low decoding accuracies, successful robot learning of subtask-assignment is feasible, with around 80% choice accuracy among four subtasks within 17 minutes of collaboration. The simulation results further reveal that scalability to more subtasks is feasible and mainly accompanied with longer robot learning times. These findings demonstrate the usability of EEG-based neuro-cognitive measures to mediate the complex and largely unsolved problem of human-robot collaborative task planning.

show abstract

Brain computer interface to distinguish between self and other related errors in human agent collaboration

Cited by 5 publications

References 72 publications

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Using event-related brain potentials to evaluate motor-auditory latencies in virtual reality

Human-robot collaborative task planning using anticipatory brain responses

Contact Info

Product

Resources

About