The Current Research of Combining Multi-Modal Brain-Computer Interfaces With Virtual Reality

Dong, Wen; Liang, Bingbing; Zhou, Yanhong; Chen, Hongqian; Jung, Tzyy-Ping

doi:10.1109/jbhi.2020.3047836

Cited by 21 publications

(10 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Augmented and virtual reality (AR/VR) —While these technologies are quickly improving, input in AR/VR is still far from perfect. Therefore, active BCIs have some chances to compete, either as a general-purpose AR/VR input mean or in connection with BCI games and BCI art (Putze, 2019 ; Cattan et al, 2020 ; Paszkiel, 2020 ; Wen et al, 2021 ). Noticeably, NextMind, the company that provided their BCI for the above-mentioned Mercedes car (Rosso, 2021 ), was recently purchased by an AR developer (Heath, 2022 ).…”

Section: Applicationsmentioning

confidence: 99%

“…Further development of transfer learning (e.g., Zanini et al, 2017 ; Fahimi et al, 2019 ; Dehghani et al, 2021 ) and more recent meta-learning (Li et al, 2021 ; Bhosale et al, 2022 ; Wei et al, 2022 ) approaches make possible applying a classifier trained on large multisubject datasets to the data from new users. Additional opportunities can be found in combining different BCI modalities and creating hybrid systems based on joint use of a BCI and other input devices (Wen et al, 2021 ).…”

Section: Perspectivesmentioning

confidence: 99%

See 1 more Smart Citation

Active Brain-Computer Interfacing for Healthy Users

Shishkin

2022

Front. Neurosci.

View full text Add to dashboard Cite

Section: Applicationsmentioning

confidence: 99%

Section: Perspectivesmentioning

confidence: 99%

Active Brain-Computer Interfacing for Healthy Users

Shishkin

2022

Front. Neurosci.

View full text Add to dashboard Cite

“…Moreover, when applied in a real-world environment, the users would need to shift their visual attention back and forth between the stimulus presentation on a monitor and their normal visual field. Recently, affordable virtual reality (VR) and augmented reality (AR) head mounted displays (HMDs), enable the integration of SSVEP BCIs with AR/VR and has gained increased attention [8]- [11]. Furthermore, the minimal setup time and portability of EEG systems make the AR/VRbased BCIs a promising approach to move BCIs out of research lab settings and realize practical real-world applications.…”

Section: Introduction Electroencephalography Based Brain Computer Interfacesmentioning

confidence: 99%

Enhanced System Robustness of Asynchronous BCI in Augmented Reality Using Steady-State Motion Visual Evoked Potential

Ravi

Lü

Pearce³

et al. 2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

This study evaluated the effect of change in background on steady state visually evoked potentials (SSVEP) and steady state motion visually evoked potentials (SSMVEP) based brain computer interfaces (BCI) in a small-profile augmented reality (AR) headset. A four target SSVEP and SSMVEP BCI was implemented using the Cognixion AR headset prototype. An active (AB) and a non-active background (NB) were evaluated. The signal characteristics and classification performance of the two BCI paradigms were studied. Offline analysis was performed using canonical correlation analysis (CCA) and complex-spectrum based convolutional neural network (C-CNN). Finally, the asynchronous pseudo-online performance of the SSMVEP BCI was evaluated. Signal analysis revealed that the SSMVEP stimulus was more robust to change in background compared to SSVEP stimulus in AR. The decoding performance revealed that the C-CNN method outperformed CCA for both stimulus types and NB background, in agreement with results in the literature. The average offline accuracies for W=1s of C-CNN were (NB vs. AB): SSVEP: 82% ±15% vs. 60% ±21% and SSMVEP: 71.4% ± 22% vs. 63.5% ± 18%. Additionally, for W=2s, the AR-SSMVEP BCI with the C-CNN method was 83.3% ± 27% (NB) and 74.1% ±22% (AB). The results suggest that with the C-CNN method, the AR-SSMVEP BCI is both robust to change in background conditions and provides high decoding accuracy compared to the AR-SSVEP BCI. This study presents novel results that highlight the robustness and practical application of SSMVEP BCIs developed with a low-cost AR headset.

show abstract

“…The BCI represents a communication pathway between the brain and a computer device using a variety of biosensors that gather and interpret the signals from body and mind to enable neural controls over computer functions. The BCI has been used in a wide variety of applications, including rehabilitation, robotics, entertainment, and virtual reality [3][4][5][6]. No longer seen as a purely assistive technology, BCI has been gaining interest as a noninvasive physiological observation mechanism applicable to health care and education settings [7].…”

Section: Introductionmentioning

confidence: 99%

Integration of Augmented Reality and Brain-Computer Interface Technologies for Health Care Applications: Exploratory and Prototyping Study

Andrews

2022

JMIR Form Res

View full text Add to dashboard Cite

Background Augmented reality (AR) and brain-computer interface (BCI) are promising technologies that have a tremendous potential to revolutionize health care. While there has been a growing interest in these technologies for medical applications in the recent years, the combined use of AR and BCI remains a fairly unexplored area that offers significant opportunities for improving health care professional education and clinical practice. This paper describes a recent study to explore the integration of AR and BCI technologies for health care applications. Objective The described effort aims to advance an understanding of how AR and BCI technologies can effectively work together to transform modern health care practice by providing new mechanisms to improve patient and provider learning, communication, and shared decision-making. Methods The study methods included an environmental scan of AR and BCI technologies currently used in health care, a use case analysis for a combined AR-BCI capability, and development of an integrated AR-BCI prototype solution for health care applications. Results The study resulted in a novel interface technology solution that enables interoperability between consumer-grade wearable AR and BCI devices and provides the users with an ability to control digital objects in augmented reality using neural commands. The article discusses this novel solution within the context of practical digital health use cases developed during the course of the study where the combined AR and BCI technologies are anticipated to produce the most impact. Conclusions As one of the pioneering efforts in the area of AR and BCI integration, the study presents a practical implementation pathway for AR-BCI integration and provides directions for future research and innovation in this area.

show abstract

The Current Research of Combining Multi-Modal Brain-Computer Interfaces With Virtual Reality

Cited by 21 publications

References 87 publications

Active Brain-Computer Interfacing for Healthy Users

Active Brain-Computer Interfacing for Healthy Users

Enhanced System Robustness of Asynchronous BCI in Augmented Reality Using Steady-State Motion Visual Evoked Potential

Integration of Augmented Reality and Brain-Computer Interface Technologies for Health Care Applications: Exploratory and Prototyping Study

Contact Info

Product

Resources

About