Towards Intelligent Environments: An Augmented Reality–Brain–Machine Interface Operated with a See-Through Head-Mount Display

Takano, Kouji; Hata, Naoki; Kansaku, Kenji

doi:10.3389/fnins.2011.00060

Cited by 59 publications

(42 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The classification accuracy tended to confirm our hypothesis as all of the conditions' classification accuracy were between 75% and 80% which is consistent with the literature [15], [16]. It also confirms the previous results of Takano et al [14] on the possibility to combine AR and BCI headsets.…”

Section: Resultssupporting

confidence: 93%

“…The use of BCIs has been less explored in the context of OST-AR. To the authors' best knowledge, only Takano et al [14] have explicitly explored this solution. Their study evaluated a home automation system in two steps.…”

Section: Optical See-through Ar and Bcimentioning

confidence: 99%

“…[7], [10]. Very few researchers have explored the use of BCI with Optical See-Through AR (OST) [14] and none went beyond assessing the mere hardware compatibility between the devices. It is also interesting to note that all the prototypes and the studies relied on simple user interfaces, and did not consider the possibility to improve the usability through different layouts for the SSVEP or P300 commands.…”

Section: Optical See-through Ar and Bcimentioning

confidence: 99%

See 2 more Smart Citations

Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation

Si-Mohammed

Petit

Jeunet

et al. 2020

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Brain-Computer Interfaces (BCIs) enable users to interact with computers without any dedicated movement, bringing new hands-free interaction paradigms. In this paper we study the combination of BCI and Augmented Reality (AR). We first tested the feasibility of using BCI in AR settings based on Optical See-Through Head-Mounted Displays (OST-HMDs). Experimental results showed that a BCI and an OST-HMD equipment (EEG headset and Hololens in our case) are well compatible and that small movements of the head can be tolerated when using the BCI. Second, we introduced a design space for command display strategies based on BCI in AR, when exploiting a famous brain pattern called Steady-State Visually Evoked Potential (SSVEP). Our design space relies on five dimensions concerning the visual layout of the BCI menu ; namely: orientation, frame-of-reference, anchorage, size and explicitness. We implemented various BCI-based display strategies and tested them within the context of mobile robot control in AR. Our findings were finally integrated within an operational prototype based on a real mobile robot that is controlled in AR using a BCI and a HoloLens headset. Taken together our results (4 user studies) and our methodology could pave the way to future interaction schemes in Augmented Reality exploiting 3D User Interfaces based on brain activity and BCIs.

show abstract

Section: Resultssupporting

confidence: 93%

Section: Optical See-through Ar and Bcimentioning

confidence: 99%

Section: Optical See-through Ar and Bcimentioning

confidence: 99%

See 1 more Smart Citation

Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation

Si-Mohammed

Petit

Jeunet

et al. 2020

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…It was shown that similar and satisfactory accuracies (76–88%) can be achieved with a see-through HMD as compared to an LCD monitor (Takano et al, 2011 ). In their study, participants used a TV control panel with 11 symbols in the matrix and a 2 × 2 light control panel for environmental control.…”

Section: Introductionmentioning

confidence: 91%

Rapid P300 brain-computer interface communication with a head-mounted display

2015

View full text Add to dashboard Cite

Visual ERP (P300) based brain-computer interfaces (BCIs) allow for fast and reliable spelling and are intended as a muscle-independent communication channel for people with severe paralysis. However, they require the presentation of visual stimuli in the field of view of the user. A head-mounted display could allow convenient presentation of visual stimuli in situations, where mounting a conventional monitor might be difficult or not feasible (e.g., at a patient's bedside). To explore if similar accuracies can be achieved with a virtual reality (VR) headset compared to a conventional flat screen monitor, we conducted an experiment with 18 healthy participants. We also evaluated it with a person in the locked-in state (LIS) to verify that usage of the headset is possible for a severely paralyzed person. Healthy participants performed online spelling with three different display methods. In one condition a 5 × 5 letter matrix was presented on a conventional 22 inch TFT monitor. Two configurations of the VR headset were tested. In the first (glasses A), the same 5 × 5 matrix filled the field of view of the user. In the second (glasses B), single letters of the matrix filled the field of view of the user. The participant in the LIS tested the VR headset on three different occasions (glasses A condition only). For healthy participants, average online spelling accuracies were 94% (15.5 bits/min) using three flash sequences for spelling with the monitor and glasses A and 96% (16.2 bits/min) with glasses B. In one session, the participant in the LIS reached an online spelling accuracy of 100% (10 bits/min) using the glasses A condition. We also demonstrated that spelling with one flash sequence is possible with the VR headset for healthy users (mean: 32.1 bits/min, maximum reached by one user: 71.89 bits/min at 100% accuracy). We conclude that the VR headset allows for rapid P300 BCI communication in healthy users and may be a suitable display option for severely paralyzed persons.

show abstract

“…In our system, BOTAS can also be controlled using a hybrid BMI system with SSVEP and P300 (blue arrow procedure in Figure 1) (Sakurada et al, 2011). Our group previously used the so-called P300 speller (Farwell and Donchin, 1988) in a BMI system (Ikegami et al, 2011; Takano et al, 2011); we also included the P300 procedure in the BOTAS system. A monitor displays a flicker matrix and each flickering icon indicates a BOTAS motion that was recorded beforehand.…”

Section: Methodsmentioning

confidence: 99%

A BMI-based occupational therapy assist suit: asynchronous control by SSVEP

Sakurada¹,

Kawase²,

Takano³

et al. 2013

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

A brain-machine interface (BMI) is an interface technology that uses neurophysiological signals from the brain to control external machines. Recent invasive BMI technologies have succeeded in the asynchronous control of robot arms for a useful series of actions, such as reaching and grasping. In this study, we developed non-invasive BMI technologies aiming to make such useful movements using the subject's own hands by preparing a BMI-based occupational therapy assist suit (BOTAS). We prepared a pre-recorded series of useful actions—a grasping-a-ball movement and a carrying-the-ball movement—and added asynchronous control using steady-state visual evoked potential (SSVEP) signals. A SSVEP signal was used to trigger the grasping-a-ball movement and another SSVEP signal was used to trigger the carrying-the-ball movement. A support vector machine was used to classify EEG signals recorded from the visual cortex (Oz) in real time. Untrained, able-bodied participants (n = 12) operated the system successfully. Classification accuracy and time required for SSVEP detection were ~88% and 3 s, respectively. We further recruited three patients with upper cervical spinal cord injuries (SCIs); they also succeeded in operating the system without training. These data suggest that our BOTAS system is potentially useful in terms of rehabilitation of patients with upper limb disabilities.

show abstract

Towards Intelligent Environments: An Augmented Reality–Brain–Machine Interface Operated with a See-Through Head-Mount Display

Cited by 59 publications

References 25 publications

Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation

Towards BCI-Based Interfaces for Augmented Reality: Feasibility, Design and Evaluation

Rapid P300 brain-computer interface communication with a head-mounted display

A BMI-based occupational therapy assist suit: asynchronous control by SSVEP

Contact Info

Product

Resources

About