My thoughts through a robot's eyes: An augmented reality-brain–machine interface

Kansaku, Kenji; Hata, Naoki; Takano, Kouji

doi:10.1016/j.neures.2009.10.006

Cited by 46 publications

(25 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…AR was combined with SSVEP BMI to provide a rich virtual environment (Faller et al, 2010), and we used AR with an LCD monitor and an agent robot in P300 BMI so that the users could operate home electronics in the robot's environment (Kansaku et al, 2010). In this study, we developed an AR–BMI system operated with a see-through HMD, which may be useful in building advanced intelligent environments (Kansaku, in press).…”

Section: Discussionmentioning

confidence: 99%

Towards Intelligent Environments: An Augmented Reality–Brain–Machine Interface Operated with a See-Through Head-Mount Display

2011

Self Cite

View full text Add to dashboard Cite

The brain–machine interface (BMI) or brain–computer interface is a new interface technology that uses neurophysiological signals from the brain to control external machines or computers. This technology is expected to support daily activities, especially for persons with disabilities. To expand the range of activities enabled by this type of interface, here, we added augmented reality (AR) to a P300-based BMI. In this new system, we used a see-through head-mount display (HMD) to create control panels with flicker visual stimuli to support the user in areas close to controllable devices. When the attached camera detects an AR marker, the position and orientation of the marker are calculated, and the control panel for the pre-assigned appliance is created by the AR system and superimposed on the HMD. The participants were required to control system-compatible devices, and they successfully operated them without significant training. Online performance with the HMD was not different from that using an LCD monitor. Posterior and lateral (right or left) channel selections contributed to operation of the AR–BMI with both the HMD and LCD monitor. Our results indicate that AR–BMI systems operated with a see-through HMD may be useful in building advanced intelligent environments.

show abstract

Section: Discussionmentioning

confidence: 99%

Towards Intelligent Environments: An Augmented Reality–Brain–Machine Interface Operated with a See-Through Head-Mount Display

2011

Self Cite

View full text Add to dashboard Cite

show abstract

“…We prepared a white (20 cd/cm)/gray (6.5 cd/cm) matrix for the white/gray condition, and a green (20 cd/cm)/blue (6.5 cd/cm) matrix for the green/blue condition, and luminance was measured using a chromatic meter (CS-200, Konica Minolta Sensing, Inc., Osaka, Japan) on the computer screen as we did in a previous study (Takano et al, 2009b). We used 100 ms of intensification and a 100-ms inter-stimulus interval because recent visual P300-BCI studies have usually used 125–300 ms for stimulus onset asynchrony (SOA) to facilitate rapid communication (Sellers et al, 2006; Takano et al, 2009b, 2011; Kansaku et al, 2010; Townsend et al, 2010; Ikegami et al, 2011; Pires et al, 2011). Each row/column of the matrix was intensified randomly.…”

Section: Methodsmentioning

confidence: 99%

“…In the P300-BCI system, the parietal and occipital electrodes are effective (Krusienski et al, 2008; Rakotomamonjy and Guigue, 2008; Cecotti et al, 2011), and a configuration with parietal and occipital electrodes is commonly used (Nijboer et al, 2008; Takano et al, 2009b, 2011; Kansaku et al, 2010; Townsend et al, 2010; Ikegami et al, 2011). The importance of the parietal and occipital electrodes for achieving better performance of the P300-BCI approach has also been reported (Bianchi et al, 2010; Brunner et al, 2010; Treder and Blankertz, 2010).…”

Section: Introductionmentioning

confidence: 99%

Effect of the Green/Blue Flicker Matrix for P300-Based Brain–Computer Interface: An EEG–fMRI Study

Ikegami¹,

Takano²,

Wada³

et al. 2012

Front. Neur.

Self Cite

View full text Add to dashboard Cite

The visual P300-brain–computer interface, a popular system for EEG-based BCI, utilizes the P300 event-related potential to select an icon arranged in a flicker matrix. In the conventional P300-BCI speller paradigm, white/gray luminance intensification of each row/column in the matrix is used. In an earlier study, we applied green/blue luminance and chromatic change in the P300-BCI system and reported that this luminance and chromatic flicker matrix was associated with better performance and greater subject comfort compared with the conventional white/gray luminance flicker matrix. In this study, we used simultaneous EEG-functional magnetic resonance imaging (fMRI) recordings to identify brain areas that were more enhanced in the green/blue flicker matrix than in the white/gray flicker matrix, as these may highlight areas devoted to improved P300-BCI performance. The peak of the positive wave in the EEG data was detected under both conditions, and the peak amplitudes were larger at the parietal and occipital electrodes, particularly in the late components, under the green/blue condition than under the white/gray condition. fMRI data showed activation in the bilateral parietal and occipital cortices, and these areas, particularly those in the right hemisphere, were more activated under the green/blue condition than under the white/gray condition. The parietal and occipital regions more involved in the green/blue condition were part of the areas devoted to conventional P300s. These results suggest that the green/blue flicker matrix was useful for enhancing the so-called P300 responses.

show abstract

“…Robot agents in this environment can be operated through their visual perspective. Operation is achievable using systems like the EEG P-300 paradigm (Kansaku et al, 2010). …”

Section: Neuroprostheticsmentioning

confidence: 99%

Neuroengineering tools/applications for bidirectional interfaces, brain–computer interfaces, and neuroprosthetic implants – a review of recent progress

Rothschild¹

2010

Front. Neuroeng.

View full text Add to dashboard Cite

The main focus of this review is to provide a holistic amalgamated overview of the most recent human in vivo techniques for implementing brain–computer interfaces (BCIs), bidirectional interfaces, and neuroprosthetics. Neuroengineering is providing new methods for tackling current difficulties; however neuroprosthetics have been studied for decades. Recent progresses are permitting the design of better systems with higher accuracies, repeatability, and system robustness. Bidirectional interfaces integrate recording and the relaying of information from and to the brain for the development of BCIs. The concepts of non-invasive and invasive recording of brain activity are introduced. This includes classical and innovative techniques like electroencephalography and near-infrared spectroscopy. Then the problem of gliosis and solutions for (semi-) permanent implant biocompatibility such as innovative implant coatings, materials, and shapes are discussed. Implant power and the transmission of their data through implanted pulse generators and wireless telemetry are taken into account. How sensation can be relayed back to the brain to increase integration of the neuroengineered systems with the body by methods such as micro-stimulation and transcranial magnetic stimulation are then addressed. The neuroprosthetic section discusses some of the various types and how they operate. Visual prosthetics are discussed and the three types, dependant on implant location, are examined. Auditory prosthetics, being cochlear or cortical, are then addressed. Replacement hand and limb prosthetics are then considered. These are followed by sections concentrating on the control of wheelchairs, computers and robotics directly from brain activity as recorded by non-invasive and invasive techniques.

show abstract

My thoughts through a robot's eyes: An augmented reality-brain–machine interface

Cited by 46 publications

References 12 publications

Towards Intelligent Environments: An Augmented Reality–Brain–Machine Interface Operated with a See-Through Head-Mount Display

Towards Intelligent Environments: An Augmented Reality–Brain–Machine Interface Operated with a See-Through Head-Mount Display

Effect of the Green/Blue Flicker Matrix for P300-Based Brain–Computer Interface: An EEG–fMRI Study

Neuroengineering tools/applications for bidirectional interfaces, brain–computer interfaces, and neuroprosthetic implants – a review of recent progress

Contact Info

Product

Resources

About