A novel brain-controlled wheelchair combined with computer vision and augmented reality

Abstract

Background Brain-controlled wheelchairs (BCWs) are important applications of brain–computer interfaces (BCIs). Currently, most BCWs are semiautomatic. When users want to reach a target of interest in their immediate environment, this semiautomatic interaction strategy is slow. Methods To this end, we combined computer vision (CV) and augmented reality (AR) with a BCW and proposed the CVAR-BCW: a BCW with a novel automatic interaction strategy. The proposed CVAR-BCW uses a translucent head-mounted display (HMD) as the user interface, uses CV to automatically detect environments, and shows the detected targets through AR technology. Once a user has chosen a target, the CVAR-BCW can automatically navigate to it. For a few scenarios, the semiautomatic strategy might be useful. We integrated a semiautomatic interaction framework into the CVAR-BCW. The user can switch between the automatic and semiautomatic strategies. Results We recruited 20 non-disabled subjects for this study and used the accuracy, information transfer rate (ITR), and average time required for the CVAR-BCW to reach each designated target as performance metrics. The experimental results showed that our CVAR-BCW performed well in indoor environments: the average accuracies across all subjects were 83.6% (automatic) and 84.1% (semiautomatic), the average ITRs were 8.2 bits/min (automatic) and 8.3 bits/min (semiautomatic), the average times required to reach a target were 42.4 s (automatic) and 93.4 s (semiautomatic), and the average workloads and degrees of fatigue for the two strategies were both approximately 20. Conclusions Our CVAR-BCW provides a user-centric interaction approach and a good framework for integrating more advanced artificial intelligence technologies, which may be useful in the field of disability assistance.

Evaluation of Different Types of Stimuli in an Event-Related Potential-Based Brain–Computer Interface Speller under Rapid Serial Visual Presentation

“…The choice will be contingent upon user preferences and needs and the intended purpose of employing the ERP-BCI. Although our interface has been tested for a speller, the commands to be selected could be modified with the aim of controlling other types of applications, such as a wheelchair or a home automation system [ 50 , 51 ]. As a result, it is advisable for such decisions to be grounded in real-world usage scenarios.…”

Semi-Autonomous Continuous Robotic Arm Control Using an Augmented Reality Brain-Computer Interface

“…The NASA-TLX questionnaire generates a workload score based on a weighted average of ratings across six factors: mental demand, physical demand, temporal demand, performance, effort and frustration level [5], [25]- [27]. After block four, participants were asked to select the largest contributor to workload for the reaching task, from each pair of factors (15 comparisons).…”

“…We evaluated the SC AR-BCI in a task where participants used the system to control a robotic arm to reach objects in a 3D environment. We hypothesised that compared to DC, SC would improve task success rate and reaching efficiency, as well as reducing participant workload, which was assessed using the NASA Task Load Index (TLX) questionnaire [5], [25]- [27]. The main contributions of our work are twofold:…”

A review on smart robotic wheelchairs with advancing mobility and independence for individuals with disabilities

“…Intelligent navigation and obstacle avoidance are essential capabilities of smart robotic wheelchairs. Liu et al (2022) developed a wheelchair navigation system based on AI and computer vision techniques. The system utilized deep learning algorithms to recognize and classify obstacles, enabling the wheelchair to autonomously plan optimal paths and avoid collisions.…”