Design and Evaluation of a Self-Correcting Gesture Interface based on Error Potentials from EEG

Abstract

Any user interface which automatically interprets the user's input using natural modalities like gestures makes mistakes. System behavior depending on such mistakes will confuse the user and lead to an erroneous interaction flow. The automatic detection of error potentials in electroencephalographic data recorded from a user allows the system to detect such states of confusion and automatically bring the interaction back on track. In this work, we describe the design of such a selfcorrecting gesture interface, implement different strategies to deal with detected errors, use a simulation approach to analyze performance and costs of those strategies and execute a user study to evaluate user satisfaction. We show that selfcorrection significantly improves gesture recognition accuracy at lower costs and with higher acceptance than manual correction.

A Review of Error-Related Potential-Based Brain–Computer Interfaces for Motor Impaired People

“…Gestures of such patients are often misclassified due to weak muscle control; nevertheless, the ErrP signal can assist in these gesture-based human-computer interactions (HCI). Recently reported studies have suggested applications of the hybrid-BCI systems that combine the use of muscle activities for sending commands and ErrP signals as a feedback for interpretation of such commands [23], [113].…”

“…Participants gesture movements were combined with the ErrP feedback to accomplish a user-specific recalibration that improved the gesture recognition rates by 6.4%. Putze et al [113] extended the work further and developed an inertial-measurement unit (IMU) based gesture recognition interface that can recognize six different classes. They mainly compared the performance of three ErrP-based correction strategies: Manual, Reprompt, and 2nd-best, and showed that self-correction-based strategies improve the efficacy of the gesture recognition system and have greater acceptance among participants compared to the manual correction.…”

“…As an example, Dal Seno et al [105] reported little to no improvement in their P300 speller with the ErrP integration due to the low ErrP classification rate. Also, Putze et al [113] reported that participants found the automatic correction strategy more confusing and unpredictable in comparison to the Reprompt strategy. Furthermore, Margaux et al [8] reported that a majority of users that were even having reasonable ErrP detection rates preferred the use of P300 speller without corrections because they perceived no benefit from it.…”

“…In detecting the ErrP signal, various strategies are employed to handle the erroneous condition, such as Reprompt, Selective Reprompt and 2nd-best [113]; however, all of these strategies suffer from troublesome false positives. Utilization of the iterative ErrP can be a potential approach in reducing false positives [103].…”

Framework for Electroencephalography-based Evaluation of User Experience

“…As in [35], we filtered EEG signals in the delta (1-3 Hz), theta (4-6 Hz), alpha (7-13 Hz), beta (14)(15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25) and gamma (26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36)(37)(38)(39)(40) bands. To reduce features dimensionality, we used for each band a set of Common Spatial Patterns (CSP) spatial filters.…”