The effect of multimodal and enriched feedback on SMR-BCI performance

Sollfrank, Teresa; Ramsay, Andrew; Perdikis, Serafeim; Williamson, John; Murray-Smith, Roderick; Leeb, Robert; Millán, José del R.; Kübler, Andrea

doi:10.1016/j.clinph.2015.06.004

Cited by 51 publications

(49 citation statements)

References 50 publications

(63 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent results show that complementary tactile feedback can enhance motor imagery BCI performances for instance [65, 66], which confirms this is a promising direction to explore. Designing an explanatory feedback for BCI is currently very challenging given the lack of fundamental knowledge on motor imagery and on BCI feedback [67]. For instance, why mental commands are sometimes erroneously recognized?…”

Section: Feedback and Trainingmentioning

confidence: 99%

Heading for new shores! Overcoming pitfalls in BCI design

Chavarriaga

Fried-Oken²,

Kleih

et al. 2016

Brain-Computer Interfaces

View full text Add to dashboard Cite

Research in brain-computer interfaces has achieved impressive progress towards implementing assistive technologies for restoration or substitution of lost motor capabilities, as well as supporting technologies for able-bodied subjects. Notwithstanding this progress, effective translation of these interfaces from proof-of concept prototypes into reliable applications remains elusive. As a matter of fact, most of the current BCI systems cannot be used independently for long periods of time by their intended end-users. Multiple factors that impair achieving this goal have already been identified. However, it is not clear how do they affect the overall BCI performance or how they should be tackled. This is worsened by the publication bias where only positive results are disseminated, preventing the research community from learning from its errors. This paper is the result of a workshop held at the 6th International BCI meeting in Asilomar. We summarize here the discussion on concrete research avenues and guidelines that may help overcoming common pitfalls and make BCIs become a useful alternative communication device.

show abstract

Section: Feedback and Trainingmentioning

confidence: 99%

Heading for new shores! Overcoming pitfalls in BCI design

Chavarriaga

Fried-Oken²,

Kleih

et al. 2016

Brain-Computer Interfaces

View full text Add to dashboard Cite

show abstract

“…We argue that for EEG-fMRI-NF, EEG and fMRI integration can be done directly at the feedback level and that this already has strong implication. Several unimodal NF/BCI studies have investigated the effects of feedback presentation as for example (Krause et al, 2017;Stoeckel et al, 2014;Sollfrank et al, 2016;Darvishi et al, 2017;Ono et al, 2014;Jeunet et al, 2015b;Kaufmann and Williamson, 2011) which is a central issue in NF/BCI design.…”

Section: Introductionmentioning

confidence: 99%

Learning 2-in-1: Towards Integrated EEG-fMRI-Neurofeedback

Perronnet

Lécuyer

Mano

et al. 2018

Preprint

View full text Add to dashboard Cite

Neurofeedback (NF) allows to exert self-regulation over specific aspects of one's own brain function by returning information extracted in real-time from brain activity measures. These measures are usually acquired from a single modality, most commonly EEG or fMRI. EEG-fMRI-neurofeedback (EEG-fMRI-NF) is a new NF approach that consists of providing a NF based simultaneously on EEG and fMRI. By exploiting the complementarity of these two modalities, EEG-fMRI-NF opens a spectrum of possibilities for defining bimodal NF targets that could be more ro-bust, flexible and effective than unimodal ones. However facing a greater amount of information, the question arises of how to represent the EEG and fMRI features simultaneously in order to allow the subject to achieve better self-regulation. In this work, we propose that the EEG and fMRI features should be represented in a single bimodal feedback, which we refer to as integrated feedback. We then introduce two integrated feedback strategies for EEG-fMRI-NF and compare their early effects on a motor imagery task with a between-group design. The first group (BI DIM, n=10) was shown a two-dimensional (2D) feedback in which each dimension de-picted the information from one modality. The second group (UNI DIM, n=10) was shown a one-dimensional (1D) feedback that integrated both types of information even further by merging them into one. Online fMRI activations were significantly higher in the UNI DIM group than in the BI DIM group, which suggests that the 1D feedback is easier to control than the 2D feedback. However when looking at posthoc activation levels, the difference of fMRI activation levels between the NF runs and the preliminary motor imagery run without NF was more significant in the 2D group. Moreover, subjects from the BI DIM group produced more specific BOLD activations with a notably stronger activation in the right superior parietal lobule (BI DIM > UNI DIM, p < 0.001, uncorrected). These results suggest that the 2D feedback encourages subjects to explore their strategies to recruit more spe-cific brain patterns. To summarize, our study shows that 1D and 2D integrated feedbacks are effective but also appear to be complementary and could therefore be used in combination in a bimodal NF training program. Altogether, our study paves the way to novel integrated feedback strategies for the development of flexible and effective bimodal NF paradigms making the most of the bimodal information and better suited to clinical applications.

show abstract

“…Nonetheless, further factors may affect the difficulty of the training: the challenge of achieving a sense of cognitive and internal control (Burde and Blankertz, 2006; Wood et al, 2014), the appropriate processing of cues to reduce impairments in mental chronometry (Liepert et al, 2012), and to increase the quality of motor imagery (Heremans et al, 2009, 2012), the specific sensory impairments of patients and their interaction with the feedback modality (e.g., visual, haptic, auditory) (Nijboer et al, 2008; Gomez-Rodriguez et al, 2011; Parker et al, 2011; Sollfrank et al, 2015), or the repetitive and fatiguing nature of training (Lee et al, 1991; Page et al, 2011). Dealing with these aspects by proper instructional design is more important for restorative than for assistive approaches (Lotte et al, 2013).…”

Section: Difficulty Of Adaptationmentioning

confidence: 99%

Constraints and Adaptation of Closed-Loop Neuroprosthetics for Functional Restoration

Bauer

Gharabaghi

2017

Front. Neurosci.

View full text Add to dashboard Cite

Closed-loop neuroprosthetics aim to compensate for lost function, e.g., by controlling external devices such as prostheses or wheelchairs. Such assistive approaches seek to maximize speed and classification accuracy for high-dimensional control. More recent approaches use similar technology, but aim to restore lost motor function in the long term. To achieve this goal, restorative neuroprosthetics attempt to facilitate motor re-learning and to strengthen damaged and/or alternative neural connections on the basis of neurofeedback training within rehabilitative environments. Such a restorative approach requires reinforcement learning of self-modulated brain activity which is considered to be beneficial for functional rehabilitation, e.g., improvement of β-power modulation over sensorimotor areas for post-stroke movement restoration. Patients with motor impairments, however, may also have a compromised ability for motor task-related regulation of the targeted brain activity. This would affect the estimation of feature weights and hence the classification accuracy of the feedback device. This, in turn, can frustrate the patients and compromise their motor learning. Furthermore, the feedback training may even become erroneous when unconstrained classifier adaptation—which is often used in assistive approaches—is also applied in this rehabilitation context. In conclusion, the conceptual switch from assistance toward restoration necessitates a methodological paradigm shift from classification accuracy toward instructional efficiency. Furthermore, a constrained feature space, a priori regularized feature weights, and difficulty adaptation present key elements of restorative brain interfaces. These factors need, therefore, to be addressed within a therapeutic framework to facilitate reinforcement learning of brain self-regulation for restorative purposes.

show abstract

The effect of multimodal and enriched feedback on SMR-BCI performance

Cited by 51 publications

References 50 publications

Heading for new shores! Overcoming pitfalls in BCI design

Heading for new shores! Overcoming pitfalls in BCI design

Learning 2-in-1: Towards Integrated EEG-fMRI-Neurofeedback

Constraints and Adaptation of Closed-Loop Neuroprosthetics for Functional Restoration

Contact Info

Product

Resources

About