A Novel Audiovisual Brain-Computer Interface and Its Application in Awareness Detection

Wang, Fei; He, Yanbin; Pan, Jiahui; Xie, Qiuyou; Yu, Rong; Zhang, Rui; Li, Yuanqing

doi:10.1038/srep09962

Cited by 34 publications

(38 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous studies mainly focused on using BCI to detect the awareness of DOC patients [13][14][15]. These studies determined whether patients were aware by verifying whether the patients followed commands.…”

Section: Discussionmentioning

confidence: 99%

“…In our previous study [15], we proposed an audiovisual BCI system for awareness detection in patients with DOC based on semantically congruent audiovisual stimuli, including visual and spoken numbers. Our experimental results for healthy subjects showed that the audiovisual BCI outperformed the corresponding visual-only and auditory-only BCIs, whereas the experimental results for the patients demonstrated satisfactory performance of the system in awareness detection.…”

Section: Discussionmentioning

confidence: 99%

“…Furthermore, we need to develop novel BCIs that are suitable for DOC patients. Previous studies have validated the effectiveness of audiovisual BCIs [15,18]. In this study, we used an audiovisual paradigm to improve BCI performance.…”

Section: Introductionmentioning

confidence: 99%

“…Several BCI systems have recently been applied in awareness detection for patients with DOC [13][14][15][16]. Cruse et al accessed 16 VS patients by having them complete a motor imagery (MI) task in which they were instructed to imagine movements of their right hand and toes in response to commands [13].…”

Section: Introductionmentioning

confidence: 99%

“…They successfully demonstrated command following in three patients (one VS, one MCS and one LIS). In our previous study, we applied an audiovisual BCI system to detect the awareness of seven DOC patients; the patients were instructed to selectively attend a target of two different number stimuli [15]. Five of the patients exhibited command following.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Enhancing clinical communication assessments using an audiovisual BCI for patients with disorders of consciousness

Wang

et al. 2017

J. Neural Eng.

Self Cite

View full text Add to dashboard Cite

Objective. The JFK coma recovery scale-revised (JFK CRS-R), a behavioral observation scale, is widely used in the clinical diagnosis/assessment of patients with disorders of consciousness (DOC). However, the JFK CRS-R is associated with a high rate of misdiagnosis (approximately 40%) because DOC patients cannot provide sufficient behavioral responses. A brain–computer interface (BCI) that detects command/intention-specific changes in electroencephalography (EEG) signals without the need for behavioral expression may provide an alternative method. Approach. In this paper, we proposed an audiovisual BCI communication system based on audiovisual ‘yes’ and ‘no’ stimuli to supplement the JFK CRS-R for assessing the communication ability of DOC patients. Specifically, patients were given situation-orientation questions as in the JFK CRS-R and instructed to select the answers using the BCI. Main results. Thirteen patients (eight vegetative state (VS) and five minimally conscious state (MCS)) participated in our experiments involving both the BCI- and JFK CRS-R-based assessments. One MCS patient who received a score of 1 in the JFK CRS-R achieved an accuracy of 86.5% in the BCI-based assessment. Seven patients (four VS and three MCS) obtained unresponsive results in the JFK CRS-R-based assessment but responsive results in the BCI-based assessment, and 4 of those later improved scores in the JFK CRS-R-based assessment. Five patients (four VS and one MCS) obtained usresponsive results in both assessments. Significance. The experimental results indicated that the audiovisual BCI could provide more sensitive results than the JFK CRS-R and therefore supplement the JFK CRS-R.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Enhancing clinical communication assessments using an audiovisual BCI for patients with disorders of consciousness

Wang

et al. 2017

J. Neural Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

Revisiting Neurological Aspects of Relevance: An EEG Study

Pinkosova¹,

McGeown²,

Moshfeghi³

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Relevance is a key topic in Information Retrieval (IR). It indicates how well the information retrieved by the search engine meets the user's information need (IN). Despite research advances in the past decades, the use of brain imaging techniques to investigate complex cognitive processes underpinning relevance is relatively recent, yet has provided valuable insight to better understanding this complex human notion. However, past electrophysiological studies have mainly employed an event-related potential (ERP) component-driven approach. While this approach is effective in exploring known phenomena, it might overlook the key cognitive aspects that significantly contribute to unexplored and complex cognitive processes such as relevance assessment formation. This paper, therefore, aims to study the relevance assessment phenomena using a data-driven approach. To do so, we measured the neural activity of twenty-five participants using electroencephalography (EEG). In particular, the neural activity was recorded in response to participants' binary relevance assessment (relevant vs. non-relevant) within the context of a Question Answering (Q/A) Task. We found significant variation associated with the user's subjective assessment of relevant and non-relevant information within the EEG signals associated with P300/CPP, N400 and, LPC components, which confirms the findings of previous studies. Additionally, the data-driven approach revealed neural differences associated with the previously not reported P100 component, which might play important role in early selective attention and working memory modulation. Our findings are an important step towards a better understanding of the cognitive mechanisms involved in relevance assessment and more effective IR systems.

show abstract