Hybrid EEG-EMG system to detect steering actions in car driving settings

Abstract

Understanding mental processes in complex human behaviour is a key issue in the context of driving, representing a milestone for developing user-centred assistive driving devices. Here we propose a hybrid method based on electroencephalographic (EEG) and electromyographic (EMG) signatures to distinguish left from right steering in driving scenarios. Twenty-four participants took part in the experiment consisting of recordings 128-channel EEG as well as EMG activity from deltoids and forearm extensors in non-ecological and ecological steering tasks. Specifically, we identified the EEG mu rhythm modulation correlates with motor preparation of self-paced steering actions in the non-ecological task, while the concurrent EMG activity of the left (right) deltoids correlates with right (left) steering. Consequently, we exploited the mu rhythm de-synchronization resulting from the non-ecological task to detect the steering side by means of a cross-correlation analysis with the ecological EMG signals. Results returned significant cross-correlation values showing the coupling between the non-ecological EEG feature and the muscular activity collected in ecological driving conditions. Moreover, such cross-correlation patterns discriminate left from right steering with an earlier dynamic with respect to the single EMG signal. This hybrid system overcomes the limitation of the EEG signals collected in ecological settings such as low reliability, accuracy and adaptability, thus adding to the EMG the characteristic predictive power of the cerebral data. These results are a proof of concept of how it is possible to complement different physiological signals to control the level of assistance needed by the driver.

Validation of a Novel Wearable Multistream Data Acquisition and Analysis System for Ergonomic Studies

“…The overall Biohub architecture, including its inter-stream synchronization, was "stress-tested" in the in-car BCI paradigm, which required the online application of advanced machine learning models for time series analysis. The successful findings reported in this test-case indicate the Biohub as a hybrid systemenabling the setting up of paradigms for measuring different psychophysiological variables, with better performance relative to conventional, unimodal BCI systems [20,39].…”

“…Then, the power spectral density (PSD) was calculated for each epoch in the range of [1,30] Hz window using the PSD multitaper function (Python MNE library [29,30]). Epochs with PSD > 60 dB in the [20,30] Hz window were rejected. In addition, each epoch was fitted with a linear fit model and rejected if the r squared linear correlation coefficient was larger than 0.85.…”

“…Due to the complexity of the underlying brain processes and the technological perspectives possibly impacting the market, driving is one of the most studied scenarios targeted by the field of neuroergonomics [14,15]. Several systems have been developed to decode brain activity anticipating salient actions during driving, such as braking [16,17] and steering [18][19][20]. Then, it is still critical to measure the brain and body activity of the driver, assess his/her mental state to prevent workload, fatigue and drowsiness [21], and decode neurophysiological temporal dynamics associated with movement preparation and execution.…”

Hybrid Systems to Boost EEG-Based Real-Time Action Decoding in Car Driving Scenarios

“…In the field of driving research, several studies addressed the issue of action detection and prediction based on the discrimination of different EEG features in simulated (Haufe et al, 2011;Gheorghe et al, 2013;Khaliliardali et al, 2015;Kim et al, 2015;Vecchiato et al, 2018Vecchiato et al, , 2020Vecchiato et al, , 2021 and real driving scenarios (Haufe et al, 2014;Zhang et al, 2015).…”

“…This result demonstrates that brain-related EEG signals significantly improve the overall decoding performance, showing that the significant contribution in predicting steering comes from non-brain-related signals, such as ocular and muscular components. Brain and muscular activities underlying steering behavior were also investigated with the final aim to increase the overall ecology of the experimental setting (Vecchiato et al, 2021). In particular, EEG feature predicting steering action and direction elicited by responding to traffic signs displayed on a computer screen was extracted and later exploited to increase the predictive power of the EMG collected in a more ecological steering task, such as a driving simulation.…”