Real-Time Multi-Modal Estimation of Dynamically Evoked Emotions Using EEG, Heart Rate and Galvanic Skin Response

Abstract

Emotion estimation systems based on brain and physiological signals such as electro encephalography (EEG), blood-volume pressure (BVP), and galvanic skin response (GSR) are gaining special attention in recent years due to the possibilities they offer. The field of human–robot interactions (HRIs) could benefit from a broadened understanding of the brain and physiological emotion encoding, together with the use of lightweight software and cheap wearable devices, and thus improve the capabilities of robots to fully engage with the users emotional reactions. In this paper, a previously developed methodology for real-time emotion estimation aimed for its use in the field of HRI is tested under realistic circumstances using a self-generated database created using dynamically evoked emotions. Other state-of-the-art, real-time approaches address emotion estimation using constant stimuli to facilitate the analysis of the evoked responses, remaining far from real scenarios since emotions are dynamically evoked. The proposed approach studies the feasibility of the emotion estimation methodology previously developed, under an experimentation paradigm that imitates a more realistic scenario involving dynamically evoked emotions by using a dramatic film as the experimental paradigm. The emotion estimation methodology has proved to perform on real-time constraints while maintaining high accuracy on emotion estimation when using the self-produced dynamically evoked emotions multi-signal database.

Advanced Bioelectrical Signal Processing Methods: Past, Present and Future Approach—Part II: Brain Signals

“…One of the most commonly applied methods is EEG monitoring, which can be combined with other measurement methods [ 28 , 285 , 286 , 287 , 288 , 289 , 290 , 291 ]: brain imaging techniques, such as MRI and fNIRS; biological signals, such as ECG and EMG; brain stimulation techniques, such as trans-cranial magnetic stimulation (TMS) and trans-cranial direct current stimulation (tDCS). Nevertheless, the multi-modal neurological imaging and/or monitoring is associated with specific signal processing and data analyses challenges, such as inter alia [ 20 , 292 , 293 , 294 , 295 , 296 , 297 ]: the EEG may obtain artifacts from other biological signals (such as EMG) or be distorted by the noise produced by accompanied devices for imaging (such as MRI) or stimulation (such as TMS). Therefore, signal processing and noise removal techniques play a particularly important role in this field; in terms of data analyses, fusing different neurological modalities to provide complimentary information poses a great issue.…”

Affective Robot Story-Telling Human-Robot Interaction: Exploratory Real-Time Emotion Estimation Analysis Using Facial Expressions and Physiological Signals

“…To record and collect the data, a self-produced software, GePHYCAM [30], is developed. This application looks forward to being accessible to the whole scientific community, providing a resourceful tool for human-behavior experimental paradigms, covering the following functionalities (see For the OpenBCI cap, four prefrontal and four temporal electrodes, {F3, T7, P7, F7, F4, T8, P8, F8}, were used as they proved to be the best areas for emotion estimation [9], [10], [31], [32]. The Empatica E4 wristband was placed on the non-dominant hand to avoid artifacts when users perform self-assessment ratings.…”

“…Historically, two main models have been developed which remain controversial: the discrete emotional model, which assumes that emotions are qualitatively differentiated neurophysiological responses [7] that produce independent emotional experiences; and the dimensional model, which assumes continuous quantified relationships among emotions [8]. For the present paper, the dimensional model has been chosen, but the assessment space is discrete, to simplify the number of labels for the users to choose from [9], [10].…”

Determination of dangerous driving behavior based on the use of information from wearable electronic devices

“…URL: https://www.who.int/violence_injury_prevention/road_safety_ status/2018/English-Summary-GSRRS2018.pdf (дата обращения: 20.03.2021). янии алкогольного опьянения и усталости 2 . Основной целью автоконцерна является улучшение навыков управления транспортным средством (ТС) с конечной целью приблизиться к нулевому числу дорожно-транспортных происшествий.…”

“…Так, например, регистрация значений электромиограммы и электроэнцефалограммы (ЭЭГ) водителя в режиме реального времени и измерение изменений кожно-гальванической реакции [2] позволяют устройствам носимой электроники определять степень усталости водителя, заблаговременно предупреждая его при помощи звукового и вибросигналов. Один из примеров таких устройств -наручный браслет со встроенным датчиком, производимый компанией Неироком [3] для мониторинга психофизиологического состояния человека путем обнаружения сонливости по результатам измерений кожно-гальванической реакции.…”