Emotion Recognition Using Physiological Signals: Laboratory vs. Wearable Sensors

Ragot, Martin; Martin, Nicolas; Em, Sonia; Pallamin, Nico; Diverrez, Jean-Marc

doi:10.1007/978-3-319-60639-2_2

Cited by 101 publications

(71 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, this result may suggest that higher stimulations of the sympathetic nervous system are needed for observing responses in wrist SC. This interpretation is in line with preliminary studies on the E4 SC signal, suggesting its relatively high sensitivity to emotional stress (Ollander et al, ; Ragot et al, ).…”

Section: Discussionsupporting

confidence: 89%

“…On the other hand, most PPG sensors have shown a satisfactory accuracy only in restricted conditions, such as at rest in absence of motion (Schäfer & Vagedes, 2013). The accuracy of the E4 wristband has been assessed by only a few preliminary studies, mainly focused on stress and emotion recognition (e.g., Ragot, Martin, Em, Pallamin, & Diverrez, 2017). Two studies focused on the PPG signal and suggested satisfactory data quality (McCarthy, Pradhan, Redpath, & Adler, 2016) and acceptable measurement error in mean HR and some HRV measures (Pietilä et al, 2017).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Stressing the accuracy: Wrist‐worn wearable sensor validation over different conditions

et al. 2019

View full text Add to dashboard Cite

Wearable sensors are promising instruments for conducting both laboratory and ambulatory research in psychophysiology. However, scholars should be aware of their measurement error and the conditions in which accuracy is achieved. This study aimed to assess the accuracy of a wearable sensor designed for research purposes, the E4 wristband (Empatica, Milan, Italy), in measuring heart rate (HR), heart rate variability (HRV), and skin conductance (SC) over five laboratory conditions widely used in stress reactivity research (seated rest, paced breathing, orthostatic, Stroop, speech task) and two ecological conditions (slow walking, keyboard typing). Forty healthy participants concurrently wore the wristband and two gold standard measurement systems (i.e., electrocardiography and finger SC sensor). The wristband accuracy was determined by evaluating the signal quality and the correlations with and the Bland‐Altman plots against gold standard‐derived measurements. Moreover, exploratory analyses were performed to assess predictors of measurement error. Mean HR measures showed the best accuracy over all conditions. HRV measures showed satisfactory accuracy in seated rest, paced breathing, and recovery conditions but not in dynamic conditions, including speaking. Accuracy was diminished by wrist movements, cognitive and emotional stress, nonstationarity, and larger wrist circumferences. Wrist SC measures showed neither correlation nor visual resemblance with finger SC signal, suggesting that the two sites may reflect different phenomena. Future studies are needed to assess the responsivity of wrist SC to emotional and cognitive stress. Limitations and implications for laboratory and ambulatory research are discussed.

show abstract

Section: Discussionsupporting

confidence: 89%

mentioning

confidence: 99%

Stressing the accuracy: Wrist‐worn wearable sensor validation over different conditions

et al. 2019

View full text Add to dashboard Cite

show abstract

“…One study combined GSR and PPG, collected by a Shimmer3 sensor [24], to classify High/Low valence and arousal [25]. In another study, unimodal PPG data collected by an expensive wrist-worn wearable device (Empatica E4 [26]) was compared with data collected by a laboratory sensor (Biopac MP150 [27]) [28]. Although these studies represent an important step towards real-world applicability, we are not aware of any studies that have explored emotion recognition using IBI PPG data of the type collected by affordable consumer fitness trackers.…”

Section: B Emotion Prediction From Unimodal Ppg Datamentioning

confidence: 99%

End-To-End Prediction of Emotion from Heartbeat Data Collected by a Consumer Fitness Tracker

Harper

Southern

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

Automatic detection of emotion has the potential to revolutionize mental health and wellbeing. Recent work has been successful in predicting affect from unimodal electrocardiogram (ECG) data. However, to be immediately relevant for real-world applications, physiology-based emotion detection must make use of ubiquitous photoplethysmogram (PPG) data collected by affordable consumer fitness trackers. Additionally, applications of emotion detection in healthcare settings will require some measure of uncertainty over model predictions. We present here a Bayesian deep learning model for end-to-end classification of emotional valence, using only the unimodal heartbeat time series collected by a consumer fitness tracker (Garmin Vívosmart 3). We collected a new dataset for this task, and report a peak F1 score of 0.7. This demonstrates a practical relevance of physiology-based emotion detection 'in the wild' today.

show abstract

“…For example, it requires more effort and time for researchers to place the electrodes on participants’ arms, face, and fingers, than to wear the wearable device. Importantly, they can measure psychophysiological signals continuously while people go about their daily lives (Garbarino, Lai, Bender, Picard, & Tognetti, ; Ragot, Martin, Em, Pallamin, & Diverrez, ).…”

Section: Introductionmentioning

confidence: 99%

“…However, skin conductivity (SC) signals again showed low correlation between the two types of devices. Recently, Ragot and colleagues () compared physiological data from the E4 wristband to those of laboratory equipment (Biopac) during an emotion recognition experiment (using emotional pictures). These authors utilized machine learning models for the analysis of their psychophysiological data and found that the mean responses of physiological data (for cardiac features of HR, AVNN, SDNN, RMSSD, pNN50, LF, HF, RF) yielded from the wearable device, were similar to those of the laboratory equipment.…”

Section: Introductionmentioning

confidence: 99%

Comparing apples and oranges or different types of citrus fruits? Using wearable versus stationary devices to analyze psychophysiological data

et al. 2020

View full text Add to dashboard Cite

Wearable devices capable of capturing psychophysiological signals are popular.However, such devices have, yet, to be established in experimental and clinical research. This study, therefore, compared psychophysiological data (skin conductance level (SCL), heart rate (HR), and heart rate variability (HRV)) captured with a wearable device (Microsoft band 2) to those of a stationary device (Biopac MP150), in an experimental pain induction paradigm. Additionally, the present study aimed to compare two analytical techniques of HRV psychophysiological data: traditional (i.e., peaks are detected and manually checked) versus automated analysis using Python programs. Forty-three university students (86% female; Mage = 21.37 years) participated in the cold pressor pain induction task. Results showed that the majority of the correlations between the two devices for the mean HR were significant and strong (rs > .80) both during baseline and experimental phases. For the time-domain measure of mean RR (function of autonomic influences) of HRV, the correlations between the two devices at baseline were almost perfect (rs = .99), whereas at the experimental phase were significantly strong (rs > .74). However, no significant correlations were found for mean SCL (p> .05). Additionally, automated analysis led to similar features for HRV stationary data as the traditional analysis. Implications for data collection include the establishment of a methodology to compare stationary to mobile devices and a new, more cost efficient way of collecting psychophysiological data. Implications for data analysis include analyzing the data faster, with less effort and allowing for large amounts of data to be recorded. K E Y W O R D SBland-Altman plots, heart rate variability, psychophysiological data, root mean square error, skin conductance response, stationary equipment, wearable device

show abstract

Emotion Recognition Using Physiological Signals: Laboratory vs. Wearable Sensors

Cited by 101 publications

References 13 publications

Stressing the accuracy: Wrist‐worn wearable sensor validation over different conditions

Stressing the accuracy: Wrist‐worn wearable sensor validation over different conditions

End-To-End Prediction of Emotion from Heartbeat Data Collected by a Consumer Fitness Tracker

Comparing apples and oranges or different types of citrus fruits? Using wearable versus stationary devices to analyze psychophysiological data

Contact Info

Product

Resources

About