Multimodal Stress Detection from Multiple Assessments

Abstract

International audienceStress is a complex phenomenon that impacts the body and the mind at several levels. It has been studied for more than a century from different perspectives, which result in different definitions and different ways to assess the presence of stress. This paper introduces a methodology for analyzing multimodal stress detection results by taking into account the variety of stress assessments. As a first step, we have collected video, depth and physiological data from 25 subjects in a stressful situation: a socially evaluated mental arithmetic test. As a second step, we have acquired 3 different assessments of stress: self-assessment, assessments from external observers and assessment from a physiology expert. Finally, we extract 101 behavioural and physiological features and evaluate their predictive power for the 3 collected assessments using a classification task. Using multimodal features, we obtain average F1 scores up to 0.85. By investigating the composition of the best selected feature subsets and the individual feature classification performances, we show that several features provide valuable information for the classification of the 3 assessments: features related to body movement, blood volume pulse and heart rate. From a methodological point of view, we argue that a multiple assessment approach provide more robust results

Determination of Optimal Heart Rate Variability Features Based on SVM-Recursive Feature Elimination for Cumulative Stress Monitoring Using ECG Sensor

“…Previous studies have similarly proposed optimal HRV features for stress monitoring [10,11,12,13,14,15,16]. However, most of those studies considered a singular exposure to a specific stressor.…”

“…In several studies, a feature selection approach such as the filter [13,14] and wrapper method [13,15,16] was used for determining optimal HRV features. Aigrain et al [15] evaluated the predictive power of various multimodal features by investigating the composition of the best feature subset and showed that the HR values (maximum and variation) and the amplitude of HR (maximum, mean, and variation) provided the best prediction among features related to ECGs.…”

“…Aigrain et al [15] evaluated the predictive power of various multimodal features by investigating the composition of the best feature subset and showed that the HR values (maximum and variation) and the amplitude of HR (maximum, mean, and variation) provided the best prediction among features related to ECGs. Ollander et al [13] used both the filter and wrapper method to search multimodal features related to stress for detecting driving stress.…”

Toward automatic detection of acute stress: Relevant nonverbal behaviors and impact of personality traits

“…3e). Motion history images (MHI) have been proven to be very robust in detecting motion and is widely employed by various research groups for action recognition and motion analysis [9], [10]. Other processing techniques such as optical flow or dense face tracking will be considered in future work.…”

“…However, most of these systems require invasive sensors that may themselves induce stress in participants. Recent advances in computer vision have led to the design of noninvasive systems capable of estimating user stress from the video analysis of facial expressions, gestures, postures, gaze and blinking, and head movements [9], [10], [11], [12]. Tasks and situations commonly used for the design and evaluation of these systems include mathematical problem solving, public speaking and global overwhelming workload.…”