Instant Stress: Detection of Perceived Mental Stress Through Smartphone Photoplethysmography and Thermal Imaging (Preprint)

Abstract

Background: A smartphone is a promising tool for daily cardiovascular measurement and mental stress monitoring. Photoplethysmography (PPG) and lowcost thermography can be used to create cheap, convenient and mobile systems. However, to achieve robustness, a person has to remain still for several minutes while a measurement is being taken. This is very cumbersome, and limits the usage in applications such producing instant measurements of stress. Objective: We propose to use smartphone-based mobile PPG and thermal imaging to provide a fast binary measure of stress responses to an event using dynamical physiological changes which occur within 20 seconds of the event finishing. Methods: We propose a system that uses a smartphone and its physiological sensors to reliably and continuously measure over a short window of time a person's blood volume pulse, the time interval between heartbeats (R-R interval) and the 1D thermal signature of the nose tip. 17 healthy participants, involved in a series of stress-inducing mental activities, measured their physiological response to stress in the 20 second-window immediately following each activity. A 10-cm Visual Analogue Scale was used by them to self-report their level of mental stress. As a main labeling strategy, normalized K-means clustering is used to better treat interpersonal differences in ratings. By taking an array of the R-R intervals and thermal directionality as a low-level feature input, we mainly use an artificial neural network to enable the automatic feature learning and the machine learning inference process.

A Survey on Cross‐Virtuality Analytics

“…An alternative could be using thermal imaging cameras to measure cognitive load by recording temperature differences between a person's forehead and nose [AVD*17]. Cho and Bianchi‐Berthouze provide a comprehensive overview of thermal imaging technology for measuring physiological and affective needs [CBB19]. However, HMDs for MR usually occlude the forehead and nose from thermal cameras, limiting their use for XVA.…”