Data-driven analysis of facial thermal responses and multimodal physiological consistency among subjects

Abstract

Facial infra-red imaging (IRI) is a contact-free technique complimenting the traditional psychophysiological measures to characterize physiological profile. However, its full potential in affective research is arguably unmet due to the analytical challenges it poses. Here we acquired facial IRI data, facial expressions and traditional physiological recordings (heart rate and skin conductance) from healthy human subjects whilst they viewed a 20-min-long unedited emotional movie. We present a novel application of motion correction and the results of spatial independent component analysis of the thermal data. Three distinct spatial components are recovered associated with the nose, the cheeks and respiration. We first benchmark this methodology against a traditional nose-tip region-of-interest based technique showing an expected similarity of signals extracted by these methods. We then show significant correlation of all the physiological responses across subjects, including the thermal signals, suggesting common dynamic shifts in emotional state induced by the movie. In sum, this study introduces an innovative approach to analyse facial IRI data and highlights the potential of thermal imaging to robustly capture emotion-related changes induced by ecological stimuli.

Autonomic Regulation of Facial Temperature during Stress: A Cross-Mapping Analysis

“…Firstly, IR images were co-registered using a motion correction framework based on optical flow [ 5 ]. We used RGB images to localize the position of the ROIs in the first frame of each IR recording.…”

“…In this study, we applied the cross-mapping procedure to investigate the nature of the facial thermal dynamics of healthy subjects in response to an external emotional stimulus. Based on the previous literature, we hypothesize that the main phenomena underlying a change in skin temperature are related to both sweat glands and cardiovascular dynamics [ 5 , 31 , 32 ]. Since these two phenomena are directly correlated to HRV and EDA, we applied the cross-mapping procedure to each couple of time series composed of a ANS correlate (i.e., HRV, EDAtonic) and a thermal signal, respectively denoted as X = (X , X , …, X ) and Y = (Y , Y , …, Y ).…”

“…The autonomic nervous system (ANS), which regulates the visceral responses to a change in mental state, seems to be responsible for the thermal modulation of the skin. Indeed, ANS regulates sweat production and influences the vasomotor activity of the subcutaneous vessels and consequently the dynamics of the blood circulation which produces the temperature variation [ 5 , 6 , 7 , 8 ]. A higher blood flux would cause a rise in local skin temperature, in contrast to perspiration which would cause a drop in temperature, due to the production of cold sweat.…”

ApeTI: A Thermal Image Dataset for Face and Nose Segmentation with Apes

“…Infrared thermography is a technique to measure heat radiation and, thus, the surface temperature of objects and living beings. In psychological research, thermal imaging is a valuable tool to measure skin temperature variation related to automatic nervous system activity changes [1][2][3]. Previous comparative psychological research used thermal imaging to study the emotional states of various animal species [2,4,5].…”

“…In psychological research, thermal imaging is a valuable tool to measure skin temperature variation related to automatic nervous system activity changes [1][2][3]. Previous comparative psychological research used thermal imaging to study the emotional states of various animal species [2,4,5]. Thermal imaging also finds its application in the human health sector [6][7][8] as in the industry [9].…”