Implementation of Wavelet Analysis on Thermal Images for Affective States Recognition of Children With Autism Spectrum Disorder

Abstract

Children with Autism Spectrum Disorder are identified as a group of people who has difficulties in socio-emotional interaction. Most of them lack the proper context in producing social response through facial expression and speech. Since emotion is the key for effective social interaction, it is justifiably vital for them to comprehend the correct emotion expressions and recognitions. Emotion is a type of affective states and can be detected through physical reaction and physiological signals. In general, recognition of affective states from physical reaction such as facial expression and speech for autistic children is often unpredictable. Hence, an alternative method of identifying the affective states through physiological signals is proposed. Though considered non-invasive, most of the current recognition methods require sensors to be patched on to the skin body to measure the signals. This would most likely cause discomfort to the children and mask their "true" affective states. The study proposed the use of thermal imaging modality as a passive medium to analyze the physiological signals associated with the affective states nonobtrusively. The study hypothesized that, the impact of cutaneous temperature changes due to the pulsating blood flow in the blood vessels at the frontal face area measured from the modality could have a direct impact to the different affective states of autistic children. A structured experimental setup was designed to measure thermal imaging data generated from different affective state expressions induced using different sets of audio-video stimuli. A wavelet-based technique for pattern detection in time series was deployed to spot the changes measured from the region of interest. In the study, the affective state model for typical developing children aged between 5 and 9 years old was used as the baseline to evaluate the performance of the affective state classifier for autistic children. The results from the classifier showed the efficacy of the technique and accorded good performance of classification accuracy at 88% in identifying the affective states of autistic children. The results were momentous in distinguishing basic affective states and the information could provide a more effective response towards improving social-emotion interaction amongst the autistic children. INDEX TERMS Autism, Affective States, Facial Skin Temperatures, Thermoregulation, Thermal Images, Wavelet I. INTRODUCTION F OR decades, numbers of research have been conducted in the field of affective state recognition, using numerous modalities and exploiting various features from the signal generated by the Autonomous Nervous System (ANS). ANS together with hypothalamus, regulates the blood pressure, breathing, pulse and arousal in response to different emotional features. Seeing the big potentials of affective state

Interpretable Cross-Subject EEG-Based Emotion Recognition Using Channel-Wise Features

“…For instance, in the medical field of psychiatry, detected emotional states of patients help identify those at a high risk of emotional disorders and depression [ 2 ]. Thus, there has been much research on emotion recognition using facial expressions [ 3 ], thermography [ 4 ], motion capture system [ 5 ], text [ 6 ], and speech [ 7 ]. However, these modes are difficult for representing people’s true feelings because they are sensitive to subject-specific variability.…”

Automatic Emotion Recognition in Children with Autism: A Systematic Literature Review

“…The emotions were grouped regarding their similarity. [21,24] attention (1): [25] bored (1): [25] fear-related 16 fear ( 12): [17,[26][27][28][29][30][31][32][33][34][35][36] anxiety (3): [20,22,25] trepidation (1): [37] joy-related 21 joy ( 5): [21,[26][27][28][29] liking (3): [19,20,22] happiness (12): [17,24,25,[30][31][32][33][34][35][37][38][39] smile (1): [40] relaxation (1): [37] There are studies that treat attention, engagement, involvement, and boredom as emotional states. They are more like mental states; however, they are emotion-related and were used in the context of emotion recognition in the analyzed papers.…”

“…joy 21 [17,[19][20][21][22][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39] anger 12 [17,[27][28][29][30][31][32][33]35,36,38,39] fear 12 [17,[26][27][28][29][30][31][32][33][34][35][36] disgust 7 [17,25,27,29,32,35,36] sadness 16 [17,…”

“…Number of Papers Papers joy 3 [19,20,22] joy, neutral 2 [21,24] joy, sadness 2 [21,37] joy, fear, sadness 2 [26,34] joy, disgust, sadness, neutral 1 [25] joy, anger, sadness, neutral 2 [38,39] joy, anger, fear, sadness, neutral 2 [28,30] joy, anger, fear, disgust, sadness, surprise 1 [17] joy, anger, fear, sadness, surprise, neutral 2 [31,33] joy, anger, fear, disgust, sadness, neutral 1 [36] joy, anger, fear, disgust, sadness, surprise, netural 4 [27,29,32,35] In some papers, only two-class recognition was performed. In those papers, joy is recognized as a class combined with no-joy in three papers, with a neutral state in two papers, and with sadness in two papers.…”

“…An interesting study combines joy and sadness with hunger, trepidation, relaxation, and nervous states [37]. Regarding the dimensional model of emotions, it is represented in seven papers-valence and arousal are always recognized together [17,23,34,35,42,43]. Some studies added a dimension of dominance [18] or analyzed arousal only [44], but they were excluded from the quantitative analysis as they were preliminary and provided no validation with children with autism.…”