Thermal Imaging Based Affective Computing for Educational Robot

Abstract

Over the recent years, Social Robots (SRs) have become more and more prominent in everyday human lives. The main goal of a SR is to interact and communicate with human by following social behaviors and affective interaction. However, they still encounter significant limitations in pursuing a natural interaction, mainly due to their hard task of recognizing and understanding human emotions thus ensuring an appropriate response. The aim of this study was to enrich the SR with affective computing capability and real time assessment of the interlocutor’s psychophysiological state, by means of computational psychophysiology based on thermal infrared imaging.

Thermal Infrared Imaging-Based Affective Computing and Its Application to Facilitate Human Robot Interaction: A Review

“…To capture thermal variation in the children's faces, they used the mobile thermal camera Therm-App. A similar set-up has been reported in Filippini et al [87]. The authors installed a mobile thermal IR imaging system, the FLIR ONE, on the head of a social robot.…”

“…In this regard, signal processing techniques need to be chosen based on their efficiency in terms of computational load to allow acceptable performance for real-time processing. In Goulart et al and Filippini et al, the tracking algorithm in thermal images relied on the visible images [86,87]. Thereby, the first phase consisted on the camera calibration done through a synchronous acquisition between visual and thermal images and using a checkerboard whose details were clearly detectable by both the visible spectrum and the thermal camera.…”

“…In Goulart et al, the Viola-Jones algorithm was used for visible ROIs detection, and then these ROIs were transferred to the corresponding thermal camera frame through a homography matrix [86]. In Filippini et al, the authors used an object detector based on the histogram of oriented gradients (HOG) for face localization in visible images [87]. The extraction of landmarks was based, instead, on a regression tree ensemble algorithm.…”

“…Although these studies were very ambitious and fascinating, they were still carried out in constrained laboratory settings, probably implying a not entirely free interaction. To the state of the art, the only two studies performed so far that have concerned HRI applications in a out-of-laboratory context are those reported in Section 7 [86,87].…”

15th International Workshop on Advanced Infrared Technology and Applications (AITA)

“…In view of the excellent talks delivered by young scientists, this year the commission decided to give an ex aequo prize to Chiara Filippini for the paper "Thermal imaging based affective computing for educational robot" [1] and to Klaudia Hackiewicz for the paper "InAs/GaSb superlattice based mid-infrared interband cascade photodetectors grown on both native GaSb and lattice-mismatched GaAs substrates" [2].…”