A method for improving semantic segmentation using thermographic images in infants

Abstract

Background Regulation of temperature is clinically important in the care of neonates because it has a significant impact on prognosis. Although probes that make contact with the skin are widely used to monitor temperature and provide spot central and peripheral temperature information, they do not provide details of the temperature distribution around the body. Although it is possible to obtain detailed temperature distributions using multiple probes, this is not clinically practical. Thermographic techniques have been reported for measurement of temperature distribution in infants. However, as these methods require manual selection of the regions of interest (ROIs), they are not suitable for introduction into clinical settings in hospitals. Here, we describe a method for segmentation of thermal images that enables continuous quantitative contactless monitoring of the temperature distribution over the whole body of neonates. Methods The semantic segmentation method, U-Net, was applied to thermal images of infants. The optimal combination of Weight Normalization, Group Normalization, and Flexible Rectified Linear Unit (FReLU) was evaluated. U-Net Generative Adversarial Network (U-Net GAN) was applied to thermal images, and a Self-Attention (SA) module was finally applied to U-Net GAN (U-Net GAN + SA) to improve precision. The semantic segmentation performance of these methods was evaluated. Results The optimal semantic segmentation performance was obtained with application of FReLU and Group Normalization to U-Net, showing accuracy of 92.9% and Mean Intersection over Union (mIoU) of 64.5%. U-Net GAN improved the performance, yielding accuracy of 93.3% and mIoU of 66.9%, and U-Net GAN + SA showed further improvement with accuracy of 93.5% and mIoU of 70.4%. Conclusions FReLU and Group Normalization are appropriate semantic segmentation methods for application to neonatal thermal images. U-Net GAN and U-Net GAN + SA significantly improved the mIoU of segmentation.

A non-invasive method to determine core temperature for cats and dogs using surface temperatures based on machine learning

“…In another study, rectal temperatures of rabbits were predicted using measurements obtained with advanced infrared cameras [ 23 ]. There were also reports for human infants, among them, Lyra et al combined deep learning-based algorithms and camera modalities to real-time monitor the temperature of neonates [ 25 ]; Yaeger et al developed a natural language processing algorithm to identify febrile infants [ 26 ]; Asano et al applied a semantic segmentation method to thermal images, which makes it possible to monitor the temperature distribution over the whole body of infants [ 27 ]. Different to those approaches in predicting core temperatures, our method focused on companion animals using convenient operations and equipment.…”

Multi-modal body part segmentation of infants using deep learning

“…In contrast to Asano et al [ 21 ] we also used transfer learning when training the neural network on LWIR images. Even though, in this case, transfer learning was performed with a grayscaled version of the CIHP Dataset, it also improved the results significantly.…”

“…Asano et al combined a U-Net with a Generative Adversarial Network (GAN) and Self-Attention (SA) for body part segmentation in long-wave infrared (LWIR) images [ 21 ]. Opposite to Hoog Antink et al, they did not pre-train their neural network but only used a dataset containing 400 images of neonates.…”

Infrared Thermography with High Accuracy in a Neonatal Incubator

“…Hence, we reported a method for the segmentation of thermal images that enables continuous non-invasive monitoring of the body temperature distribution over the whole body of neonates. 6 The newly established equation (10) will give new evidence in this field combined with this segmentation method.…”