ThermalGAN: Multimodal Color-to-Thermal Image Translation for Person Re-identification in Multispectral Dataset

Kniaz, V. V.; Knyaz, V. A.; Hladůvka, Jiřı́; Kropatsch, Walter G.; Mizginov, V. A.

doi:10.1007/978-3-030-11024-6_46

Cited by 92 publications

(68 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The VLD domain could not be easily replicated in the thermal domain due to the relatively small amount of training data available and the domain gap between thermal and visible light [72]. In recent times, the use of generative adversarial networks (GAN) [90] in arbitrary image-to-image translation applications has shown encouraging results [91]. Using an antagonistic game approach, GANs significantly increased the quality of image-to-image translation.…”

Section: Image Processing Techniques Associated To Thermal Imagingmentioning

confidence: 99%

Noncontact Sensing of Contagion

Khanam

Chahl

Chahl³

et al. 2021

J. Imaging

View full text Add to dashboard Cite

The World Health Organization (WHO) has declared COVID-19 a pandemic. We review and reduce the clinical literature on diagnosis of COVID-19 through symptoms that might be remotely detected as of early May 2020. Vital signs associated with respiratory distress and fever, coughing, and visible infections have been reported. Fever screening by temperature monitoring is currently popular. However, improved noncontact detection is sought. Vital signs including heart rate and respiratory rate are affected by the condition. Cough, fatigue, and visible infections are also reported as common symptoms. There are non-contact methods for measuring vital signs remotely that have been shown to have acceptable accuracy, reliability, and practicality in some settings. Each has its pros and cons and may perform well in some challenges but be inadequate in others. Our review shows that visible spectrum and thermal spectrum cameras offer the best options for truly noncontact sensing of those studied to date, thermal cameras due to their potential to measure all likely symptoms on a single camera, especially temperature, and video cameras due to their availability, cost, adaptability, and compatibility. Substantial supply chain disruptions during the pandemic and the widespread nature of the problem means that cost-effectiveness and availability are important considerations.

show abstract

Section: Image Processing Techniques Associated To Thermal Imagingmentioning

confidence: 99%

Noncontact Sensing of Contagion

Khanam

Chahl

Chahl³

et al. 2021

J. Imaging

View full text Add to dashboard Cite

show abstract

“…Existing multi‐modality fusion Re‐ID focuses on RGB–D modules [21–23], visible‐thermal (VT) modules [8, 24] and RGB–IR modules [7]. RGB–D Re‐ID combines human RGB image and depth information, and depth information is used to provide more stable body information to reduce the impact of changed clothes or extreme illumination on Re‐ID.…”

Section: Related Workmentioning

confidence: 99%

HPILN: a feature learning framework for cross‐modality person re‐identification

et al. 2019

View full text Add to dashboard Cite

Most video surveillance systems use both RGB and infrared cameras, making it a vital technique to re‐identify a person cross the RGB and infrared modalities. This task can be challenging due to both the cross‐modality variations caused by heterogeneous images in RGB and infrared, and the intra‐modality variations caused by the heterogeneous human poses, camera position, light brightness etc. To meet these challenges, a novel feature learning framework, hard pentaplet and identity loss network (HPILN), is proposed. In the framework existing single‐modality re‐identification models are modified to fit for the cross‐modality scenario, following which specifically designed hard pentaplet loss and identity loss are used to increase the accuracy of the modified cross‐modality re‐identification models. Based on the benchmark of the SYSU‐MM01 dataset, extensive experiments have been conducted, showing that the authors’ method outperforms all existing ones in terms of cumulative match characteristic curve and mean average precision.

show abstract

“…Another category are the synthetic thermal images generated by the Generative Adversarial Networks (GANs) [ 25 ]. Methods, like References [ 26 , 27 ], use the pre-trained deep convolutional networks to convert RGB images into the pseudo-thermal-like images, so the generated pseudo-thermal image covers the same scene as the RGB does. Later, these methods perform the object detection process on the original RGB images using the YOLO network and annotations are then finally be used with the GAN-generated pseudo thermal image as a ground truth.…”

Section: Introductionmentioning

confidence: 99%

Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data

Ligocki

Jelinek

Žalud

et al. 2021

Sensors

View full text Add to dashboard Cite

One of the biggest challenges of training deep neural network is the need for massive data annotation. To train the neural network for object detection, millions of annotated training images are required. However, currently, there are no large-scale thermal image datasets that could be used to train the state of the art neural networks, while voluminous RGB image datasets are available. This paper presents a method that allows to create hundreds of thousands of annotated thermal images using the RGB pre-trained object detector. A dataset created in this way can be used to train object detectors with improved performance. The main gain of this work is the novel method for fully automatic thermal image labeling. The proposed system uses the RGB camera, thermal camera, 3D LiDAR, and the pre-trained neural network that detects objects in the RGB domain. Using this setup, it is possible to run the fully automated process that annotates the thermal images and creates the automatically annotated thermal training dataset. As the result, we created a dataset containing hundreds of thousands of annotated objects. This approach allows to train deep learning models with similar performance as the common human-annotation-based methods do. This paper also proposes several improvements to fine-tune the results with minimal human intervention. Finally, the evaluation of the proposed solution shows that the method gives significantly better results than training the neural network with standard small-scale hand-annotated thermal image datasets.

show abstract

ThermalGAN: Multimodal Color-to-Thermal Image Translation for Person Re-identification in Multispectral Dataset

Cited by 92 publications

References 48 publications

Noncontact Sensing of Contagion

Noncontact Sensing of Contagion

HPILN: a feature learning framework for cross‐modality person re‐identification

Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data

Contact Info

Product

Resources

About