Thermal Image Enhancement using Convolutional Neural Network

Choi, Yukyung; Kim, Namil; Hwang, Soonmin; Kweon, In So

doi:10.1109/iros.2016.7759059

Cited by 109 publications

(81 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Over the years, extensive features and algorithms have been proposed, including both traditional detectors [14,12,38,56] and the lately dominated CNN-based detectors [37,22,1,50,58]. Recently, multispectral data have shown great advantages, especially for the all-day vision [25,7,8]. Hence the release of large-scale multispectral pedestrian benchmarks [24,17] is encouraging the research community to advance the state-of-the-art by efficiently exploiting multispectral input data.…”

Section: Related Workmentioning

confidence: 99%

Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection

Zhang

Zhu²,

Chen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

132

120

View full text Add to dashboard Cite

Multispectral pedestrian detection has shown great advantages under poor illumination conditions, since the thermal modality provides complementary information for the color image. However, real multispectral data suffers from the position shift problem, i.e. the color-thermal image pairs are not strictly aligned, making one object has different positions in different modalities. In deep learning based methods, this problem makes it difficult to fuse the feature maps from both modalities and puzzles the CNN training. In this paper, we propose a novel Aligned Region CNN (AR-CNN) to handle the weakly aligned multispectral data in an end-to-end way. Firstly, we design a Region Feature Alignment (RFA) module to capture the position shift and adaptively align the region features of the two modalities. Secondly, we present a new multimodal fusion method, which performs feature re-weighting to select more reliable features and suppress the useless ones. Besides, we propose a novel RoI jitter strategy to improve the robustness to unexpected shift patterns of different devices and system settings. Finally, since our method depends on a new kind of labelling: bounding boxes that match each modality, we manually relabel the KAIST dataset by locating bounding boxes in both modalities and building their relationships, providing a new KAIST-Paired Annotation. Extensive experimental validations on existing datasets are performed, demonstrating the effectiveness and robustness of the proposed method. Code and data are available at: https://github.com/luzhang16/AR-CNN .

show abstract

Section: Related Workmentioning

confidence: 99%

Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection

Zhang

Zhu²,

Chen

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

132

120

View full text Add to dashboard Cite

show abstract

“…We evaluated our proposed EFTS method against other existing state-of-the-art techniques in the literature. First, we compared our method with the methods developed for thermal images (TEN [17], CNN [19]) and as well as those which are developed for visible images (VDSR [7], LapSRN [13], RDN [16]). We use the same training dataset for all these models and the parameters they used in their respective papers for a fair comparison.…”

Section: Edge Extraction Modulementioning

confidence: 99%

“…Most of these models source codes were available online except TEN [17] and CNN [19]. For these models, we have implemented their codes based on what is reported in [17,19].…”

Section: Edge Extraction Modulementioning

confidence: 99%

“…While there are many published works for visible (RGB) images, the thermal images have received less attention. Authors in [17] proposed TEN, a thermal enhancement network inspired by SRCNN with three convolutional layers. In [18] authors used two inputs for their networks: a low resolution near infrared and a high-resolution visible image.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Edge focused super-resolution of thermal images

Zoetgnande

Dillenseger

Alirezaie

2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

In this work, a super-resolution method is proposed for indoor scenes captured by low-resolution thermal cameras. The proposed method is called Edge Focused Thermal Superresolution (EFTS) which contains an edge extraction module enforcing the neural networks to focus on the edge of images. Utilizing edge information, our model, based on residual dense blocks, can perform super-resolution for thermal images, while enhancing the visual information of the edges. Experiments on benchmark datasets showed that our EFTS method achieves better performance in comparison to the state-of-the-art techniques.

show abstract

“…Over the past few years, some studies have been carried out to enhance the resolution of thermal images to address these low-resolution limitations [5][6][7][8]. Recently, with starting with a study by Dong et al [9], the deep learning method has been successfully applied to improve the resolution of a single image, so many studies using deep convolutional networks have been actively conducted [10][11][12]. Choi et al [12] have demonstrated that a convolution neural network can be successfully applied to thermal image enhancement.…”

Section: Introductionmentioning

confidence: 99%

Resolution Enhancement for Thermographic Inspection in Industrial Plant Using Deep Convolutional Networks

Choi¹,

Seong²,

Son³

et al. 2018

Proceedings of the International Symposium on Automation and Robotics in Construction (IAARC)

View full text Add to dashboard Cite

As the importance of maintainability for the energy efficiency of industrial plants is emphasized, thermographic inspection has been widely used as a key technique. However, the thermographic inspection cannot provide clear information due to the limitation of low resolution in inspecting wide areas where the worker cannot acquire data in close, such as hazardous environments and the ceiling with pipelines. Therefore, this paper proposes a resolution enhancement method for thermographic inspection in industrial plants based on deep convolutional network. The proposed network involves patch extraction and representation, non-linear mapping, and reconstruction. The proposed method was validated with 210 thermal images obtained from real cogeneration plants. Experimental results show that the method can provide the clear boundaries of instruments with an average PSNR of 32.51, which is superior than the bicubic interpolation of 29.92. The proposed method can be implemented in energy efficiency monitoring and automatic defect detection for thermal inspection in industrial plants by helping to detect defects that can be unchecked in low resolution images.

show abstract

Thermal Image Enhancement using Convolutional Neural Network

Cited by 109 publications

References 19 publications

Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection

Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection

Edge focused super-resolution of thermal images

Resolution Enhancement for Thermographic Inspection in Industrial Plant Using Deep Convolutional Networks

Contact Info

Product

Resources

About