Thermal infrared colorization via conditional generative adversarial network

Kuang, Xiaodong; Zhu, Jin; Sui, Xiubao; Liu, Yuan; Li, Chengwei; Chen, Qian; Gu, Guohua

doi:10.1016/j.infrared.2020.103338

Cited by 62 publications

(31 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Infrared images have also been converted to natural color images using CNNs (e.g., Limmer and Lensch 2016) (Fig. 8a) and GANs (e.g., Kuang et al 2020;Suarez et al 2017).…”

Section: Colorizationmentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

This paper reviews the current state of the art in artificial intelligence (AI) technologies and applications in the context of the creative industries. A brief background of AI, and specifically machine learning (ML) algorithms, is provided including convolutional neural networks (CNNs), generative adversarial networks (GANs), recurrent neural networks (RNNs) and deep Reinforcement Learning (DRL). We categorize creative applications into five groups, related to how AI technologies are used: (i) content creation, (ii) information analysis, (iii) content enhancement and post production workflows, (iv) information extraction and enhancement, and (v) data compression. We critically examine the successes and limitations of this rapidly advancing technology in each of these areas. We further differentiate between the use of AI as a creative tool and its potential as a creator in its own right. We foresee that, in the near future, ML-based AI will be adopted widely as a tool or collaborative assistant for creativity. In contrast, we observe that the successes of ML in domains with fewer constraints, where AI is the ‘creator’, remain modest. The potential of AI (or its developers) to win awards for its original creations in competition with human creatives is also limited, based on contemporary technologies. We therefore conclude that, in the context of creative industries, maximum benefit from AI will be derived where its focus is human-centric—where it is designed to augment, rather than replace, human creativity.

show abstract

“…Infrared images have also been converted to natural color images using CNNs (e.g., Limmer and Lensch 2016) (Fig. 8a) and GANs (e.g., Kuang et al 2020;Suarez et al 2017).…”

Section: Colorizationmentioning

confidence: 99%

Artificial intelligence in the creative industries: a review

2021

View full text Add to dashboard Cite

show abstract

“…missing objects from the scene, object deformations and some failure images. Kuang et al in [Kuang et al, 2018] used a conditional generative adversarial loss to generate a realistic Visible image, with the perceptual loss based on the VGG-16 model, the TV loss to ensure spatial smoothness, and the MSE as content loss. Their work presented better realistic colour representations with fine details but also suffered from the same artefacts, missing objects and object deformations.…”

Section: Related Workmentioning

confidence: 99%

“…All experiments were implemented in Pytorch and performed on an NVIDIA TITAN XP graphics card. TIR2Lab [Berg et al, 2018] and TIC-CGAN [Kuang et al, 2018] were re-implemented and trained as explained in the original papers.…”

Section: Training Setupmentioning

confidence: 99%

Robust Perceptual Night Vision in Thermal Colorization

Almasri

Debeir

2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

Figure 1: An example of mapping a thermal image to a color Visible image is presented. (left): a thermal image from the ULB17-VT.V2 test set, and, (right): its colorized counterpart. This approach generates color values consistent with the color Visible ground truth and preserves objects textures from the thermal representation. Abstract:Transforming a thermal infrared image into a robust perceptual colour Visible image is an ill-posed problem due to the differences in their spectral domains and in the objects' representations. Objects appear in one spectrum but not necessarily in the other, and the thermal signature of a single object may have different colours in its Visible representation. This makes a direct mapping from thermal to Visible images impossible and necessitates a solution that preserves texture captured in the thermal spectrum while predicting the possible colour for certain objects. In this work, a deep learning method to map the thermal signature from the thermal images spectrum to a Visible representation in their low-frequency space is proposed. A pan-sharpening method is then used to merge the predicted low-frequency representation with the high-frequency representation extracted from the thermal image. The proposed model generates colour values consistent with the Visible ground truth when the object does not vary much in its appearance and generates averaged grey values in other cases. The proposed method shows robust perceptual night vision images in preserving the object's appearance and image context compared with the existing state-of-the-art.

show abstract

“…Deep neural network (DNN) models have been developed based on diverse architectures, ranging from convolutional neural networks (CNNs) [9], [10], [16] to generative adversarial networks ‚ L. Wang (GANs) [17], [18], [19]. In general, SoTA-DNN-based methods differ in terms of five major aspects: network design that considers the number and domain of input LDR images [9], [10], [14], purpose of HDR imaging in multitask learning [20], [21], different sensors being used to obtain deep HDR imaging [22], [23], [24], novel learning strategies [17], [25], [26], and practical applications [27], [28], [29].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Wang

Yoon

2021

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.

show abstract

Thermal infrared colorization via conditional generative adversarial network

Cited by 62 publications

References 18 publications

Artificial intelligence in the creative industries: a review

Artificial intelligence in the creative industries: a review

Robust Perceptual Night Vision in Thermal Colorization

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Contact Info

Product

Resources

About