Thermal Image Super-Resolution Challenge Results - PBVS 2022

Rivadeneira, Rafael E.; Sappa, Ángel D.; Vintimilla, Boris X.; Kim, Jin; Kim, Do-Gun; Li, Zhihao; Jian, Yingchun; Yan, Bo; Cao, Leilei; Qi, Fengliang; Wang, Hongbin; Wu, Rongyuan; Sun, Lingchen; Zhao, Yue; Li, Lin; Wang, Kai; Wang, Yicheng; Zhang, Xuanming; Wei, Huiyuan; Lv, Chonghua; Sun, Quansen; Tian, Xiaolin; Jia, Zhuang; Hu, Jiakui; Wang, Chenyang; Zhong, Zhang; Liu, Xianming; Jiang, Junjun

doi:10.1109/cvprw56347.2022.00057

Cited by 10 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning, as a data-driven technology, necessitates a significant amount of well-registered thermal infrared-visible light images for training data. To achieve this objective, we combined three popular multimodal datasets: M3FD [ 41 ], FLIR ADAS, and TISR [ 42 ]. Sample images from the dataset are exemplified in Figure 6 .…”

Section: Methodsmentioning

confidence: 99%

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Jiang

Liu

Zhan

et al. 2023

Entropy

View full text Add to dashboard Cite

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network’s ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

show abstract

Section: Methodsmentioning

confidence: 99%

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Jiang

Liu

Zhan

et al. 2023

Entropy

View full text Add to dashboard Cite

show abstract

“…We use the dataset proposed by Rivadeneira et al [47] as the training dataset. This dataset was serviced as the training and testing dataset for the PBVS [55] Thermal Image SR (TISR) challenge, which we simply abbreviate as the Challenge dataset. The Challenge dataset was created by capturing thermal images from three thermal cameras mounted on a panel.…”

Section: Training and Testing Datasetsmentioning

confidence: 99%

Thermal Image Super-Resolution Based on Lightweight Dynamic Attention Network for Infrared Sensors

Zhang,

Hu,

Yan

2023

Sensors

View full text Add to dashboard Cite

Infrared sensors capture infrared rays radiated by objects to form thermal images. They have a steady ability to penetrate smoke and fog, and are widely used in security monitoring, military, etc. However, civilian infrared detectors with lower resolution cannot compare with megapixel RGB camera sensors. In this paper, we propose a dynamic attention mechanism-based thermal image super-resolution network for infrared sensors. Specifically, the dynamic attention modules adaptively reweight the outputs of the attention and non-attention branches according to features at different depths of the network. The attention branch, which consists of channel- and pixel-wise attention blocks, is responsible for extracting the most informative features, while the non-attention branch is adopted as a supplement to extract the remaining ignored features. The dynamic weights block operates with 1D convolution instead of the full multi-layer perceptron on the global average pooled features, reducing parameters and enhancing information interaction between channels, and the same structure is adopted in the channel attention block. Qualitative and quantitative results on three testing datasets demonstrate that the proposed network can superior restore high-frequency details while improving the resolution of thermal images. And the lightweight structure of the proposed network with lower computing cost can be practically deployed on edge devices, effectively improving the imaging perception quality of infrared sensors.

show abstract

“…Current state-of-the-art super-resolution models use a number of techniques such as residual connections, edge-based features, 1 feature fusion, 2,3 generative adversarial networks, 4,5 and attention networks [6][7][8] for the SISR problem. These techniques tend to work well on reconstructing fine details that traditional techniques (e.g., bicubic and linear interpolation) do not.…”

Section: Introductionmentioning

confidence: 99%

Efficient infrared super-resolution

Chiapputo

Bailey

2023

Image Sensing Technologies: Materials, Devices, Systems, and Applications X

View full text Add to dashboard Cite

As modern displays continue to increase in resolution, means of capturing images and videos at such high resolutions can be prohibitively expensive. This is especially true in the infrared domain. Image super-resolution, or upsampling, has often been applied to improve the aforementioned problem. Deep learning models have been proposed to reconstruct high quality high-resolution images from a low-resolution base. Previous solutions require a massive number of parameters which necessitate a large amount of free memory and computation power or they fall apart when applied to the infrared domain. As a result, many modern super-resolution models are not entirely practical. One difficult aspect in IR super-resolution is that IR images are inherently noisy, causing a poor signal-to-noise ratio, due to characteristics of IR sensors and internal reflections within the lenses. Because of this, super-resolution in IR must also act somewhat as a denoiser. Therefore, we propose a highly efficient, super-resolution model capable of producing single-image super-resolution in the IR domain.

show abstract

Thermal Image Super-Resolution Challenge Results - PBVS 2022

Cited by 10 publications

References 16 publications

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Thermal Image Super-Resolution Based on Lightweight Dynamic Attention Network for Infrared Sensors

Efficient infrared super-resolution

Contact Info

Product

Resources

About