Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline

Liu, Yu-Lun; Lai, Wei-Sheng; Chen, Yu‐Sheng; Kao, Yi-Lung; Yang, Ming–Hsuan; Chuang, Yung-Yu; Huang, Jia-Bin

doi:10.48550/arxiv.2004.01179

Cited by 4 publications

(14 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Marnerides et al [17] proposed a CNN model that trains to infer a direct mapping function between LDR and HDR images. To overcome the dataset quantity challenges, Kim et al [12] and Liu et al [15] utilized the dynamic range constrained dataset, which consists of images crawled and extracted from the Internet, and the virtual dataset, respectively. However, since the datasets have diverse dynamic ranges, the normalization or standardization process for the images becomes difficult.…”

Section: Deep Learning-based Hdr Reconstructionmentioning

confidence: 99%

“…Datasets We trained our model on the VDS dataset [13], where the training set has 48 multi-exposure stacks and the testing set has 48 stacks. In addition, we evaluated our model on the stacks of HDR-Eye dataset [13,15,19], which is widely used for the perforamce evaluation. Input images were upscaled or downscaled into 256 × 256 pixel resolutions by the Lanczos interpolation method [11], and all LDR images are in the sRGB color space.…”

Section: Experiments Setupsmentioning

confidence: 99%

“…Evaluation metrics Our model was evaluated with the HDR-VDP-2 score [15][16][17] for HDR reconstruction results. We preprocessed reference HDR images with the range normalization [17] before measuring the HDR-VDP-2 score, which is set to a viewing distance of 0.5 m, the color encoding of RGB-BT.709 for a 24-inch display.…”

Section: Experiments Setupsmentioning

confidence: 99%

“…Deep neural networks, especially convolutional neural networks (CNNs), have shown their significant role in reconstructing the HDR image. Two primary approaches exist in reconstructing the HDR image: direct reconstruction methods [5,15,17] and multi-exposure stack-based synthesis methods [6,13,14]. Direct reconstruction aims to recover a HDR image (32bits/pixel) from a given single low dynamic range (LDR) image (8bits/pixel).…”

Section: Introductionmentioning

confidence: 99%

“…In this case, a large number of LDR-HDR image pair data is required to train a deep neural network [17]. There have been many attempts to solve the data quantity problem by crawling image pairs from the internet or generating synthetic image pairs [15]. On the other hand, HDR synthesis with the multi-exposure stack focuses on transferring exposures to accurately generate the multi-exposure stack.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

Kim¹,

Lee²,

Kang³

2020

Preprint

View full text Add to dashboard Cite

Recent deep learning-based methods have reconstructed a high dynamic range (HDR) image from a single low dynamic range (LDR) image by focusing on the exposure transfer task to reconstruct the multi-exposure stack. However, these methods often fail to fuse the multi-exposure stack into a perceptually pleasant HDR image as the local inversion artifacts are formed in the HDR imaging (HDRI) process. The artifacts arise from the impossibility of learning the whole HDRI process due to its non-differentiable structure of the camera response recovery. Therefore, we tackle the major challenge in stack reconstruction-based methods by proposing a novel framework with the fully differentiable HDRI process. Our framework enables a neural network to train the HDR image generation based on the end-to-end structure. Hence, a deep neural network can train the precise correlations between multi-exposure images in the HDRI process using our differentiable HDR synthesis layer. In addition, our network uses the image decomposition and the recursive process to facilitate the exposure transfer task and to adaptively respond to recursion frequency. The experimental results show that the proposed network outperforms the state-of-the-art quatitative and qualitative results in terms of both the exposure transfer tasks and the whole HDRI process. * equal contribution Preprint. Under review.

show abstract

Section: Deep Learning-based Hdr Reconstructionmentioning

confidence: 99%

Section: Experiments Setupsmentioning

confidence: 99%

Section: Experiments Setupsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

Kim¹,

Lee²,

Kang³

2020

Preprint

View full text Add to dashboard Cite

show abstract

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

Kim

Lee²,

Kang

2021

AAAI

View full text Add to dashboard Cite

Recently, high dynamic range (HDR) image reconstruction based on the multiple exposure stack from a given single exposure utilizes a deep learning framework to generate high-quality HDR images. These conventional networks focus on the exposure transfer task to reconstruct the multi-exposure stack. Therefore, they often fail to fuse the multi-exposure stack into a perceptually pleasant HDR image as the inversion artifacts occur. We tackle the problem in stack reconstruction-based methods by proposing a novel framework with a fully differentiable high dynamic range imaging (HDRI) process. By explicitly using the loss, which compares the network's output with the ground truth HDR image, our framework enables a neural network that generates the multiple exposure stack for HDRI to train stably. In other words, our differentiable HDR synthesis layer helps the deep neural network to train to create multi-exposure stacks while reflecting the precise correlations between multi-exposure images in the HDRI process. In addition, our network uses the image decomposition and the recursive process to facilitate the exposure transfer task and to adaptively respond to recursion frequency. The experimental results show that the proposed network outperforms the state-of-the-art quantitative and qualitative results in terms of both the exposure transfer tasks and the whole HDRI process.

show abstract

Luminance Attentive Networks for HDR Image and Panorama Reconstruction

Yu¹,

Li²,

Long³

et al. 2021

Preprint

View full text Add to dashboard Cite

It is very challenging to reconstruct a high dynamic range (HDR) from a low dynamic range (LDR) image as an ill-posed problem. This paper proposes a luminance attentive network named LANet for HDR reconstruction from a single LDR image. Our method is based on two fundamental observations: (1) HDR images stored in relative luminance are scale-invariant, which means the HDR images will hold the same information when multiplied by any positive real number. Based on this observation, we propose a novel normalization method called " HDR calibration " for HDR images stored in relative luminance, calibrating HDR images into a similar luminance scale according to the LDR images. (2) The main difference between HDR images and LDR images is in under-/over-exposed areas, especially those highlighted. Following this observation, we propose a luminance attention module with a two-stream structure for LANet to pay more attention to the under-/over-exposed areas. In addition, we propose an extended network called panoLANet for HDR panorama reconstruction from an LDR panorama and build a dualnet structure for panoLANet to solve the distortion problem caused by the equirectangular panorama. Extensive experiments show that our proposed approach LANet can reconstruct visually convincing HDR images and demonstrate its superiority over state-of-the-art approaches in terms of all metrics in inverse tone mapping. The image-based lighting application with our proposed panoLANet also demonstrates that our method can simulate natural scene lighting using only LDR panorama. Our source code is available at https://github.com/LWT3437/LANet.

show abstract

Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline

Cited by 4 publications

References 56 publications

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

Luminance Attentive Networks for HDR Image and Panorama Reconstruction

Contact Info

Product

Resources

About