End-to-End Differentiable Learning to HDR Image Synthesis for Multi-exposure Images

Abstract

Recent deep learning-based methods have reconstructed a high dynamic range (HDR) image from a single low dynamic range (LDR) image by focusing on the exposure transfer task to reconstruct the multi-exposure stack. However, these methods often fail to fuse the multi-exposure stack into a perceptually pleasant HDR image as the local inversion artifacts are formed in the HDR imaging (HDRI) process. The artifacts arise from the impossibility of learning the whole HDRI process due to its non-differentiable structure of the camera response recovery. Therefore, we tackle the major challenge in stack reconstruction-based methods by proposing a novel framework with the fully differentiable HDRI process. Our framework enables a neural network to train the HDR image generation based on the end-to-end structure. Hence, a deep neural network can train the precise correlations between multi-exposure images in the HDRI process using our differentiable HDR synthesis layer. In addition, our network uses the image decomposition and the recursive process to facilitate the exposure transfer task and to adaptively respond to recursion frequency. The experimental results show that the proposed network outperforms the state-of-the-art quatitative and qualitative results in terms of both the exposure transfer tasks and the whole HDRI process. * equal contribution Preprint. Under review.

Fast and flexible stack‐based inverse tone mapping

“…The proposed method is compared with seven recent state-ofthe-art Convolutional Neural Networks-based approaches: Deep reverse tone mapping opearator (DrTMO) [9], Deep recursive high dynamic range imaging (HDRI) [11], Deep Single HDRI [18], Deep Chain HDRI [18], Deep Diff HDRI [22], Deep Mask HDRI [16], and Deep HDR-UNet [21]. The commonly used HDR-VDP-2.2 [34] index is adopted to measure the quality of HDR reconstruction.…”

“…and the histogram loss in [22] to ensure that the generated image has a similar global tone with the target image: where O denotes the ground truth image, L denotes the intensity levels, and cnt l indicates the number of pixels, which has a rounded down intensity l in the input image. The total loss of EAM can be formulated as:…”

“…Meanwhile, they overcame the above problem by training a single increasing/decreasing model and reusing it to change the EV. Kim et al [22] proposed a novel framework with a fully differentiable HDR imaging process by introducing the differentiable HDR synthesis layer. Compared to the previous stack-based ITM methods, the proposed method can generate the multi exposure stack with flexible exposure setting and achieve more accurate results.…”

“…Note that the running time of stack-based ITM methods only indicates the time of estimating the MES. We set the length of MES to 7 for the U-Net [17], Deep Chain HDRI [10], Deep Recursive HDRI [11], and Deep Diff HDRI [22] as described in their papers. The time of the once increasing/decreasing process of these methods can be calculated by dividing the running time in Table 3 by six.…”

HSVNet: Reconstructing HDR Image from a Single Exposure LDR Image with CNN

“…A better estimation is expected by separating these attributes. Reconstructing HDR in RGB color space may feature a weakness such as severe local inversion artifacts on reconstructed HDR images and insufficient contrast [2,15,16]. This is partly because loss functions were simple mean-absolute error (MAE) or mean-squared error (MSE) between the predicted and target images.…”