Deep Inverse Tone Mapping for Compressed Images

Abstract

Converting a single low dynamic range (LDR) image into a high dynamic range (HDR) image, which is the so-called inverse tone mapping (ITM), is a challenging ill-posed problem since a lot of information is lost during compression and storage. Traditional ITM techniques mainly focus on high-quality LDR images without compression artifacts. However, in practice, LDR images are usually stored as a lossy compression format for the convenience of transmission, which will cause artifacts, i.e., blocking and ringing. Hence, these ITM methods suffer from severe performance drop for some real-world applications. In this paper, we propose a novel decomposition-based network to reconstruct HDR images from compressed LDR images; in other words, the proposed network can simultaneously remove blocking/ringing artifacts and recover high-quality HDR information. Considering compression artifacts mainly embedded in high frequency part, we thus decompose the input image into the high-frequency component (also known as detail layer) and low-frequency component (also known as a base layer). The detail layer mainly contains the information of texture, noise, and artifacts, while the base layer contains the information of structure and large object. Based on this, we design two subnetworks, i.e., detail layer recovery subnetwork and base layer recovery subnetwork, to restore the two parts separately. The detail layer recovery subnetwork is responsible for the artifacts removal with texture preserving, while the base layer recovery subnetwork is designed for tone expansion and overexposed/underexposed region restoration. Finally, in order to further reconstruct the serious overexposed regions, we adapt a merge subnetwork to fuse the result from the previous two subnetworks. The experimental results on compressed images demonstrate that the proposed method significantly outperforms other state-of-the-art methods. INDEX TERMS High dynamic range, decomposition, inverse tone mapping, compression artifacts removal, deep learning.

Single Exposure High Dynamic Range Image Reconstruction Based on Deep Dual-Branch Network

“…Kinoshita et al [19] proposed a loss function based on tone-mapped image to solve the inaccuracy problem caused by the nonlinear relationship between LDR and HDR images. Wang et al [20] decomposed the image into high-frequency and low-frequency components, and designed two sub-networks to process them separately. jang et al [21] designed the network to learn the cumulative histogram of HDR images, and used the results for histogram matching of LDR images.…”

Deep Tone-Mapping Operator Using Image Quality Assessment Inspired Semi-Supervised Learning

“…Later work Marnerides et al [39] first applied multi-branch structure by assigning different kernel size in each branch to focus on global/local features. Later, decomposing and multi-branch had become more popular, as they were used by [31], [25], [26], [27], [34], [40], [41], [42], [45], [5] and [47]. Moreover, Wang et al [42] first tackled denoising, while Xu et al [46] first introduced 3D convolution considering temporal information of HDR videos.…”

“…The Polishing Network (NP) consists of 2 MGRBs and 4 convolutional layers (totally 8-layers). Among related works where filter decomposing and branch network structure were adopted ( [31], [25], [42], [45], [5] and [47]), polishing network is used by 4 of them ( [25], [45], [5] and [47]). We took the same design, and an experiment was later conducted to prove its necessity.…”

“…First, resizing i.e., processing full-size input image with local/detail/large-scale branch while sending down-sampled image into global/smallscale branch ( [26], [27], [39] and [41]). Second, filtering i.e., decomposing image into base and detail components using filter (usually edge-preserving filter like bilateral filter) and respectively sending them into global and local/detail branches ( [31], [42], [45], [5] and [47]). Third, "decomposing" by color channel ( [31], [34] and [40]).…”

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

“…Early work in this domain utilized heuristic approaches [3,8,44,53], but often does not provide satisfying HDR reconstructions [1,40]. Building upon these works, deep learning has been used to hallucinate HDR content from LDR images [9,12,47,10,66,34,35,63,39,50,27]. These approaches generate plausible reconstructions of low-light regions but fail to recover saturated details accurately.…”