Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System

Jang, Hanbyol; Bang, Kihun; Jang, Jong Whan; Hwang, Dosik

doi:10.1109/access.2018.2870295

Cited by 11 publications

(13 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Early work in this domain utilized heuristic approaches [3,8,44,53], but often does not provide satisfying HDR reconstructions [1,40]. Building upon these works, deep learning has been used to hallucinate HDR content from LDR images [9,12,47,10,66,34,35,63,39,50,27]. These approaches generate plausible reconstructions of low-light regions but fail to recover saturated details accurately.…”

Section: Related Workmentioning

confidence: 99%

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

Sun

Tseng

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

High-dynamic range (HDR) imaging is an essential imaging modality for a wide range of applications in uncontrolled environments, including autonomous driving, robotics, and mobile phone cameras. However, existing HDR techniques in commodity devices struggle with dynamic scenes due to multi-shot acquisition and postprocessing time, e.g. mobile phone burst photography, making such approaches unsuitable for real-time applications. In this work, we propose a method for snapshot HDR imaging by learning an optical HDR encoding in a single image which maps saturated highlights into neighboring unsaturated areas using a diffractive optical element (DOE). We propose a novel rank-1 parameterization of the DOE which drastically reduces the optical search space while allowing us to efficiently encode high-frequency detail. We propose a reconstruction network tailored to this rank-1 parametrization for the recovery of clipped information from the encoded measurements. The proposed end-to-end framework is validated through simulation and real-world experiments and improves the PSNR by more than 7 dB over state-ofthe-art end-to-end designs.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

Sun

Tseng

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

View full text Add to dashboard Cite

show abstract

“…The raw images taken with a digital camera without compression or processing are 12-to 14-bit images [31]. We adjusted the exposure values (−2, −1, 0, +1, +2) from a single raw image and then generated five different LDR images [32]- [34]. We merged these five LDR images using the HDR Pro algorithm.…”

Section: E Transfer Learningmentioning

confidence: 99%

“…Jang et al [34] also experimented with image-to-image learning on brightness images. However, the characteristic of the LDR-HDR pair image was only the change of dynamic range, not the change of structure information in the image.…”

Section: B Hdr Brightness Images Using Histogram Matchingmentioning

confidence: 99%

Dynamic Range Expansion Using Cumulative Histogram Learning for High Dynamic Range Image Generation

et al. 2020

Self Cite

View full text Add to dashboard Cite

In modern digital photographs, most images have low dynamic range (LDR) formats, which means that the range of light intensities from the darkest to the brightest is much lower than the range that can be perceived by the human eye. Therefore, to visualize images as naturally as possible on devices that display them in high dynamic range (HDR) format, the LDR images need to be converted into HDR images. The aim of this study was to develop an adaptive inverse tone mapping operator (iTMO) that can convert a single LDR image into a realistic HDR image based on artificial neural networks. In contrast to conventional iTMO algorithms, our technique was developed by learning the complicated relationship between various LDR-HDR pair images, which enabled nearly ground-truth HDR images to be generated from various types of LDR images. The novel learning technique is called cumulative histogram-based learning and color difference learning. The superior performance of our technique over conventional methods was assessed through objective evaluations of various types of LDR and HDR images. INDEX TERMS High dynamic range, inverse tone mapping operator, cumulative histogram, color difference, convolutional neural network.

show abstract

“…Ning et al [10] introduce the generative adversarial regularizer to improve the quality of results. Jang et al [24] maintain the color is vital for ITM, hence, they adapt a network architecture to learn dynamic range and color difference respectively. As opposed to direct methods, in-direct method won't generate HDR images directly, these methods first generate a multi-exposed LDR images, and then merge them with conventional methods.…”

Section: Related Work a Inverse Tone Mappingmentioning

confidence: 99%

Deep Inverse Tone Mapping for Compressed Images

2019

View full text Add to dashboard Cite

Converting a single low dynamic range (LDR) image into a high dynamic range (HDR) image, which is the so-called inverse tone mapping (ITM), is a challenging ill-posed problem since a lot of information is lost during compression and storage. Traditional ITM techniques mainly focus on high-quality LDR images without compression artifacts. However, in practice, LDR images are usually stored as a lossy compression format for the convenience of transmission, which will cause artifacts, i.e., blocking and ringing. Hence, these ITM methods suffer from severe performance drop for some real-world applications. In this paper, we propose a novel decomposition-based network to reconstruct HDR images from compressed LDR images; in other words, the proposed network can simultaneously remove blocking/ringing artifacts and recover high-quality HDR information. Considering compression artifacts mainly embedded in high frequency part, we thus decompose the input image into the high-frequency component (also known as detail layer) and low-frequency component (also known as a base layer). The detail layer mainly contains the information of texture, noise, and artifacts, while the base layer contains the information of structure and large object. Based on this, we design two subnetworks, i.e., detail layer recovery subnetwork and base layer recovery subnetwork, to restore the two parts separately. The detail layer recovery subnetwork is responsible for the artifacts removal with texture preserving, while the base layer recovery subnetwork is designed for tone expansion and overexposed/underexposed region restoration. Finally, in order to further reconstruct the serious overexposed regions, we adapt a merge subnetwork to fuse the result from the previous two subnetworks. The experimental results on compressed images demonstrate that the proposed method significantly outperforms other state-of-the-art methods. INDEX TERMS High dynamic range, decomposition, inverse tone mapping, compression artifacts removal, deep learning.

show abstract

Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System

Cited by 11 publications

References 37 publications

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging

Dynamic Range Expansion Using Cumulative Histogram Learning for High Dynamic Range Image Generation

Deep Inverse Tone Mapping for Compressed Images

Contact Info

Product

Resources

About