Deep Tone Mapping Operator for High Dynamic Range Images

Abstract

A computationally fast tone mapping operator (TMO) that can quickly adapt to a wide spectrum of high dynamic range (HDR) content is quintessential for visualization on varied low dynamic range (LDR) output devices such as movie screens or standard displays. Existing TMOs can successfully tone-map only a limited number of HDR content and require an extensive parameter tuning to yield the best subjective-quality tone-mapped output. In this paper, we address this problem by proposing a fast, parameter-free and scene-adaptable deep tone mapping operator (DeepTMO) that yields a high-resolution and high-subjective quality tone mapped output. Based on conditional generative adversarial network (cGAN), DeepTMO not only learns to adapt to vast scenic-content (e.g., outdoor, indoor, human, structures, etc.) but also tackles the HDR related scenespecific challenges such as contrast and brightness, while preserving the fine-grained details. We explore 4 possible combinations of Generator-Discriminator architectural designs to specifically address some prominent issues in HDR related deep-learning frameworks like blurring, tiling patterns and saturation artifacts. By exploring different influences of scales, loss-functions and normalization layers under a cGAN setting, we conclude with adopting a multi-scale model for our task. To further leverage on the large-scale availability of unlabeled HDR data, we train our network by generating targets using an objective HDR quality metric, namely Tone Mapping Image Quality Index (TMQI). We demonstrate results both quantitatively and qualitatively, and showcase that our DeepTMO generates high-resolution, high-quality output images over a large spectrum of real-world scenes. Finally, we evaluate the perceived quality of our results by conducting a pair-wise subjective study which confirms the versatility of our method.

Tone Mapping Operators: Progressing Towards Semantic-Awareness

“…They tend to learn the tone curve from a high dimensional feature representation based on training images. Rana et al [9] follow such a data-driven approach and propose a deep learning based parameter-free TMO. Such approaches might implicitly incorporate semantic attributes but do not explicitly use semantic information.…”

Vision models fine-tuned by cinema professionals for High Dynamic Range imaging in movies

“…We will also compare our TMO with the most popular TM methods used in the cinema industry during production and post-production: camera manufacturer look-up tables (LUTs) (Canon provided LUT, Arri Alexa V3 to Rec709 LUT, and REDRAW default processing in Resolve), Resolve Luminance Mapping, Baselight TM (Truelight CAM DRT). Some very recent tone mapping works which we would like to compare with in the future are those of Zhang et al [91], that use a retinal model, and Rana et al [62], that uses a deep learning approach.…”

Differential Evolution-based Approach for Tone-Mapping of High Dynamic Range Images

“…Recently Rana et al [44] provided a Deep Convolutional Neural Network (DCNN) for tone-mapping of HDR images. The researchers gathered a large group of tone-mapped images generated by a number of existing TMOs.…”