A New Journey from SDRTV to HDRTV

Abstract

Modern displays are capable of rendering video content with high dynamic range (HDR) and wide color gamut (WCG). However, the majority of available resources are still in standard dynamic range (SDR). As a result, there is significant value in transforming existing SDR content into the HDRTV standard. In this paper, we define and analyze the SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Our analysis and observations indicate that a naive end-to-end supervised training pipeline suffers from severe gamut transition errors. To address this issue, we propose a novel three-step solution pipeline called HDRTVNet++, which includes adaptive global color mapping, local enhancement, and highlight refinement. The adaptive global color mapping step uses global statistics as guidance to perform image-adaptive color mapping. A local enhancement network is then deployed to enhance local details. Finally, we combine the two sub-networks above as a generator and achieve highlight consistency through GAN-based joint training. Our method is primarily designed for ultra-high-definition TV content and is therefore effective and lightweight for processing 4K resolution images. We also construct a dataset using HDR videos in the HDR10 standard, named HDRTV1K that contains 1235 and 117 training images and 117 testing images, all in 4K resolution. Besides, we select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Our final results demonstrate state-of-the-art performance both quantitatively and visually. The code, model and dataset are available at https://github.com/xiaom233/HDRTVNet-plus.

Global Priors Guided Modulation Network for Joint Super-Resolution and Inverse Tone-Mapping

“…These methods are inappropriate for directly recovering HDR videos with PQ-OETF. Recently, some ITM methods [31]- [34] have been developed in order to recover HDR videos from the corresponding SDR videos. Chen et al [34] analyzed the SDR video to HDR video formation pipeline and proposed the HDRTVNet, a cas-caded approach containing three deep networks for SDRTVto-HDRTV problem.…”

“…Recently, some ITM methods [31]- [34] have been developed in order to recover HDR videos from the corresponding SDR videos. Chen et al [34] analyzed the SDR video to HDR video formation pipeline and proposed the HDRTVNet, a cas-caded approach containing three deep networks for SDRTVto-HDRTV problem. HDRTVNet implemented HDR video recovery with a three-stage training strategy.…”

“…CSRNet [36] extracts the global priors using N k × N k (N k > 1) filters for image retouching. Furthermore, AGCM [34] uses 1 × 1 filters to focus on extracting global color priors for the SDRTV-to-HDRTV task. Thus, inspired by these methods, we introduce the CCP extraction branch for ITM.…”

“…(a) Spatial feature transform [38], [39], which performs feature-wise modulation. (b) Global feature modulation block [34], [36], which modulates the feature map directly using global priors. (c) Our proposed global priors guided spatial-wise modulation block (GSMB), which performs spatial-wise modulation on feature maps after fusing spatial location information.…”

“…For joint SR-ITM problem, it is necessary to perform global mapping and spatialwise modulation. Therefore, we revisit spatial feature transform (SFT) [38], [39] and global feature modulation (GFM) [34], [36]. The SFT generates a pair (γ, β) ∈ R C×H×W by a condition maps, and then performs spatial modulations through scaling and shifting, respectively.…”