Deep Tone-Mapping Operator Using Image Quality Assessment Inspired Semi-Supervised Learning

Guo, Cheng; Jiang, Xiuhua

doi:10.1109/access.2021.3080331

Cited by 13 publications

(12 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We include in our comparisons fourteen tone mapping operators including ten traditional methods which for simplicity we refer to as: Mantiuk [MMS06], Shan [SJB09], Durand [DD02], Drago [DMAC03], Mertens [MKVR07], Reinhard [RSSF02], Ma [MYZW15], Liang [LXZ ∗ 18], Shibata [STO16], and Li [LJZ18]; and four recent learning‐based methods: Guo [GJ21], Zhang [ZZWW21], DeepTMO [RSV ∗ 19] and TMO‐Net [PKO ∗ 21]. We use the publicly available implementations of these methods or if not available, their implementation in HDRToolBox [BADC17].…”

Section: Results and Ablation Studymentioning

confidence: 99%

“…Different tone mapping techniques have been developed for decades [RHD ∗ 10, BADC17], however the performance of even the most prominent techniques strongly depends on the HDR image content and specific parameter settings [ZWZW19,GJ21,PKO ∗ 21]. Subjective evaluations of these different techniques indicate that both specific algorithms as well as default parameter settings, as often proposed by their respective authors, do not generalize well across scenes [LCTS05,YBMS05,ČWNA08].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning a self‐supervised tone mapping operator via feature contrast masking loss

Wang

Chen

Seidel

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid development of capture technologies. Nevertheless, the dynamic range of common display devices is still limited, therefore tone mapping (TM) remains a key challenge for image visualization. Recent work has demonstrated that neural networks can achieve remarkable performance in this task when compared to traditional methods, however, the quality of the results of these learning‐based methods is limited by the training data. Most existing works use as training set a curated selection of best‐performing results from existing traditional tone mapping operators (often guided by a quality metric), therefore, the quality of newly generated results is fundamentally limited by the performance of such operators. This quality might be even further limited by the pool of HDR content that is used for training. In this work we propose a learning‐based self‐supervised tone mapping operator that is trained at test time specifically for each HDR image and does not need any data labeling. The key novelty of our approach is a carefully designed loss function built upon fundamental knowledge on contrast perception that allows for directly comparing the content in the HDR and tone mapped images. We achieve this goal by reformulating classic VGG feature maps into feature contrast maps that normalize local feature differences by their average magnitude in a local neighborhood, allowing our loss to account for contrast masking effects. We perform extensive ablation studies and exploration of parameters and demonstrate that our solution outperforms existing approaches with a single set of fixed parameters, as confirmed by both objective and subjective metrics.

show abstract

Section: Results and Ablation Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learning a self‐supervised tone mapping operator via feature contrast masking loss

Wang

Chen

Seidel

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Other parameter settings follow the original HDRnet from Chen's implementation. We adopted the implementation from IQATM's repository [7] and re-wrote it for our data pipeline so that it worked on PQ HDR images, as the authors mentioned they have not tried it in the PQ domain.…”

Section: Trainingmentioning

confidence: 99%

“…Yang et al [6] used a Laplacian pyramid to separate pre-processed HDR images into base and detail layers in multiple sizes and trained their LaplacianNet on multi-branch CNN for tone mapping. Most recently, Guo et al based their work upon LaplacianNet and proposed IQATM [7] with a semi-supervised training strategy of HDR tone mapping by branching out the base layer and the detail layer from the Laplacian pyramid for separate training and later combining them for joint training. The loss function consists of both supervised and unsupervised terms.…”

Section: Introductionmentioning

confidence: 99%

CNN-Based Local Tone Mapping in the Perceptual Quantization Domain

You

Chen

Wang

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

A typical way of local tone mapping (TM) is based on multilayer decomposition of the source image. For this, the source image is decomposed into a base layer and a detail layer to compress from high dynamic range (HDR) to low dynamic range (LDR). Perceptual quantization (PQ) is a standardized non-linear transfer function for HDR content. It mimics the non-linearity of human vision by compressing more strongly in bright regions and less in dark areas, when converting luminance values to electrical signals. We propose a CNN-based pipeline for local TM, which operates on the low-frequency base layer of the luminance signal, while keeping the detail layer unchanged. The proposed method works entirely in the PQ domain and is adaptable to display peak luminance. The tone-mapped LDR images obtained with our learning-based approach show significant improvements in PSNR, while the network size is reduced compared to previous work. Our experiments on the HDR datasets from Fairchild and Funt show PSNR improvements of 8 dB compared to the state-of-the-art approaches.

show abstract

“…To further validate the performance of our approach we additionally performed a subjective study. We included the six best performing methods in terms of average TMQI score according to our previous objective evaluation, in particular: Guo [GJ21], Zhang [ZZWW21], Liang [LXZ * 18], Drago [DMAC03], Durand [DD02], and ours. The study was approved by the <hidden for anonymity> Ethical Board, and participants provided written consent for participating in the study.…”

Section: Subjective Evaluationmentioning

confidence: 99%

Learning a self-supervised tone mapping operator via feature contrast masking loss

Wang¹,

Chen²,

Seidel³

et al. 2021

Preprint

View full text Add to dashboard Cite

HDR Tone mapped LDR VGG feature map Feature contrast maskingFigure 1: Our self-supervised tone mapping operator is directly guided by the input HDR image and a novel feature contrast masking loss that takes into account masking effects present in the Human Visual System. We minimize the difference between the HDR image and the tone mapped image in feature space, after applying our contrast masking model. Left: The input HDR image and our tone mapped result. Right: VGG feature map (1st layer, 12th channel) and our corresponding feature contrast masking response that effectively enhances low contrast image details while compressing high contrasts.

show abstract

Deep Tone-Mapping Operator Using Image Quality Assessment Inspired Semi-Supervised Learning

Cited by 13 publications

References 63 publications

Learning a self‐supervised tone mapping operator via feature contrast masking loss

Learning a self‐supervised tone mapping operator via feature contrast masking loss

CNN-Based Local Tone Mapping in the Perceptual Quantization Domain

Learning a self-supervised tone mapping operator via feature contrast masking loss

Contact Info

Product

Resources

About