Saliency Map-Aided Generative Adversarial Network for RAW to RGB Mapping

Zhao, Yuzhi; Zhou, Chang; Yu, Wing Yin; Po, Lai-Man; Zhang, Tiantian; Liao, Zongbang; Shi, Xiang; Zhang, Yujia; Ou, Weifeng; Xian, Pengfei; Xiong, Jian

doi:10.1109/iccvw.2019.00428

Cited by 9 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For comparing enhancement performance on sRGB images, we choose publicly available supervised-learning-based algorithms such as DALE [28], DLN [61], DSLR [32], Enlight-enGAN [20], GLAD [63], MBLLEN [41], KinD [79], RetinexNet [67] and URIE [57], whereas for RAW images, we choose Rawpy 1 , RAW2RGB-GAN [82], TENet [46], PyNet [17], ELD [68], AWNet [12]. We summarize the qualitative performance on LOL and SID-Sony datasets along with computational requirement in GMAC (Giga-Multiplication and Accumulation Operations) 2 3 in Tab.…”

Section: Performance Evaluation With Sota Algorithmsmentioning

confidence: 99%

Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks

Shyam¹,

Sengar²,

Yoon³

et al. 2022

Preprint

View full text Add to dashboard Cite

The limited dynamic range of commercial compact camera sensors results in an inaccurate representation of scenes with varying illumination conditions, adversely affecting image quality and subsequently limiting the performance of underlying image processing algorithms. Current state-of-the-art (SoTA) convolutional neural networks (CNN) are developed as post-processing techniques to independently recover under-/over-exposed images. However, when applied to images containing real-world degradations such as glare, high-beam, color bleeding with varying noise intensity, these algorithms amplify the degradations, further degrading image quality. We propose a lightweight two-stage image enhancement algorithm sequentially balancing illumination and noise removal using frequency priors for structural guidance to overcome these limitations. Furthermore, to ensure realistic image quality, we leverage the relationship between frequency and spatial domain properties of an image and propose a Fourier spectrum-based adversarial framework (AFNet) for consistent image enhancement under varying illumination conditions. While current formulations of image enhancement are envisioned as postprocessing techniques, we examine if such an algorithm could be extended to integrate the functionality of the Image Signal Processing (ISP) pipeline within the camera sensor benefiting from RAW sensor data and lightweight CNN architecture. Based on quantitative and qualitative evaluations, we also examine the practicality and effects of image enhancement techniques on the performance of common perception tasks such as object detection and semantic segmentation in varying illumination conditions.

show abstract

Section: Performance Evaluation With Sota Algorithmsmentioning

confidence: 99%

Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks

Shyam¹,

Sengar²,

Yoon³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The image enhancement is a general idea to improve the image quality. It is addressed by a list of subtasks including demosaicking [58,5], deblurring [24,25], super-resolution [45,57,44], etc. The performance of image enhancement has been greatly improved through the data-driven deep learning approaches.…”

Section: Related Workmentioning

confidence: 99%

Legacy Photo Editing with Learned Noise Prior

Zhao

Lin

et al. 2021

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)

Self Cite

View full text Add to dashboard Cite

There are quite a number of photographs captured under undesirable conditions in the last century. Thus, they are often noisy, regionally incomplete, and grayscale formatted. Conventional approaches mainly focus on one point so that those restoration results are not perceptually sharp or clean enough. To solve these problems, we propose a noise prior learner NEGAN to simulate the noise distribution of real legacy photos using unpaired images. It mainly focuses on matching high-frequency parts of noisy images through discrete wavelet transform (DWT) since they include most of noise statistics. We also create a large legacy photo dataset for learning noise prior. Using learned noise prior, we can easily build valid training pairs by degrading clean images. Then, we propose an IEGAN framework performing image editing including joint denoising, inpainting and colorization based on the estimated noise prior. We evaluate the proposed system and compare it with state-of-the-art image enhancement methods. The experimental results demonstrate that it achieves the best perceptual quality. Please see the webpage https://github.com/zhaoyuzhi/Legacy-Photo-Editing-with-Learned-Noise-Prior for the codes and the proposed LP dataset.

show abstract

“…The W-Net is another model with the two-stage approach proposed by [16], which stacks two U-Net [12] structure with channel attention module. The SalGAN [21] employs the U-Net structure as the generator of the adversarial training scheme and incorporates spatial attention scheme into the loss function. The HERN [11] modifies the channel attention module of the residual in residual module of [19] to construct a dual-path network and incorporates global feature information with a separate full-image encoder.…”

Section: Related Workmentioning

confidence: 99%

PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing

Kim

Song

et al. 2020

Computer Vision – ECCV 2020 Workshops

View full text Add to dashboard Cite

Reconstructing RGB image from RAW data obtained with a mobile device is related to a number of image signal processing (ISP) tasks, such as demosaicing, denoising, etc. Deep neural networks have shown promising results over hand-crafted ISP algorithms on solving these tasks separately, or even replacing the whole reconstruction process with one model. Here, we propose PyNET-CA, an end-to-end mobile ISP deep learning algorithm for RAW to RGB reconstruction. The model enhances PyNET, a recently proposed state-of-the-art model for mobile ISP, and improve its performance with channel attention and subpixel reconstruction module. We demonstrate the performance of the proposed method with comparative experiments and results from the AIM 2020 learned smartphone ISP challenge. The source code of our implementation is available at https://github.com/egyptdj/skyb-aim2020-public

show abstract

Saliency Map-Aided Generative Adversarial Network for RAW to RGB Mapping

Cited by 9 publications

References 15 publications

Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks

Lightweight HDR Camera ISP for Robust Perception in Dynamic Illumination Conditions via Fourier Adversarial Networks

Legacy Photo Editing with Learned Noise Prior

PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing

Contact Info

Product

Resources

About