AIM 2019 Challenge on RAW to RGB Mapping: Methods and Results

Ignatov, Andrey; Li, Juncheng; Zhang, Jiajie; Wu, Haoyu; Li, Jie; Huang, Rui; Haris, Muhammad; Shakhnarovich, Greg; Ukita, Norimichi; Zhao, Yuzhi; Po, Lai-Man; Timofte, Radu; Zhang, Tiantian; Liao, Zongbang; Shi, Xiang; Zhang, Yujia; Ou, Weifeng; Xian, Pengfei; Xiong, Jian; Zhou, Chang; Yu, Wing Yin; Yubin, Yubin; Ko, Sung-Jea; Hou, Bingxin; Park, Bumjun; Yu, Songhyun; Kim, Sang-Min; Kim, Seung Wook; Uhm, Kwang-Hyun; Ji, Seo-Won; Cho, Sung‐Jin; Hong, Jun‐Pyo; Mei, Kangfu

doi:10.1109/iccvw.2019.00443

Cited by 30 publications

(13 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Learned End-to-End ISP: [25,29] • Perceptual Image Enhancement: [28,22] • Bokeh Effect Rendering: [21,27] • Image Super-Resolution: [28,41,4,48] Hsyu, Wen-Chia Tsai, Chao-Wei Chen Affiliations: Industrial Technology Research Institute (ITRI), Taiwan…”

Section: Additional Literaturementioning

confidence: 99%

“…To address this problem, in [30] the authors proposed to work directly with the original RAW Bayer sensor data and learn all ISP steps with a single deep neural network. The experiments conducted on the collected Zurich RAW to RGB dataset containing RAW-RGB image pairs captured by a mobile camera sensor and a high-end DSLR camera demonstrated that the proposed solution was able to get to the level of commercial ISP of the Huawei P20 cameraphone, while these results were later improved in [29,7,45,34,25]. In this challenge, we take one step further in solving this problem by using more advanced data and by putting additional efficiency-related constraints on the developed solutions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

Ignatov¹,

Chiang²,

Kuo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

As the quality of mobile cameras starts to play a crucial role in modern smartphones, more and more attention is now being paid to ISP algorithms used to improve various perceptual aspects of mobile photos. In this Mobile AI challenge, the target was to develop an end-to-end deep learning-based image signal processing (ISP) pipeline that can replace classical hand-crafted ISPs and achieve nearly real-time performance on smartphone NPUs. For this, the participants were provided with a novel learned ISP dataset consisting of RAW-RGB image pairs captured with the Sony IMX586 Quad Bayer mobile sensor and a professional 102megapixel medium format camera. The runtime of all models was evaluated on the MediaTek Dimensity 1000+ platform with a dedicated AI processing unit capable of accelerating both floating-point and quantized neural networks. The proposed solutions are fully compatible with the above NPU and are capable of processing Full HD photos under 60-100 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

show abstract

Section: Additional Literaturementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

Ignatov¹,

Chiang²,

Kuo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Zurich RAW2RGB Dataset [10] (ZRR) is a novel datasets for the task of phone camera images enhancement. It provides over 90K aligned image patch pairs include RAW images from a phone camera and RGB images from a DSLR correspondingly.…”

Section: Datasetsmentioning

confidence: 99%

HighEr-Resolution Network for Image Demosaicing and Enhancing

Mei

Zhang³

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

Neural-networks based image restoration methods tend to use low-resolution image patches for training. Although higher-resolution image patches can provide more global information, state-of-the-art methods cannot utilize them due to their huge GPU memory usage, as well as the instable training process. However, plenty of studies have shown that global information is crucial for image restoration tasks like image demosaicing and enhancing. In this work, we propose a HighEr-Resolution Network (HERN) to fully learning global information in high-resolution image patches. To achieve this, the HERN employs two parallel paths to learn image features in two different resolution, respectively. By combining global-aware features and multi-scale features, our HERN is able to learn global information with feasible GPU memory usage. Besides, we introduce a progressive training method to solve the instability issue and accelerate model convergence. On the task of image demosaicing and enhancing, our HERN achieves state-of-the-art performance on the AIM2019 RAW to RGB mapping challenge. The source code of our implementation is available at https://github.com/MKFMIKU/ RAW2RGBNet.

show abstract

“…AIM 2019 raw-to-RGB mapping challenge [11] consists of two tracks: fidelity track (Track 1) and perceptual track (Track 2). In the Track 1, the average PSNR and SSIM are calculated.…”

Section: Aim 2019 Raw-to-rgb Mapping Challengementioning

confidence: 99%

“…In this paper, we describe our solution for the AIM 2019 challenge on raw-to-RGB mapping [11]. The challenge releases a Zurich Raw-to-RGB (ZRR) data set for the task.…”

Section: Introductionmentioning

confidence: 99%

W-Net: Two-Stage U-Net With Misaligned Data for Raw-to-RGB Mapping

Uhm

Kim

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

Self Cite

View full text Add to dashboard Cite

Recent research on learning a mapping between raw Bayer images and RGB images has progressed with the development of deep convolutional neural network. A challenging data set namely the Zurich Raw-to-RGB data set (ZRR) has been released in the AIM 2019 raw-to-RGB mapping challenge. In ZRR, input raw and target RGB images are captured by two different cameras and thus not perfectly aligned. Moreover, camera metadata such as white balance gains and color correction matrix are not provided, which makes the challenge more difficult. In this paper, we explore an effective network structure and a loss function to address these issues. We exploit a two-stage U-Net architecture, and also introduce a loss function that is less variant to alignment and more sensitive to color differences. In addition, we show an ensemble of networks trained with different loss functions can bring a significant performance gain. We demonstrate the superiority of our method by achieving the highest score in terms of both the peak signal-to-noise ratio and the structural similarity and obtaining the second-best mean-opinion-score in the challenge.

show abstract

AIM 2019 Challenge on RAW to RGB Mapping: Methods and Results

Cited by 30 publications

References 9 publications

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

Learned Smartphone ISP on Mobile NPUs with Deep Learning, Mobile AI 2021 Challenge: Report

HighEr-Resolution Network for Image Demosaicing and Enhancing

W-Net: Two-Stage U-Net With Misaligned Data for Raw-to-RGB Mapping

Contact Info

Product

Resources

About