Local denoising applied to RAW images may outperform non-local patch-based methods applied to the camera output

Ghimpeteanu, Gabriela; Batard, Thomas; Seybold, Tamara; Bertalmío, Marcelo

doi:10.2352/issn.2470-1173.2016.18.dpmi-015

Cited by 7 publications

(1 citation statement)

References 9 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Later on, another non-local denoising approach, referred to as BM3D [13] uses the sample idea and exploits the sparsity further. There are also discussions with respect to patch-based scheme [19]. As the deep learning era arrives, more and more works are using Convolutional Neural Network(CNN) or Generative Adversarial Network(GAN) that beats most of the classical, sophisticated methods [55,21,50,38,24].…”

Section: Image Filtering and Enhancementmentioning

confidence: 99%

Recursive Self-Improvement for Camera Image and Signal Processing Pipeline

Bajaj¹,

Wang²,

Yang³

2021

Preprint

View full text Add to dashboard Cite

Current camera image and signal processing pipelines (ISPs), including deep trained versions, tend to apply a single filter that is uniformly applied to the entire image. This despite the fact that most acquired camera images have spatially heterogeneous artifacts. This spatial heterogeneity manifests itself across the image space as varied Moire ringing, motion-blur, color-bleaching or lens based projection distortions. Moreover, combinations of these image artifacts can be present in small or large pixel neighborhoods, within an acquired image. Here, we present a deep reinforcement learning model that works in learned latent subspaces, recursively improves camera image quality through a patch-based spatially adaptive artifact filtering and image enhancement. Our RSE-RL model views the identification and correction of artifacts as a recursive self-learning and self-improvement exercise and consists of two major sub-modules: (i) The latent feature sub-space clustering/grouping obtained through an equivariant variational auto-encoder enabling rapid identification of the correspondence and discrepancy between noisy and clean image patches. (ii) The adaptive learned transformation controlled by a trust-region soft actor-critic agent that progressively filters and enhances the noisy patches using its closest feature distance neighbors of clean patches. Artificial artifacts that may be introduced in a patch-based ISP, are also removed through a reward based de-blocking recovery and image enhancement. We demonstrate the self-improvement feature of our model by recursively training and testing on images, wherein the enhanced images resulting from each epoch provide a natural data augmentation and robustness to the RSE-RL training-filtering pipeline.

show abstract