When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach

Abstract

Conventionally, image denoising and high-level vision tasks are handled separately in computer vision. In this paper, we cope with the two jointly and explore the mutual influence between them. First we propose a convolutional neural network for image denoising which achieves the state-of-theart performance. Second we propose a deep neural network solution that cascades two modules for image denoising and various high-level tasks, respectively, and use the joint loss for updating only the denoising network via back-propagation. We demonstrate that on one hand, the proposed denoiser has the generality to overcome the performance degradation of different high-level vision tasks. On the other hand, with the guidance of highlevel vision information, the denoising network can generate more visually appealing results. To the best of our knowledge, this is the first work investigating the benefit of exploiting image semantics simultaneously for image denoising and high-level vision tasks via deep learning. The code is available online 1 .

A Review on Deep Learning in Medical Image Reconstruction

“…Recently, the authors of [155] proposes to realize task-based quality metric by "hooking" a image reconstruction network from unrolling dynamics with a image analysis DNN, so that the reconstructed images by the first network will be implicitly evaluated by the second which effectively makes the quality metric task-based. Similar idea appeared in computer vision for image denoising [94,93]. On the other than, these work also suggested a new "raw-to-task" modeling philosophy with encouraging empirical results.…”

“…The most simple and natural way of joining image reconstruction and image analysis is to connect the two networks together and conduct end-to-end training (from scratch or by fine-tuning). Such idea was first introduced by [155] in medical imaging and by [94,93] in computer vision for image denoising. By doing so, the second network for image analysis can be regarded as a task-based image quality metric that is learned from the data.…”

Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study

“…In [40,142], the joint optimization pipeline for low-resolution recognition is examined. In [41,42], Liu et al discussed the impact of denoising for semantic segmentation and advocated their mutual optimization. Lately, in [143], the algorithmic impact of enhancement algorithms for both visual quality and automatic object recognition is thoroughly examined, on a real image set with highly compound degradations.…”

“…Yet it remains questionable whether restoration-based approaches would actually boost the visual understanding performance, as the restoration/enhancement step is not optimized towards the target task and may bring in misleading information and artifacts too. For example, a recent line of researches [8,[39][40][41][42][43][44][45][46][47][48] discuss on the intrinsic interplay relationship of low-level vision and high-level recognition/detection tasks, showing that their goals are not always aligned.…”

VisionISP: Repurposing the Image Signal Processor for Computer Vision Applications

“…Recently, there has also been interest in designing image restoration and enhancement methods with the overall goal of improving high-level computer vision performance rather than perceptual image quality [10,11]. Earlier work showed how image degradations affect image recognition accuracy [12], how conventional low-level image restoration techniques do not necessarily improve the computer vision performance [10,13], and how jointly handling image restoration and recognition tasks can improve the results [10,11,14]. It has also been shown that imaging devices can save power when they are used for computer vision tasks by running in a lower resolution and precision mode and skipping processing steps that are traditionally used for enhancing image quality [13].…”