An Unsupervised Weight Map Generative Network for Pixel-Level Combination of Image Denoisers

Abstract

Image denoising is a classic but still important issue in image processing as the denoising effect has a significant impact on subsequent image processing results, such as target recognition and edge detection. In the past few decades, various denoising methods have been proposed, such as model-based and learning-based methods, and they have achieved promising results. However, no stand-alone method consistently outperforms the others in different complex imaging situations. Based on the complementary strengths of model-based and learning-based methods, in this study, we design a pixel-level image combination strategy to leverage their respective advantages for the denoised images (referred to as initial denoised images) generated by individual denoisers. The key to this combination strategy is to generate a corresponding weight map of the same size for each initial denoised image. To this end, we introduce an unsupervised weight map generative network that adjusts its parameters to generate a weight map for each initial denoised image under the guidance of our designed loss function. Using the weight maps, we are able to fully utilize the internal and external information of various denoising methods at a finer granularity, ensuring that the final combined image is close to the optimal. To the best of our knowledge, our enhancement method of combining denoised images at the pixel level is the first proposed in the image combination field. Extensive experiments demonstrate that the proposed method shows superior performance, both quantitatively and visually, and stronger generalization. Specifically, in comparison with the stand-alone denoising methods FFDNet and BM3D, our method improves the average peak signal-to-noise ratio (PSNR) by 0.18 dB to 0.83 dB on two benchmarking datasets crossing different noise levels. Its denoising effect is also greater than other competitive stand-alone methods and combination methods, and has surpassed the denoising effect of the second-best method by 0.03 dB to 1.42 dB. It should be noted that since our image combination strategy is generic, the proposed combined strategy can not only be used for image denoising but can also be extended to low-light image enhancement, image deblurring or image super-resolution.

Zero-shot denoising of microscopy images recorded at high-resolution limits

“…The diversity of the different methods studied here may also be of interest from another perspective: Especially methods based on different principles and showing strengths in different aspects and for different tasks can potentially be combined very well by meta-approaches such as boosting. One recent example in this direction is a method that combines denoised images from different methods based on a pixel-wise weight map learned from an unsupervised generative network [ 61 ], which shows that improved denoising performance can be achieved. Future work may use similar combinations and take denoising as well as sharpness preserving measures into account, or future work may study integrations of, e.g., methods with strong noise reduction and methods with good sharpness preservation.…”

Learning from Multiple Instances: A Two-Stage Unsupervised Image Denoising Framework Based on Deep Image Prior

“…The two methods we previously proposed achieved very good performance on synthetic noisy image denoising. However, it should be noted that the denoiser proposed in [32] did not improve the performance of real-world noisy images, and the method proposed in [31] cannot be used for real-world image denoising.…”

“…In Table 3, we also compare the performance of two methods proposed in our previous works [31,32]. In [31], we first employed a set of state-of-the-art image denoisers to preprocess the given noisy image and obtain multiple intermediate denoised images.…”

“…These intermediate denoised images were then concatenated and fed into an unsupervised deep fusion network to generate the final denoised image. In [32], we utilized two state-of-the-art denoisers to generate the preliminary denoised images and proposed an unsupervised fusion network to fuse the two preliminary denoised images into the final denoised image. As shown in Table 3, the performance of the proposed method is slightly lower than that of our previous methods.…”

“…Therefore, the performance of state-of-the-art denoising methods degrades sharply when denoising noisy real-world images. To further demonstrate the effectiveness of our proposed method, we tested it on real-world noisy images and compared its denoised results with those of CBM3D [1], LIME [34], FFDNet, Restormer [12], VDNet [33], Yu2022 [32], and DIP. Figure 10 shows one low-light image from the SCIE [35] dataset and its denoised versions generated using different methods.…”