Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots

Wang, Zejin; Liu, Jiazheng; Li, Guoqing; Han, Ho Jae

doi:10.1109/cvpr52688.2022.00207

Cited by 78 publications

(49 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this way, the masked pixels would never be seen during training, which could easily lead to details loss or over-smoothing in the image. Blind2Unblind [33] used a global masker to generate interleaved blind spots and collected blind spots after model denoising to reconstruct the denoised output. The use of global masker solves this problem perfectly, but when the noise is large, the point-like mask used in Blind2Unblind [33] will be powerless.…”

Section: B Self-supervised Image Denoising Methodsmentioning

confidence: 99%

“…Blind2Unblind [33] used a global masker to generate interleaved blind spots and collected blind spots after model denoising to reconstruct the denoised output. The use of global masker solves this problem perfectly, but when the noise is large, the point-like mask used in Blind2Unblind [33] will be powerless. For spatially connected noise, AP-BSN [22] used pixel-shuffle downsampling (PD) to break the spatial connection of the noise, and then used the masked convolution kernel to extract features at the very beginning of the model.…”

Section: B Self-supervised Image Denoising Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Self-Supervised Image Denoising for Real-World Images With Context-Aware Transformer

Zhang¹,

Zhou²

2023

IEEE Access

View full text Add to dashboard Cite

In recent years, the development of deep learning has been pushing image denoising to a new level. Among them, self-supervised denoising is increasingly popular because it does not require any prior knowledge. Most of the existing self-supervised methods are based on convolutional neural networks (CNN), which are restricted by the locality of the receptive field and would cause color shifts or textures loss. In this paper, we propose a novel Denoise Transformer for real-world image denoising, which is mainly constructed with Context-aware Denoise Transformer (CADT) units and Secondary Noise Extractor (SNE) block. CADT is designed as a dual-branch structure, where the global branch uses a window-based Transformer encoder to extract the global information, while the local branch focuses on the extraction of local features with small receptive field. By incorporating CADT as basic components, we build a hierarchical network to directly learn the noise distribution information through residual learning and obtain the first stage denoised output. Then, we design SNE in low computation for secondary global noise extraction. Finally the blind spots are collected from the Denoise Transformer output and reconstructed, forming the final denoised image. Extensive experiments on the real-world SIDD benchmark achieve 50.62/0.990 for PSNR/SSIM, which is competitive with the current state-of-the-art method and only 0.17/0.001 lower. Visual comparisons on public sRGB, Raw-RGB and greyscale datasets prove that our proposed Denoise Transformer has a competitive performance, especially on blurred textures and low-light images, without using additional knowledge, e.g., noise level or noise type, regarding the underlying unknown noise. Image denoising, self-supervised, real-world, transformer, dual-branch. INDEX TERMS

show abstract

Section: B Self-supervised Image Denoising Methodsmentioning

confidence: 99%

Section: B Self-supervised Image Denoising Methodsmentioning

confidence: 99%

Self-Supervised Image Denoising for Real-World Images With Context-Aware Transformer

Zhang¹,

Zhou²

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Removing image noise and preserving image details are the pivotal problems of high-quality image denoising. Deep learning denoising technique [1], [2], [3], [4], relying on its powerful learning ability, has made outstanding achievements in the field of image denoising. Nevertheless, deep learning denoising methods depend on the training of massive data, which induces the denoising model less interpretable.…”

Section: Introductionmentioning

confidence: 99%

Image Denoising Algorithm Based on Gradient Domain Guided Filtering and NSST

Liu

Cheng

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Traditional image denoising methods, which do not depend on data training, have good interpretability. However, traditional image denoising methods hardly achieve the denoising effect of deep learning methods. Based on traditional image processing techniques, this paper proposes a new hybrid image denoising model. The block-batching and 3-D filtering (BM3D) algorithm is used to obtain the first denoised image. The weighted kernel norm minimization (WNNM) and non-subsampled shearlet transform (NSST) algorithms are successively adopted to get the second denoised image. By the gradient domain guided filtering, the texture information of the first denoised image is extracted to enhance the details of the second denoised image. Specially, we propose the adaptive iterative NSST algorithm based on the improved soft thresholding, in order to solve the problems about the discontinuity of the hard thresholding and the constant deviation of the soft thresholding. Our approach can not only attenuate excessive smoothing, but also restore the natural appearance of the image. Experiments are conducted to demonstrate that our proposed method enjoys PSNR and SSIM performance gains over several deep learning denoising methods.INDEX TERMS Gradient domain guided filtering, BM3D algorithm, improved soft threshold, nonsubsampled shearlet transform.

show abstract

“…Recently, a large number of U‐Net‐based models have been proposed for image denoising. For example, Wang et al 38 . proposed a modified U‐Net based network for achieving a self‐supervised image denoising task.…”

Section: Introductionmentioning

confidence: 99%

Domain‐adaptive denoising network for low‐dose CT via noise estimation and transfer learning

Wang

Tang

et al. 2022

Medical Physics

View full text Add to dashboard Cite

Background: In recent years, low-dose computed tomography (LDCT) has played an important role in the diagnosis CT to reduce the potential adverse effects of X-ray radiation on patients, while maintaining the same diagnostic image quality. Purpose: Deep learning (DL)-based methods have played an increasingly important role in the field of LDCT imaging. However, its performance is highly dependent on the consistency of feature distributions between training data and test data. Due to patient's breathing movements during data acquisition, the paired LDCT and normal dose CT images are difficult to obtain from realistic imaging scenarios. Moreover, LDCT images from simulation or clinical CT examination often have different feature distributions due to the pollution by different amounts and types of image noises. If a network model trained with a simulated dataset is used to directly test clinical patients' LDCT data, its denoising performance may be degraded. Based on this, we propose a novel domain-adaptive denoising network (DADN) via noise estimation and transfer learning to resolve the out-of -distribution problem in LDCT imaging. Methods: To overcome the previous adaptation issue, a novel network model consisting of a reconstruction network and a noise estimation network was designed. The noise estimation network based on a double branch structure is used for image noise extraction and adaptation. Meanwhile, the U-Net-based reconstruction network uses several spatially adaptive normalization modules to fuse multi-scale noise input. Moreover, to facilitate the adaptation of the proposed DADN network to new imaging scenarios, we set a two-stage network training plan. In the first stage, the public simulated dataset is used for training. In the second transfer training stage, we will continue to fine-tune the network model with a torso phantom dataset, while some parameters are frozen. The main reason using the two-stage training scheme is based on the fact that the feature distribution of image content from the public dataset is complex and diverse,whereas the feature distribution of noise pattern from the torso phantom dataset is closer to realistic imaging scenarios. Results: In an evaluation study, the trained DADN model is applied to both the public and clinical patient LDCT datasets. Through the comparison of visual inspection and quantitative results, it is shown that the proposed DADN network model can perform well in terms of noise and artifact suppression, while effectively preserving image contrast and details. Conclusions: In this paper, we have proposed a new DL network to overcome the domain adaptation problem in LDCT image denoising. Moreover, the results demonstrate the feasibility and effectiveness of the application of our proposed DADN network model as a new DL-based LDCT image denoising method.

show abstract

Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots

Cited by 78 publications

References 27 publications

Self-Supervised Image Denoising for Real-World Images With Context-Aware Transformer

Self-Supervised Image Denoising for Real-World Images With Context-Aware Transformer

Image Denoising Algorithm Based on Gradient Domain Guided Filtering and NSST

Domain‐adaptive denoising network for low‐dose CT via noise estimation and transfer learning

Contact Info

Product

Resources

About