Weighted Tensor Nuclear Norm Minimization for Color Image Restoration

Abstract

Non-local self-similarity (NLSS) is widely used as prior information in an image restoration method. In particular, a low-rankness-based prior has a significant effect on performance. On the other hand, a number of color extensions of NLSS-based grayscale image restoration methods have been developed. These extensions focus on the pixel-wise correlation among color channels. However, a natural color image also has a complex dependency, known as an inter-channel dependency, among local regions from different color channels. As a result, color artifacts appear in a denoised image obtained by using the existing methods. In this paper, we propose a novel non-local and inter-channel dependency-aware prior called the weighted tensor nuclear norm (WTNN). The proposed prior is derived by incorporating inter-channel dependency to low-rank-based NLSS prior. The WTNN is a low-rankness-of-the-third-order patch tensor, and we apply it to the tensors constructed with non-local similar patches. It enables us to naturally represent the higher-order dependencies among similar color patches. We propose an image denoising algorithm using the WTNN and image restoration algorithm by using a non-trivial generalization of this algorithm. The experimental results clearly show that the proposed WTNN-based color image denoising and restoration algorithms outperform state-of-the-art methods.

MSGAN: Generative Adversarial Networks for Image Seasonal Style Transfer

“…Traditional methods mainly focus on low-dimensional image features, such as pixels, edges, colors and contents [8], [11]. These features are very important in many tasks, such as image restoration [12], image enhancement [13] and image segmentation [14], but they are not completely consistent to style transfer task. Some image processing operations must be carried out to generate training data pairs.…”

Real Color Image Denoising Using t-Product- Based Weighted Tensor Nuclear Norm Minimization

“…However, the transformed color images inevitably suffer from the information loss without fully considering cross-channel correlation. The third one was to jointly denoise the RGB channels by fully considering the cross-channel dependency [18], [19], [31]- [33]. References [31]- [33] concatenated the patches from RGB channels as a vector and utilized the weighted nuclear norm minimization (WNNM), dictionary learning and weighted sparse coding schemes respectively for real-world image denoising.…”

“…Directly unfolding or flattening a tensor would lead to information loss by destroying the multi-way structure of the data. Hosono et al proposed WTNNM for color image processing [18], [19]. The differences between WTNNM in [18], [19] and the proposed t-product based WTNNM lie in two aspects: the calculation of the tensor nuclear norm (TNN) and the assumption of noise for different color channels.…”

“…Hosono et al proposed WTNNM for color image processing [18], [19]. The differences between WTNNM in [18], [19] and the proposed t-product based WTNNM lie in two aspects: the calculation of the tensor nuclear norm (TNN) and the assumption of noise for different color channels. In [18] and [19], the TNN was calculated via tensor's matricization in [22] while the TNN in this paper is a new definition rigorously deduced from t-product [26].…”

“…The differences between WTNNM in [18], [19] and the proposed t-product based WTNNM lie in two aspects: the calculation of the tensor nuclear norm (TNN) and the assumption of noise for different color channels. In [18] and [19], the TNN was calculated via tensor's matricization in [22] while the TNN in this paper is a new definition rigorously deduced from t-product [26]. We assume the noise to be additive white Gaussian noise (AWGN) with different variances for different color channels as in [27] and [20], whereas the noise was assumed to be homogeneous AWGN without considering different variances for different channels in [18], [19].…”