Wavelet Frame Based Image Restoration via Combined Sparsity and Nonlocal Prior of Coefficients

Abstract

In digital images, the performance of optical aberration is a multivariate degradation, where the spectral of the scene, the lens imperfections, and the field of view together contribute to the results. Besides eliminating it at the hardware level, the post-processing system, which utilizes various prior information, is significant for correction. However, due to the content differences among priors, the pipeline that aligns these factors shows limited efficiency and unoptimized restoration.Here, we propose a prior quantization model to correct the optical aberrations in image processing systems. To integrate these messages, we encode various priors into a latent space and quantify them by the learnable codebooks. After quantization, the prior codes are fused with the image restoration branch to realize targeted optical aberration correction. Comprehensive experiments demonstrate the flexibility of the proposed method and validate its potential to accomplish targeted restoration for a specific camera. Furthermore, our model promises to analyze the correlation between the various priors and the optical aberration of devices, which is helpful for joint soft-hardware design.

Wavelet frame-based image restoration using sparsity, nonlocal, and support prior of frame coefficients

“…Therefore, inspired by the previous work [22,23], the authors in [11] proposed a nonlocal estimation method of the frame coefficients.…”

“…However, the computation costs of these methods are even higher than some patch-based approaches, e.g., the MATLAB implementation of the algorithm in [38] takes about 9 minute for a natural image with size 256 × 256, on a normal laptop. Chen et.al [11] proposed an l 0 -l 2 minimization model to balance the computational time and recovery quality. In their proposed model, nonlocal prior of the frame coefficients is incorporated in the variational model, in terms of the l 2 norm.…”

“…• Different with the l 0 -l 2 minimization model proposed in [11], where both sparsity prior and nonlocal prior are exploited, the support prior of frame coefficients is also respected 3 in our developed truncated l 0 -l 2 minimization model, leading to significantly better edge preserving performance.…”

“…In the next section, we first give a brief review of wavelet tight frames. Then we further revisit the current wavelet frame based image recovery models using either the single l 1 or l 0 regularization term [6,17], and the combined l 0 -l 2 minimization model [11]. In section 3, we propose a new wavelet frame based image recovery model utilizing the truncated l 0 -l 2 regularizer and develop a corresponding efficient algorithm in the spirit of iterative support detection.…”

“…Section 4 is devoted to the experimental evaluation of the proposed image restoration method. We compare it with several state-of-art methods such as the split bregman method for solving the wavelet frame based l 1 minimization model [6], the MDAL method for solving the l 0 regularization model [17], the algorithm proposed in [11] for l 0 -l 2 regularizer, and the famous Iterative Decoupled Deblurring BM3D (IDD-BM3D) algorithm [18,19]. Section 5 is devoted to the conclusion of this paper and discussions on some possible future work.…”