Super Resolution Reconstruction via Multiple Frames Joint Learning

Abstract

This paper presents a novel multi-frame joint learning approach for image super resolution via sparse representation. Based on the assumption that several low-resolution patches degraded from a same high-resolution patch under subpixel translation can preserve similar structures, we can use those similar low-resolution patches together to recover the sparse coefficients for the corresponding high-resolution patch, and the differences between them can help to supply more information. So, unlike the learning-based super resolution algorithm from single image which uses one patch in the learning process, we take into consideration some other well matched patches in 3D domain. Computer simulations demonstrate that, comparing with those single frame learning algorithms, our method will not only restore more details but also can effectively overcome the over learning and is more robust to noise.

Multi-frame image super resolution based on sparse coding

“…To the best of the authors' knowledge, there is only one extant multi-frame SR method based on sparse coding (Wang et al, 2011). In their paper, all of the observed LR images are used for HR image reconstruction, while in the proposed method, only informative LR images for reconstruction are selected for each patch.…”

“…Then, y 1 and D l 1 can be extracted and reconstructed in the same manner as singleframe SR. We have to extract and reconstruct other N − 1 patches and dictionaries. The previous study on multi-frame SR based on sparse coding (Wang et al, 2011) does not involve explicit sub-pixel accuracy block matching in the HR space. By adopting sub-pixel accuracy block matching, we can expect higher accuracy in the HR image reconstruction.…”

“…(3) for the target image Y 1 and X may not be satisfied after patch-wise SR. Hence, the back-projection (Irani & Peleg, 1993) is performed for maintaining the global consistency (Wang, Hu, Xuan, Mu, & Peng, 2011;Yang et al, 2008Yang et al, , 2010. Let X 0 be the obtained HR image by patch-wise SR and weighted averaging operation explained in the previous section.…”

Design of a Compressive Remote Imaging System Compensating a Highly Lightweight Encoding With a Refined Decoding Scheme

“…A straightforward approach would be to apply some of the wellestablished 2-dimensional interpolation techniques, such as a bilinear or a bicubic one. However, recent works (Freeman et al, 2002;Yang et al, 2008;Wang et al, 2011;Zhang et al, 2011) have shown that a super-resolution method results in images with superior quality. In the proposed CVS decoder we employ a dictionary-based super-resolution approach, as it is described in (Yang et al, 2008).…”

“…Moreover, the required bit-rate of our proposed encoder can be further decreased by downsampling the non-reference frames, followed by an additional super-resolution step at the decoder to restore the reconstructed frames in their original resolution. The use of super-resolution as a tool to resize the frames in their original dimension is motivated by recent works on sparse representation-based image super-resolution via dictionary learning (Freeman et al, 2002;Yang et al, 2008;Wang et al, 2011;Zhang et al, 2011), where it has been shown that a super-resolution method results in images with superior quality when compared to the commonly used 2-dimensional interpolation schemes (e.g., bilinear, bicubic, spline).…”