Advances and challenges in super‐resolution

Farsiu, Sina; Robinson, Michael; Elad, Michael; Milanfar, Peyman

doi:10.1002/ima.20007

Cited by 546 publications

(254 citation statements)

References 44 publications

Supporting

Mentioning

242

Contrasting

Order By: Relevance

“…In that case, we would shift the kernel toward the wrong direction, and the MASK weights would be less effective for temporal upscaling. Therefore, one should incorporate a test of the reliability of m 4 into the process, and use vectors m i instead of m ′ i if it is found to be unreliable. Our specific technique to compute the reliability of motion vectors is described in Section 4.2.…”

Section: Spatial Upscaling and Temporal Frame Interpolationmentioning

confidence: 99%

“…Although many algorithms have been proposed for image and video interpolation, spatial upscaling and frame interpolation (temporal upscaling) are generally treated separately. The conventional super-resolution technique for spatial upscaling consists of image reconstruction from irregularly sampled pixels, provided by registering multiple low resolution frames onto a high resolution grid using motion estimation, see [16,4] for overviews. A recent work by Narayanan et al ([14]) proposed a video-to-video super resolution algorithm using a partition filtering technique, in which local image structures are classified into vertical, horizontal, and diagonal edges, textures, and flat areas by vector quantization [6] (involving off-line learning), and prepare a suitable filter for each structure class beforehand.…”

Section: Introductionmentioning

confidence: 99%

“…Although we consider local motion vectors in MASK, we propose an algorithm that is robust against errors in the estimated motion field. Prior multi-frame resolution-enhanced or super-resolution (SR) reconstruction methods ( [4,16]) often consider only global translational or affine motions; local motion and object occlusions are often not addressed. Many SR methods require explicit motion compensation, which may involve interpolation or rounding of displacements to grid locations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Spatiotemporal Video Upscaling Using Motion-Assisted Steering Kernel (MASK) Regression

Takeda

Beek²,

Milanfar

2010

Signals and Communication Technology

Self Cite

View full text Add to dashboard Cite

In this chapter, we present Motion Assisted Steering Kernel (MASK) regression, a novel multi-frame approach for interpolating video data spatially, temporally, or spatiotemporally, and for video noise reduction, including compression artifact removal. The MASK method takes both local spatial orientations and local motion vectors into account and adaptively constructs a suitable filter at every position of interest. Moreover, we present a practical algorithm based on MASK that is both robust and computationally efficient. In order to reduce the computational and memory requirements, we process each frame in a block-by-block manner, utilizing a block-based motion model. Instead of estimating the local dominant orientation by singular value decomposition, we estimate the orientations based on a technique similar to vector quantization. We develop a technique to locally adapt the regression order, which allows enhancing the denoising effect in flat areas, while effectively preserving major edges and detail in texture areas. Comparisons between MASK and other state-of-the-art video upscaling methods demonstrate the effectiveness of our approach.

show abstract

Section: Spatial Upscaling and Temporal Frame Interpolationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatiotemporal Video Upscaling Using Motion-Assisted Steering Kernel (MASK) Regression

Takeda

Beek²,

Milanfar

2010

Signals and Communication Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…We refer the interested readers to [1][2][3] for a broad review of recent algorithmic development in this area. Perhaps, the most fundamental component to multi-frame super-resolution is the accurate registration of aliased images.…”

Section: Introductionmentioning

confidence: 99%

Optimal Registration Of Aliased Images Using Variable Projection With Applications To Super-Resolution

Robinson

Farsiu

Milanfar

2008

The Computer Journal

View full text Add to dashboard Cite

Accurate registration of images is the most important and challenging aspect of multiframe image restoration problems such as super-resolution. The accuracy of super-resolution algorithms is quite often limited by the ability to register a set of low-resolution images. The main challenge in registering such images is the presence of aliasing. In this paper, we analyse the problem of jointly registering a set of aliased images and its relationship to super-resolution. We describe a statistically optimal approach to multiframe registration which exploits the concept of variable projections to achieve very efficient algorithms. Finally, we demonstrate how the proposed algorithm offers accurate estimation under various conditions when standard approaches fail to provide sufficient accuracy for super-resolution.

show abstract

“…The first SR methods did not involve any deblurring; they just tried to register the LR images with subpixel accuracy and then to resample them on a high-resolution grid. A good survey of SR techniques can be found in Park et al (2003), Farsui et al (2004). Maximum likelihood (ML), maximum a posteriori (MAP), the set theoretic approach using POCS (projection on convex sets), and fast Fourier techniques can all provide a solution to the SR problem.…”

Section: D(·) = S(g * ·)mentioning

confidence: 99%

Image Fusion: A Powerful Tool for Object Identification

Šroubek

Flusser

Zitová

Imaging for Detection and Identification

View full text Add to dashboard Cite

Abstract. Due to imperfections of imaging devices (optical degradations, limited resolution of CCD sensors) and instability of the observed scene (object motion, media turbulence), acquired images are often blurred, noisy and may exhibit insufficient spatial and/or temporal resolution. Such images are not suitable for object detection and recognition. Reliable detection requires recovering the original image. If multiple images of the scene are available, this can be achieved by image fusion.In this chapter we review the respective methods of image fusion. We address all three major steps -image registration, blind deconvolution and resolution enhancement. Image registration brings the acquired images into spatial alignment, multiframe deconvolution estimates and removes the blur, and the spatial resolution of the image is increased by so-called superresolution fusion. Superresolution is the main topic of the chapter. We propose a unifying system that simultaneously estimates blurs and recovers the original undistorted image, all in high resolution, without any prior knowledge of the blurs and original image. We accomplish this by formulating the problem as constrained least squares energy minimization with appropriate regularization terms, which guarantees a close-to-perfect solution.We demonstrate the performance of the method on many examples, namely on car license plate recognition and face recognition. Both of these tasks are of great importance in security and surveillance systems.

show abstract

Advances and challenges in super‐resolution

Cited by 546 publications

References 44 publications

Spatiotemporal Video Upscaling Using Motion-Assisted Steering Kernel (MASK) Regression

Spatiotemporal Video Upscaling Using Motion-Assisted Steering Kernel (MASK) Regression

Optimal Registration Of Aliased Images Using Variable Projection With Applications To Super-Resolution

Image Fusion: A Powerful Tool for Object Identification

Contact Info

Product

Resources

About