Space-Time Super-Resolution Using Graph-Cut Optimization

Abstract

We address the problem of super-resolution—obtaining high-resolution images and videos from multiple low-resolution inputs. The increased resolution can be in spatial or temporal dimensions, or even in both. We present a unified framework which uses a generative model of the imaging process and can address spatial super-resolution, space-time super-resolution, image deconvolution, single-image expansion, removal of noise, and image restoration. We model a high-resolution image or video as a Markov random field and use maximum a posteriori estimate as the final solution using graph-cut optimization technique. We derive insights into what super-resolution magnification factors are possible and the conditions necessary for super-resolution. We demonstrate spatial super-resolution reconstruction results with magnifications higher than predicted limits of magnification. We also formulate a scheme for selective super-resolution reconstruction of videos to obtain simultaneous increase of resolutions in both spatial and temporal directions. We show that it is possible to achieve space-time magnification factors beyond what has been suggested in the literature by selectively applying super-resolution constraints. We present results on both synthetic and real input sequences.

MAP-MRF-Based Super-Resolution Reconstruction Approach for Coded Aperture Compressive Temporal Imaging

“…MAP-MRF can be adopted for the estimation on multi-frame reconstructed HR images and results in the energy minimization problem being suitable for inference algorithms based on graph cuts or belief propagation [29].…”

Space–time super-resolution with patch group cuts prior

“…To suppress these two visual artifacts in video sequence, Shechtman et al [19] is the first to propose a space-time SR reconstruction algorithm, in which a series of LR video sequences with (sub-pixel) spatial and (sub-frame) temporal misalignments is used to generate a space-time HR sequence. Then, improvements on efficiency and visual effects have been discussed in many literatures [20][21][22], which are fundamentally based on the multiple-video SR model in [19]. For example, Mudenagudi et al [21] extend the multiple-video SR model in [19] using a nonlinear maximum a posterioriMarkov random field (MAP-MRF) framework, and propose to minimize the cost function using graph-cut optimization.…”

“…Then, improvements on efficiency and visual effects have been discussed in many literatures [20][21][22], which are fundamentally based on the multiple-video SR model in [19]. For example, Mudenagudi et al [21] extend the multiple-video SR model in [19] using a nonlinear maximum a posterioriMarkov random field (MAP-MRF) framework, and propose to minimize the cost function using graph-cut optimization. However, to make the cost function to satisfy the regularity constraint of graph-cut optimization, sophisticated prior cannot be used.…”

Learning-Based Video Superresolution Reconstruction Using Spatiotemporal Nonlocal Similarity

“…SR techniques have been developed to solve SR problems from the frequency domain to the spatial domain. Currently relevant studies include three main categories: interpolation-based SR methods [5,6], multiframe-based SR methods [7][8][9], and learning-based SR methods [10,11]. Interpolation-based SR methods have relatively low computational cost and therefore are well suited for real-time applications.…”