Robust super-resolution for interactive video navigation

Abstract

One of the main technical limitations in interactive systems for next-generation audio-visual experiences is the limited resolution of the captured contents. The presented method tackles the problem of generating super-resolved versions of input video frames, thus allowing the user to visualize the captured visual contents at any desired scale with minimal degradation. First, the low-frequency band of the super-resolved video frame is estimated as an up-scaled interpolation of the low-resolution frame. Then, the high-frequency band is extrapolated from the low-resolution frame by exploiting local cross-scale self-similarity. The introduction of a suitable image prior in both stages allows to robustly enhance the spatial resolution even in video sequences containing aliasing. The most demanding processing stages of the presented algorithm have been implemented on graphics hardware (GPU). The experimental results show a level of quality similar to that of state-of-the-art methods, with the advantages of real-time processing and robustness against spatial aliasing.

Optimized neighbor embeddings for single-image super-resolution

“…For the reported results of [9,10], images courtesy of their project web-pages. We acknowledge authors of [11] for providing the results of their algorithm. Fig.…”

“…After carrying a strictly-localized NN search in the low-frequency band, the missing high-frequency information of the initially interpolated image is retrieved from the high-frequency band of the input LR image. A similar technique has been proposed in [11] exploiting local cross-scale self-similarities to extrapolate the high-frequency band of the given LR image. This algorithm additionally imposes a total variation regularization [12] in its stages for a robust SR. Further multi-scale (e.g., pyramidal) SR models have also been considered in [13,14].…”