Space–time super-resolution with patch group cuts prior

Li, Tao; He, Xiaohai; Teng, Qizhi; Wang, Zhengyong; Ren, Chao

doi:10.1016/j.image.2014.10.007

Cited by 22 publications

(13 citation statements)

References 27 publications

(61 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this problem, most existing research works [10], [25], [37], [49], [51], [56] often introduce some prior knowledge, make strong assumptions, and adopt hand-crafted regularization. For instance, Shechtman et al [51] adopted space-time directional smoothness prior.…”

Section: Introductionmentioning

confidence: 99%

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiang

Tian

Zhang

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

174

210

View full text Add to dashboard Cite

In this paper, we address the space-time video super-resolution, which aims at generating a high-resolution (HR) slow-motion video from a low-resolution (LR) and low frame rate (LFR) video sequence. A naive method is to decompose it into two sub-tasks: video frame interpolation (VFI) and video super-resolution (VSR). Nevertheless, temporal interpolation and spatial upscaling are intra-related in this problem. Two-stage approaches cannot fully make use of this natural property. Besides, state-of-the-art VFI or VSR deep networks usually have a large frame reconstruction module in order to obtain high-quality photo-realistic video frames, which makes the two-stage approaches have large models and thus be relatively time-consuming. To overcome the issues, we present a one-stage space-time video super-resolution framework, which can directly reconstruct an HR slow-motion video sequence from an input LR and LFR video. Instead of reconstructing missing LR intermediate frames as VFI models do, we temporally interpolate LR frame features of the missing LR frames capturing local temporal contexts by a feature temporal interpolation module. Extensive experiments on widely used benchmarks demonstrate that the proposed framework not only achieves better qualitative and quantitative performance on both clean and noisy LR frames but also is several times faster than recent state-of-the-art two-stage networks. The source code is released in https:// github.com/ Mukosame/ Zooming-Slow-Mo-CVPR-2020.

show abstract

Section: Introductionmentioning

confidence: 99%

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Xiang

Tian

Zhang

et al. 2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

174

210

View full text Add to dashboard Cite

show abstract

“…While this formulation handles spatial and temporal information separately, earlier methods [52,53] have solved the joint problem of STSR by optimizing very large objective terms. Mudenagudi et al [36] adopt graph-cut, and Li et al [24] utilize group cuts prior to deal with the problem. Recently, STARnet [15] has proposed an end-to-end learnable framework for STSR problem using deep CNNs.…”

Section: Related Workmentioning

confidence: 99%

NTIRE 2021 Challenge on Video Super-Resolution

Son¹,

Lee²,

Nah³

et al. 2021

Preprint

View full text Add to dashboard Cite

Super-Resolution (SR) is a fundamental computer vision task that aims to obtain a high-resolution clean image from the given low-resolution counterpart. This paper reviews the NTIRE 2021 Challenge on Video Super-Resolution. We present evaluation results from two competition tracks as well as the proposed solutions. Track 1 aims to develop conventional video SR methods focusing on the restoration quality. Track 2 assumes a more challenging environment with lower frame rates, casting spatio-temporal SR problem. In each competition, 247 and 223 participants have registered, respectively. During the final testing phase, 14 teams competed in each track to achieve state-of-the-art performance on video SR tasks.

show abstract

“… 1 In addition to the development of faster and more sensitive cameras and clever pixel rebinning methods, 2 various sensing and computational approaches to increase the temporal resolution of microscopes have been proposed. Some rely on multiple observations of a signal 3 – 6 or make clever use of the signal structure itself, e.g., its sparsity in a known basis 7 – 9 or its repeatable nature. 10 , 11 The ability to modulate the illumination rapidly in a controlled and cost-effective way (in particular, with LED-based illuminators 12 ) also opens the way for promising methods.…”

Section: Introductionmentioning

confidence: 99%

Aliasing mitigation in optical microscopy of dynamic biological samples by use of temporally modulated color illumination and a standard RGB camera

Jaques

Liebling

2020

J. Biomed. Opt.

View full text Add to dashboard Cite

Significance: Despite recent developments in microscopy, temporal aliasing can arise when imaging dynamic samples. Modern sampling frameworks, such as generalized sampling, mitigate aliasing but require measurement of temporally overlapping and potentially negative-valued inner products. Conventional cameras cannot collect these directly as they operate sequentially and are only sensitive to light intensity. Aim: We aim to mitigate aliasing in microscopy of dynamic monochrome samples by implementing generalized sampling via the use of a color camera and modulated color illumination. Approach: We solve the overlap problem by spectrally multiplexing the acquisitions and using (positive) B-spline segments as projection kernels. Reconstruction involves spectral unmixing and inverse filtering. We implemented this method using a color LED illuminator. We evaluated its performance by imaging a rotating grid and its applicability by imaging the beating zebrafish embryo heart in transmission and light-sheet microscopes. Results: Compared to stroboscopic imaging, our method mitigates aliasing with performance improving as the projection order increases. The approach can be implemented in conventional microscopes but is limited by the number of available LED colors and camera channels. Conclusions: Generalized sampling can be implemented via color modulation in microscopy to mitigate temporal aliasing. The simple hardware requirements could make it applicable to other optical imaging modalities.

show abstract

Space–time super-resolution with patch group cuts prior

Cited by 22 publications

References 27 publications

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution

NTIRE 2021 Challenge on Video Super-Resolution

Aliasing mitigation in optical microscopy of dynamic biological samples by use of temporally modulated color illumination and a standard RGB camera

Contact Info

Product

Resources

About