A Convex Approach for Variational Super-Resolution

Unger, Markus; Pock, Thomas; Werlberger, Manuel; Bischof, Horst

doi:10.1007/978-3-642-15986-2_32

Cited by 35 publications

(52 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To highlight these results in this paper, we show log intensity and velocity field estimates at certain time-stamps to highlight how they resemble images and optical flow fields from standard cameras. However, we believe that these results are best viewed in the accompanying video on our project webpage 1 , where also show that we can estimate super-resolution log-intensity and velocity, via a simple extension to our formulation based on the work of Unger et al for standard cameras [22]. Although we do not discuss the super-resolution method in detail here, it is important to highlight that event camera data allows us to perform intensity and velocity estimation at sub-pixel resolution.…”

Section: Methodsmentioning

confidence: 71%

Simultaneous Optical Flow and Intensity Estimation from an Event Camera

Bardow

Davison

Leutenegger

2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

222

219

View full text Add to dashboard Cite

Event cameras are bio-inspired vision sensors which mimic retinas to measure per-pixel intensity change rather than outputting an actual intensity image. This proposed paradigm shift away from traditional frame cameras offers significant potential advantages: namely avoiding high data rates, dynamic range limitations and motion blur. Unfortunately, however, established computer vision algorithms may not at all be applied directly to event cameras. Methods proposed so far to reconstruct images, estimate optical flow, track a camera and reconstruct a scene come with severe restrictions on the environment or on the motion of the camera, e.g. allowing only rotation. Here, we propose, to the best of our knowledge, the first algorithm to simultaneously recover the motion field and brightness image, while the camera undergoes a generic motion through any scene. Our approach employs minimisation of a cost function that contains the asynchronous event data as well as spatial and temporal regularisation within a sliding window time interval. Our implementation relies on GPU optimisation and runs in near real-time. In a series of examples, we demonstrate the successful operation of our framework, including in situations where conventional cameras suffer from dynamic range limitations and motion blur.

show abstract

Section: Methodsmentioning

confidence: 71%

Simultaneous Optical Flow and Intensity Estimation from an Event Camera

Bardow

Davison

Leutenegger

2016

2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

222

219

View full text Add to dashboard Cite

show abstract

“…The energy combines discrete and continuous components: a sum over the n discrete pixel samples, and a continuous integral over spatial locations x, and it is our claim that optimization of this discrete/continuous energy has previously been proposed only in strictly special cases. Work that comes close to minimizing (6) includes [20], who compute an integral over a piecewise-continuous grid representation of u, and [13], who compute the integral over a quadrilateral. Similarly, discrete superresolution techniques [3, §5.4], describe an area-sampling approximation to the integral, but none refine the underlying grid, or allow arbitrary boundary positions.…”

Section: The New Modelmentioning

confidence: 99%

“…Again given V, the integral becomes a sum over triangles κ1(x)u t φ(x)dx, computed as the sum over all triangles t which intersect Qi. The integral is easily computed by convex polygon clipping, then applying Green's theorem on the returned boundary (20). Note that the pixels at the image edge need not be specially treated: triangles which intersect any pixel contribute to the integral, and those which intersect no pixel will be filled by the prior.…”

Section: Computing the Regularizer Termmentioning

confidence: 99%

A unifying resolution-independent formulation for early vision

Viola

Fitzgibbon

Cipolla

2012

2012 IEEE Conference on Computer Vision and Pattern Recognition

View full text Add to dashboard Cite

We present a model for early vision tasks such as denoising, super-resolution, deblurring, and demosaicing. The model provides a resolution-independent representation of discrete images which admits a truly rotationally invariant prior. The model generalizes several existing approaches: variational methods, finite element methods, and discrete random fields. The primary contribution is a novel energy functional which has not previously been written down, which combines the discrete measurements from pixels with a continuous-domain world viewed through continousdomain point-spread functions. The value of the functional is that simple priors (such as total variation and generalizations) on the continous-domain world become realistic priors on the sampled images. We show that despite its apparent complexity, optimization of this model depends on just a few computational primitives, which although tedious to derive, can now be reused in many domains. We define a set of optimization algorithms which greatly overcome the apparent complexity of this model, and make possible its practical application. New experimental results include infinite-resolution upsampling, and a method for obtaining "subpixel superpixels".

show abstract

“…It is inherently ill-posed, as several different HR images can map to the very same LR image. The field can be mainly divided into methods where an edge-preserving smoothness term is utilized [38], co-occurrences of patches within the same image are exploited [14], or, currently most successful, a mapping from LR to HR image patches is learned [8,32,37,42].…”

Section: Related Workmentioning

confidence: 99%

A Deep Primal-Dual Network for Guided Depth Super-Resolution

Riegler¹,

Ferstl²,

Rüther³

et al. 2016

Procedings of the British Machine Vision Conference 2016

Self Cite

View full text Add to dashboard Cite

In this paper we present a novel method to increase the spatial resolution of depth images. We combine a deep fully convolutional network with a non-local variational method in a deep primal-dual network. The joint network computes a noise-free, highresolution estimate from a noisy, low-resolution input depth map. Additionally, a highresolution intensity image is used to guide the reconstruction in the network. By unrolling the optimization steps of a first-order primal-dual algorithm and formulating it as a network, we can train our joint method end-to-end. This not only enables us to learn the weights of the fully convolutional network, but also to optimize all parameters of the variational method and its optimization procedure. The training of such a deep network requires a large dataset for supervision. Therefore, we generate high-quality depth maps and corresponding color images with a physically based renderer. In an exhaustive evaluation we show that our method outperforms the state-of-the-art on multiple benchmarks.

show abstract

A Convex Approach for Variational Super-Resolution

Cited by 35 publications

References 16 publications

Simultaneous Optical Flow and Intensity Estimation from an Event Camera

Simultaneous Optical Flow and Intensity Estimation from an Event Camera

A unifying resolution-independent formulation for early vision

A Deep Primal-Dual Network for Guided Depth Super-Resolution

Contact Info

Product

Resources

About