Robust patch-based hdr reconstruction of dynamic scenes

Sen, Pradeep; Kalantari, Nima Khademi; Yaesoubi, Maziar; Darabi, Soheil; Goldman, Dan B; Shechtman, Eli

doi:10.1145/2366145.2366222

Cited by 358 publications

(389 citation statements)

References 44 publications

Supporting

Mentioning

383

Contrasting

Unclassified

Order By: Relevance

“…Another advantage is that the saturated pixels are not organized in large regions. Indeed, some recent multi-image methods tackle motion problems by taking a reference image and then by estimating motion or reconstructing the image relative to this reference [31], [35]. A problem encountered by these approaches is the need to inpaint very large saturated and underexposed regions in the reference frame.…”

Section: Snapshotmentioning

confidence: 99%

A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, With an Application to HDR Imaging

Aguerrebere

Almansa

Delon

et al. 2017

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Abstract-Recently, impressive denoising results have been achieved by Bayesian approaches which assume Gaussian models for the image patches. This improvement in performance can be attributed to the use of per-patch models. Unfortunately such an approach is particularly unstable for most inverse problems beyond denoising. In this work, we propose the use of a hyperprior to model image patches, in order to stabilize the estimation procedure. There are two main advantages to the proposed restoration scheme: Firstly it is adapted to diagonal degradation matrices, and in particular to missing data problems (e.g. inpainting of missing pixels or zooming). Secondly it can deal with signal dependent noise models, particularly suited to digital cameras. As such, the scheme is especially adapted to computational photography. In order to illustrate this point, we provide an application to high dynamic range imaging from a single image taken with a modified sensor, which shows the effectiveness of the proposed scheme.Index Terms-Non-local patch-based restoration, Bayesian restoration, Maximum a Posteriori, Gaussian Mixture Models, hyper-prior, conjugate distributions, high dynamic range imaging, single shot HDR, hierarchical models.

show abstract

Section: Snapshotmentioning

confidence: 99%

A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, With an Application to HDR Imaging

Aguerrebere

Almansa

Delon

et al. 2017

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

show abstract

“…To reconstruct the HDR images from the LDR inputs, Sen et al [2012] had proposed a patch-based optimization system for still HDR photography that satisfied two properties: 1) the final HDR image Hn should be very close to the reference image n after mapping it to the radiance domain h(Lref,n) wherever Lref,n is wellexposed, and 2) Hn should include information from the captured images at the M different exposures neighboring frame n. Although this often works well for still images, their method is unsuitable for our application since it lacks temporal coherency (see ThrowingTowel3Exp in the supplementary materials), a necessity for high-quality HDR video. Furthermore, their method can also generate unsatisfactory results when a large region of the reference image is under-or over-exposed.…”

Section: Exposuresmentioning

confidence: 99%

“…However, most of the methods that address artifacts in dynamic scenes (e.g., [Zimmer et al 2011;Sen et al 2012]) only produce still images and cannot be used for HDR video.…”

Section: Introductionmentioning

confidence: 99%

“…Our method builds upon the recent work on HDR reconstruction for still images that poses the problem as a patch-based optimization [Sen et al 2012]. Although this approach produces highquality still HDR images, it is unsuitable for HDR video due to the lack of temporal coherency (see, e.g., ThrowingTowel3Exp in the supplementary materials 1 ).…”

Section: Introductionmentioning

confidence: 99%

“…Our key contribution is to combine optical flow with a patch-based synthesis approach similar to Sen et al [2012] to achieve temporal coherency. We show that a simple combination of the two components does not work well and propose a method to compute spatially-varying search windows for handling complex motions.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Patch-based high dynamic range video

et al. 2013

Self Cite

View full text Add to dashboard Cite

Figure 1: (top row) Input video acquired using an off-the-shelf camera, which alternates between three exposures separated by two stops. (bottom row) Our algorithm reconstructs the missing LDR images and generates an HDR image at each frame. The HDR video result for this ThrowingTowel3Exp scene can be found in the supplementary materials. This layout is adapted from Kang et al. [2003]. AbstractDespite significant progress in high dynamic range (HDR) imaging over the years, it is still difficult to capture high-quality HDR video with a conventional, off-the-shelf camera. The most practical way to do this is to capture alternating exposures for every LDR frame and then use an alignment method based on optical flow to register the exposures together. However, this results in objectionable artifacts whenever there is complex motion and optical flow fails. To address this problem, we propose a new approach for HDR reconstruction from alternating exposure video sequences that combines the advantages of optical flow and recently introduced patch-based synthesis for HDR images. We use patch-based synthesis to enforce similarity between adjacent frames, increasing temporal continuity. To synthesize visually plausible solutions, we enforce constraints from motion estimation coupled with a search window map that guides the patch-based synthesis. This results in a novel reconstruction algorithm that can produce high-quality HDR videos with a standard camera. Furthermore, our method is able to synthesize plausible texture and motion in fast-moving regions, where either patch-based synthesis or optical flow alone would exhibit artifacts. We present results of our reconstructed HDR video sequences that are superior to those produced by current approaches.

show abstract

High Dynamic Range Imaging

Mantiuk

Seidel

2015

Wiley Encyclopedia of Electrical and Electronics Engineering

View full text Add to dashboard Cite

High dynamic range (HDR) images and video contain pixels, which can represent much greater range of colors and brightness levels than that offered by existing, standard dynamic range images. Such “better pixels” greatly improve the overall quality of visual content, making it appear much more realistic and appealing to the audience. HDR is one of the key technologies of the future imaging pipeline, which will change the way the digital visual content is represented and manipulated. This article offers a broad review of the HDR methods and technologies with an introduction to fundamental concepts behind the perception of HDR imagery. It serves as both an introduction to the subject and a review of the current state of the art in HDR imaging. It covers the topics related to capture of HDR content with cameras and its generation with computer graphics methods; encoding and compression of HDR images and video; tone mapping for displaying HDR content on standard dynamic range displays; inverse tone mapping for upscaling legacy content for presentation on HDR displays; the display technologies offering HDR range; and finally image and video quality metrics suitable for HDR content.

show abstract

Robust patch-based hdr reconstruction of dynamic scenes

Cited by 358 publications

References 44 publications

A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, With an Application to HDR Imaging

A Bayesian Hyperprior Approach for Joint Image Denoising and Interpolation, With an Application to HDR Imaging

Patch-based high dynamic range video

High Dynamic Range Imaging

Contact Info

Product

Resources

About