Learning HDR video reconstruction for dual-exposure sensors with temporally-alternating exposures

Cogalan, Ugur; Bemana, Mojtaba; Seidel, Hans‐Peter; Ritschel, Tobias

doi:10.1016/j.cag.2022.04.008

Cited by 6 publications

(3 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The dynamic range that we can reconstruct is limited by the exposure ratio of four that we assume in this work. For larger ratios, the accuracy of HDR frame reconstruction by the MakeHDR network might be reduced [CBM * 22], which could adversely affect the accuracy of HDR video interpolation. Moreover, when capturing an HDR scene, we adjust the lowest exposure time in such a way that the long exposure is not very saturated so that there is enough valuable blurry information.…”

Section: Resultsmentioning

confidence: 99%

“…Multi‐exposure sensors [CMV21, Son22, Sam22] that, as we show in this work, are greatly beneficial for HDR VFI, require motion deblurring in longer exposures to reconstruct sharp HDR video [HST * 14, CKL14, JCJG21]. This task is relatively easy for machine learning solutions [CBM * 22], where recently proposed neural sensors [MMC * 20, NMW22] can learn spatially varying pixel exposures for efficient motion deblurring. The scope of all these methods is mainly limited to HDR video reconstruction, and they do not aim for the VFI task.…”

Section: Previous Workmentioning

confidence: 99%

“…Programmable sensors with spatially varying exposures greatly expand the dynamic range of contrast in captured video [HKU14, GKSK19, CBK17, HST * 14, CBM * 22, CKL14], and become an attractive choice for modern smartphones [GSM22], e.g. Sony's Quad Bayer [Son22], and Samsung's Tetracell/Nonacell [Sam22] technologies.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Video frame interpolation for high dynamic range sequences captured with dual‐exposure sensors

Cogalan

Bemana

Seidel

2023

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Video frame interpolation (VFI) enables many important applications such as slow motion playback and frame rate conversion. However, one major challenge in using VFI is accurately handling high dynamic range (HDR) scenes with complex motion. To this end, we explore the possible advantages of dual‐exposure sensors that readily provide sharp short and blurry long exposures that are spatially registered and whose ends are temporally aligned. This way, motion blur registers temporally continuous information on the scene motion that, combined with the sharp reference, enables more precise motion sampling within a single camera shot. We demonstrate that this facilitates a more complex motion reconstruction in the VFI task, as well as HDR frame reconstruction that so far has been considered only for the originally captured frames, not in‐between interpolated frames. We design a neural network trained in these tasks that clearly outperforms existing solutions. We also propose a metric for scene motion complexity that provides important insights into the performance of VFI methods at test time.

show abstract

Section: Resultsmentioning

confidence: 99%