HDR Video Reconstruction with Tri-Exposure Quad-Bayer Sensors

Jiang, Yitong; Choi, Inchang; Jiang, Jun; Gu, Jun

doi:10.48550/arxiv.2103.10982

Cited by 2 publications

(7 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other works use a custom rolling shutter [11] or deblur using information from a camera's default rolling shutter [40]. Jeon et al [41] use communication theory to design fluttering patterns, while more recently, Jiang et al [14] use an interlaced short, medium, and long exposure pattern with a specialized network for reconstruction. We compare this exposure pattern with ours, while using a simpler network for reconstruction.…”

Section: Motion Deblurringmentioning

confidence: 99%

“…• Quad (B, S): Following Jiang et al [14], we use a fixed tile coded arrangement of LMMS (long-mediummedium-short), where the short pixel is 240 fps, medium pixels are 120 fps, and long pixel is 30 fps. We use this baseline to also initialize L-SVPE which explains its resemblance (Fig.…”

Section: Nonad (Nine-tuple) (B S)mentioning

confidence: 99%

“…It would be additionally useful to expand E2E methods like ours to programmable sensors that can capture different color channels as well. Different colors captured at different exposures such as that of Jiang et al [14] can provide helpful cues in reconstruction.…”

Section: Future Workmentioning

confidence: 99%

“…On the other hand, computational imaging methods have excelled in combining software and hardware solutions to simplify many ill-posed computer vision tasks. These include engineering point spread functions (PSFs) [7], [8], [9] and custom coded exposures [10], [11], [12], [13], [14] for motion deblurring. However, these methods are often limited to heuristic designs, and often restricted by fabrication limits, sensor capabilities, or human intuition.…”

Section: Introductionmentioning

confidence: 99%

“…The ground truth is the first frame of an input video segment, and the blurred image is all eight frames of the video segment averaged, which is the same as a 30 fps capture. The reconstruction from the Quad exposure [14] with bilinear interpolation shows more motion and color artifacts compared to that of the learned spatially varying pixel exposure (L-SVPE).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Nguyen¹,

Martel²,

Wetzstein³

2022

Preprint

View full text Add to dashboard Cite

Computationally removing the motion blur introduced by camera shake or object motion in a captured image remains a challenging task in computational photography. Deblurring methods are often limited by the fixed global exposure time of the image capture process. The post-processing algorithm either must deblur a longer exposure that contains relatively little noise or denoise a short exposure that intentionally removes the opportunity for blur at the cost of increased noise. We present a novel approach of leveraging spatially varying pixel exposures for motion deblurring using next-generation focal-plane sensor-processors along with an end-to-end design of these exposures and a machine learning-based motion-deblurring framework. We demonstrate in simulation and a physical prototype that learned spatially varying pixel exposures (L-SVPE) can successfully deblur scenes while recovering high frequency detail. Our work illustrates the promising role that focal-plane sensor-processors can play in the future of computational imaging.

show abstract

Section: Motion Deblurringmentioning

confidence: 99%

Section: Nonad (Nine-tuple) (B S)mentioning

confidence: 99%

Section: Future Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Nguyen¹,

Martel²,

Wetzstein³

2022

Preprint

View full text Add to dashboard Cite

show abstract

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Wang

Yoon

2021

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) imaging is a technique that allows an extensive dynamic range of exposures, which is important in image processing, computer graphics, and computer vision. In recent years, there has been a significant advancement in HDR imaging using deep learning (DL). This study conducts a comprehensive and insightful survey and analysis of recent developments in deep HDR imaging methodologies. We hierarchically and structurally group existing deep HDR imaging methods into five categories based on (1) number/domain of input exposures, (2) number of learning tasks, (3) novel sensor data, (4) novel learning strategies, and (5) applications. Importantly, we provide a constructive discussion on each category regarding its potential and challenges. Moreover, we review some crucial aspects of deep HDR imaging, such as datasets and evaluation metrics. Finally, we highlight some open problems and point out future research directions.

show abstract

HDR Video Reconstruction with Tri-Exposure Quad-Bayer Sensors

Cited by 2 publications

References 37 publications

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Deep Learning for HDR Imaging: State-of-the-Art and Future Trends

Contact Info

Product

Resources

About