Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisation

Reinbacher, Christian; Graber, Gottfried; Pock, Thomas

doi:10.48550/arxiv.1607.06283

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bardow et al (Bardow, Davison, and Leutenegger 2016) employed primal-dual algorithm to simultaneously estimate optical flow and light intensity. Some other works (Reinbacher, Graber, and Pock 2016;Scheerlinck, Barnes, and Mahony 2018) reconstructed images with direct event integration. Recently, many works (Scheerlinck et al 2020;Rebecq et al 2019a,b;Stoffregen et al 2020;Wang et al 2019a;Ahmed et al 2021;Choi, Yoon et al 2020;Jiang et al 2020) explored to use deep convolutional networks for event camera reconstruction.…”

Section: Related Workmentioning

confidence: 99%

Learning to Super-resolve Dynamic Scenes for Neuromorphic Spike Camera

Zhao

Xiong

Zhang

et al. 2023

AAAI

View full text Add to dashboard Cite

Spike camera is a kind of neuromorphic sensor that uses a novel ``integrate-and-fire'' mechanism to generate a continuous spike stream to record the dynamic light intensity at extremely high temporal resolution. However, as a trade-off for high temporal resolution, its spatial resolution is limited, resulting in inferior reconstruction details. To address this issue, this paper develops a network (SpikeSR-Net) to super-resolve a high-resolution image sequence from the low-resolution binary spike streams. SpikeSR-Net is designed based on the observation model of spike camera and exploits both the merits of model-based and learning-based methods. To deal with the limited representation capacity of binary data, a pixel-adaptive spike encoder is proposed to convert spikes to latent representation to infer clues on intensity and motion. Then, a motion-aligned super resolver is employed to exploit long-term correlation, so that the dense sampling in temporal domain can be exploited to enhance the spatial resolution without introducing motion blur. Experimental results show that SpikeSR-Net is promising in super-resolving higher-quality images for spike camera.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning to Super-resolve Dynamic Scenes for Neuromorphic Spike Camera

Zhao

Xiong

Zhang

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…However, results show that naively composing the different color channels into an RGB image produces results that suffer from high salt-and-pepper type noise and poor color quality (see Figure 4 subfigure C in [29]). A more sophisticated approach to color interpolation from different color channels, such as the ones employed in [29][30][31] (classical algorithms or filters) and [32] (neural network solution) produce better results, especially in terms of noise, but still suffer from poor color quality (see the comparison between [29] and our method in the Result section).…”

Section: Related Workmentioning

confidence: 99%

Illumination-Based Color Reconstruction for the Dynamic Vision Sensor

Cohen,

Hershko,

Levy

et al. 2023

Sensors

View full text Add to dashboard Cite

This work demonstrates a novel, state-of-the-art method to reconstruct colored images via the dynamic vision sensor (DVS). The DVS is an image sensor that indicates only a binary change in brightness, with no information about the captured wavelength (color) or intensity level. However, the reconstruction of the scene’s color could be essential for many tasks in computer vision and DVS. We present a novel method for reconstructing a full spatial resolution, colored image utilizing the DVS and an active colored light source. We analyze the DVS response and present two reconstruction algorithms: linear-based and convolutional-neural-network-based. Our two presented methods reconstruct the colored image with high quality, and they do not suffer from any spatial resolution degradation as other methods. In addition, we demonstrate the robustness of our algorithm to changes in environmental conditions, such as illumination and distance. Finally, compared with previous works, we show how we reach the state-of-the-art results. We share our code on GitHub.

show abstract

“…Most of the previous works model the contrast threshold as being consistent across all pixels in the event camera and ignore biased pixels altogether [24,22,11,12]. This can be seen as a special case of our more general model in (5) and can be written as…”

Section: Special Pixelsmentioning

confidence: 99%

“…In DVS, contrast threshold defines the logarithmic intensity change that will trigger an event. It is commonly assumed that the contrast threshold is a constant (typically 10%) for all pixels [24,22,11,12]. However, due to a number of factors like event generation speed, intensity change, circuit noise, and manufacturing imperfections, the actual contrast threshold at each pixel may be different from the desired contrast threshold.…”

Section: Introductionmentioning

confidence: 99%

Event Camera Calibration of Per-pixel Biased Contrast Threshold

Wang,

Ng,

van Goor

et al. 2020

Preprint

View full text Add to dashboard Cite

Event cameras output asynchronous events to represent intensity changes with a high temporal resolution, even under extreme lighting conditions. Currently, most of the existing works use a single contrast threshold to estimate the intensity change of all pixels. However, complex circuit bias and manufacturing imperfections cause biased pixels and mismatch contrast threshold among pixels, which may lead to undesirable outputs. In this paper, we propose a new event camera model and two calibration approaches which cover event-only cameras and hybrid image-event cameras. When intensity images are simultaneously provided along with events, we also propose an efficient online method to calibrate event cameras that adapts to time-varying event rates. We demonstrate the advantages of our proposed methods compared to the state-of-the-art on several different event camera datasets.

show abstract

Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisation

Cited by 3 publications

References 0 publications

Learning to Super-resolve Dynamic Scenes for Neuromorphic Spike Camera

Learning to Super-resolve Dynamic Scenes for Neuromorphic Spike Camera

Illumination-Based Color Reconstruction for the Dynamic Vision Sensor

Event Camera Calibration of Per-pixel Biased Contrast Threshold

Contact Info

Product

Resources

About