Event-Guided Image Super-Resolution Reconstruction

Abstract

The event camera efficiently detects scene radiance changes and produces an asynchronous event stream with low latency, high dynamic range (HDR), high temporal resolution, and low power consumption. However, the large output data caused by the asynchronous imaging mechanism makes the increase in spatial resolution of the event camera limited. In this paper, we propose a novel event camera super-resolution (SR) network (EFSR-Net) based on a deep learning approach to address the problems of low spatial resolution and poor visualization of event cameras. The network model is capable of reconstructing high-resolution (HR) intensity images using event streams and active sensor pixel (APS) frame information. We design the coupled response blocks (CRB) in the network that are able of fusing the feature information of both data to achieve the recovery of detailed textures in the shadows of real images. We demonstrate that our method is able to reconstruct high-resolution intensity images with more details and less blurring in synthetic and real datasets, respectively. The proposed EFSR-Net can improve the peak signal-to-noise ratio (PSNR) metric by 1–2 dB compared with state-of-the-art methods.

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

“…Asynchronous spatiotemporal pulse signals are presented as sparse discrete lattices in three-dimensional space in both time and space domains, which are more flexible in signal processing and feature expression than the traditional "image frame" paradigm, especially in the time domain length of the pulse signal processing unit or the choice of the number of pulses also increases the difficulty of inputting the visual analysis algorithm of asynchronous spatiotemporal pulse signals. Therefore, how to express the characteristics of asynchronous spatiotemporal pulse signals (Guo et al, 2023), as shown in Figure 10, mining the spatiotemporal characteristics of asynchronous spatiotemporal pulse signals for the visual analysis tasks of "high precision" and "high speed" is the most important method in the field of neuromorphic vision. Important and core research issues also determine the promotion and application of neuromorphic vision sensors.…”

“…Deep learning method: The frequency accumulation image is input into the deep learning network based on "image frame". Lai Image representation of frequency accumulation of asynchronous pulse signal (Guo et al, 2023).…”

“…Hand-designed features have better performance in specific vision task applications, but hand-designed features require a lot of prior knowledge, in-depth understanding of task requirements and data characteristics, and a lot of debugging work. Therefore, using the task-driven cascade method to supervise learning and expressing features (Guo et al, 2023) can better exploit the spatiotemporal characteristics of asynchronous spatiotemporal pulse signals.…”

“…FIGURE 10Image representation of frequency accumulation of asynchronous pulse signal(Guo et al, 2023).…”