EvIntSR-Net: Event Guided Multiple Latent Frames Reconstruction and Super-resolution

Han, Jin; Yang, Yixin; Zhou, Chu; Xu, Chao; Shi, Boxin

doi:10.1109/iccv48922.2021.00484

Cited by 30 publications

(19 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These data contain dynamic information about the edges of the object. However, the event streams are represented in terms of logarithmic intensity changes, which have a completely different data format than the general intensity images [ 17 ]. Therefore, the event data are generally processed when reconstruction operations are performed.…”

Section: Related Workmentioning

confidence: 99%

Event-Guided Image Super-Resolution Reconstruction

Guo

Feng

et al. 2023

Sensors

View full text Add to dashboard Cite

The event camera efficiently detects scene radiance changes and produces an asynchronous event stream with low latency, high dynamic range (HDR), high temporal resolution, and low power consumption. However, the large output data caused by the asynchronous imaging mechanism makes the increase in spatial resolution of the event camera limited. In this paper, we propose a novel event camera super-resolution (SR) network (EFSR-Net) based on a deep learning approach to address the problems of low spatial resolution and poor visualization of event cameras. The network model is capable of reconstructing high-resolution (HR) intensity images using event streams and active sensor pixel (APS) frame information. We design the coupled response blocks (CRB) in the network that are able of fusing the feature information of both data to achieve the recovery of detailed textures in the shadows of real images. We demonstrate that our method is able to reconstruct high-resolution intensity images with more details and less blurring in synthetic and real datasets, respectively. The proposed EFSR-Net can improve the peak signal-to-noise ratio (PSNR) metric by 1–2 dB compared with state-of-the-art methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Event-Guided Image Super-Resolution Reconstruction

Guo

Feng

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Zhang et al [44] emphasize the temporal correlation among consecutive frames and design a multipatch convolutional LSTM to exploit such correlation. Han et al [7] extend this idea by modeling the intensity residual between neighboring sharp frames. Xu et al [41] also identify the importance of temporal correlation and propose to utilize the optical flow estimation instead.…”

Section: Event-enhanced Deblurringmentioning

confidence: 99%

“…In the refinement stage, we solve the first issue by optimizing the frames independent of polynomial formulation. We solve the second issue by encouraging visual features to propagate between consecutive frames (i.e., enforcing motion priors), a popular technique used extensively in video synthesis via optical flows [3,41], residuals [7,18], or deformable convolutional kernels [37,38]. Details of our refinement process are presented as Algorithm 1.…”

Section: Algorithm 1 Refinementmentioning

confidence: 99%

“…Sharp video reconstruction is an ill-posed problem because there are infinitely many motion trajectories whose temporal averages correspond to the same blurry frame. To compensate for the ambiguity, previous works [7,8,21,25,26,36,40,41,44] exploit event data as an auxiliary input, which provides additional information during the exposure interval at a finer temporal resolution, as shown in Figure 1. Even with the event input, difficult challenges remain.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

E-CIR: Event-Enhanced Continuous Intensity Recovery

Chen¹,

Huang²,

Bajaj³

2022

Preprint

View full text Add to dashboard Cite

A camera begins to sense light the moment we press the shutter button. During the exposure interval, relative motion between the scene and the camera causes motion blur, a common undesirable visual artifact. This paper presents E-CIR, which converts a blurry image into a sharp video represented as a parametric function from time to intensity. E-CIR leverages events as an auxiliary input. We discuss how to exploit the temporal event structure to construct the parametric bases. We demonstrate how to train a deep learning model to predict the function coefficients. To improve the appearance consistency, we further introduce a refinement module to propagate visual features among consecutive frames. Compared to state-of-the-art event-enhanced deblurring approaches, E-CIR generates smoother and more realistic results.

show abstract

“…Single image super-resolution (SISR) is a classical computer vision problem that tries to infer a high-resolution (HR) image from a single low-resolution (LR) input image. This problem is still an active research field in the computer vision community (e.g., [ 1 , 2 , 3 , 4 ]). Several applications in different fields can benefit from super-resolution (SR) representations, for instance, security (e.g., [ 5 , 6 ]), medical imaging (e.g., [ 7 ]), object detection (e.g., [ 8 ]), and astronomical images (e.g., [ 9 ]), among others.…”

Section: Introductionmentioning

confidence: 99%

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

Rivadeneira

Sappa

Vintimilla

et al. 2022

Sensors

View full text Add to dashboard Cite

This paper presents a transfer domain strategy to tackle the limitations of low-resolution thermal sensors and generate higher-resolution images of reasonable quality. The proposed technique employs a CycleGAN architecture and uses a ResNet as an encoder in the generator along with an attention module and a novel loss function. The network is trained on a multi-resolution thermal image dataset acquired with three different thermal sensors. Results report better performance benchmarking results on the 2nd CVPR-PBVS-2021 thermal image super-resolution challenge than state-of-the-art methods. The code of this work is available online.

show abstract

EvIntSR-Net: Event Guided Multiple Latent Frames Reconstruction and Super-resolution

Cited by 30 publications

References 33 publications

Event-Guided Image Super-Resolution Reconstruction

Event-Guided Image Super-Resolution Reconstruction

E-CIR: Event-Enhanced Continuous Intensity Recovery

A Novel Domain Transfer-Based Approach for Unsupervised Thermal Image Super-Resolution

Contact Info

Product

Resources

About