Learning Event-Based Motion Deblurring

Aarents

IEEE Robot. Autom. Lett.

et al. 2021

160

140

Once an academic venture, autonomous driving has received unparalleled corporate funding in the last decade. Still, operating conditions of current autonomous cars are mostly restricted to ideal scenarios. This means that driving in challenging illumination conditions such as night, sunrise, and sunset remains an open problem. In these cases, standard cameras are being pushed to their limits in terms of low light and high dynamic range performance. To address these challenges, we propose, DSEC, a new dataset that contains such demanding illumination conditions and provides a rich set of sensory data. DSEC offers data from a wide-baseline stereo setup of two color frame cameras and two highresolution monochrome event cameras. In addition, we collect lidar data and RTK GPS measurements, both hardware synchronized with all camera data. One of the distinctive features of this dataset is the inclusion of high-resolution event cameras. Event cameras have received increasing attention for their high temporal resolution and high dynamic range performance. However, due to their novelty, event camera datasets in driving scenarios are rare. This work presents the first high resolution, large scale stereo dataset with event cameras. The dataset contains 53 sequences collected by driving in a variety of illumination conditions and provides ground truth disparity for the development and evaluation of event-based stereo algorithms.

Section: Introductionmentioning

confidence: 99%

DSEC: A Stereo Event Camera Dataset for Driving Scenarios

Aarents

IEEE Robot. Autom. Lett.

et al. 2021

160

140

“…We compare our method with other state-of-the-art deblurring methods [16], [17], [32], [33], [22], [20], [18], [21], [29], [45] on three datasets as mentioned above. Unless stated otherwise, all the reported results are directly copied from the original paper and the null stand for the result cannot be found in the original paper and its open source code cannot be found.…”

Section: B Performance Comparisonsmentioning

confidence: 99%

Multi-Scale Grid Network for Image Deblurring With High-Frequency Guidance

Liu

Fang

Wang

et al. 2022

IEEE Trans. Multimedia

It has been demonstrated that the blurring process reduces the high-frequency information of the original sharp image, so the main challenge for image deblurring is to reconstruct high-frequency information from the blurry image. In this paper, we propose a novel image deblurring framework to focus on the reconstruction of high-frequency information, which consists of two main subnetworks: a high-frequency reconstruction subnetwork (HFRSN) and a multi-scale grid subnetwork (MSGSN). The HFRSN is built to reconstruct latent high-frequency information from multiple scale blurry images. The MSGSN performs deblurring processes with high-frequency guidance at different scales simultaneously. Besides, in order to better use high-frequency information to restore sharpening images, we designed a high-frequency information aggregation (HFAG) module and a high-frequency information attention (HFAT) module in MSGSN. The HFAG module is designed to fuse high-frequency features and image features at the feature extraction stage, and the HFAT module is built to enhance the feature reconstruction stage. Extensive experiments on different datasets show the effectiveness and efficiency of our method.

IEEE Robot. Autom. Lett.

“…9b the same scene as the camera rotates 360 • /s. Existing studies on image deblurring in event cameras focus on deblurring the low frame rate intensity image of the DAVIS camera using the high temporal resolution event data [14], [15], [16].…”

Section: Motion Blur In Event Camerasmentioning

confidence: 99%

Aggregation Functions for Simultaneous Attitude and Image Estimation With Event Cameras at High Angular Rates

Soh

et al. 2022

For fast-moving event cameras, projection of events onto the image frame exhibits smearing of events analogous to high motion blur. For camera attitude estimation, this presents a causality dilemma where motion prior is required to unsmear events, but an image prior is required to estimate motion. This dilemma is typically circumvented by including an IMU to provide motion priors. However, IMUs limited dynamic range of ±2000 • /s are shown to be insufficient for high angular rate rotorcrafts. Contrast Maximization is an event-only optimization framework that computes the optimal motion compensation parameter while generating an event image simultaneously. This paper analyses the performance of existing aggregation functions of the contrast maximization framework and proposes a nonconvolution-based aggregation function that outperforms existing implementations. The use of discrete event images for optimizers is discussed, demonstrating alternate avenues of the framework to exploit. The effect of motion blur in motion-compensated images is defined and studied for Contrast Maximisation at high angular rates. Lastly, the framework is applied to rotation datasets with angular rates exceeding 2000 • /s to demonstrate high angular rate motion estimation without motion priors.