End-to-End Learning with Memory Models for Complex Autonomous Driving Tasks in Indoor Environments

Abstract

The interest in autonomous vehicles has increased exponentially in recent years. While Lidar is a proven autonomous driving technology, end-to-end learning approaches have become popular as computer performance has improved. A fully end-to-end method—NVIDIA’s PilotNet has shown its ability to predict speed and steering angle with only camera images. This method achieved the Lidar-based methods’ performance in simple driving tasks. However, a significant drawback was no past spatiotemporal information, imposing an error-sensitive performance, especially in complex driving tasks. Spurred by this deficiency, this paper introduces two novel models: CNN + LSTM and CNN3D, aiming for complex autonomous driving tasks in indoor environments.

Artificial intelligence-based spatio-temporal vision sensors: applications and prospects

“…Autonomous driving: Feng et al (Feng et al, 2020) used CeleX for the driver's fatigue detection system. Lai et al (Lai and Braunl, 2023) used DVS for steering wheel angle prediction in autonomous driving scenarios. Ryan et al (Ryan et al, 2021) used a combination of DVS and APS for The performance of joint detection of targets for autonomous driving is significantly improved in high-speed motion scenes and extreme lighting conditions.…”

“…et al(Lai and Braunl, 2023) accumulated the ON and OFF pulse streams into grayscale images according to the frequency in the time domain, and then used ResNet to predict the steering wheel angle of the autonomous driving scene Zeng et al (Zeng et al, 2023). used the pseudo-label of APS for vehicle detection in autonomous driving scenes after mapping the pulse stream output by DVS into a grayscale image.…”