Towards a framework for end-to-end control of a simulated vehicle with spiking neural networks

Kaiser, Jacques; Tieck, J. Camilo Vasquez; Hubschneider, Christian; Wolf, Peter; Weber, Michael; Hoff, Michael; Friedrich, Alexander; Wojtasik, Konrad; Roennau, Arne; Kohlhaas, Ralf; Dillmann, Rüdiger; Zöllner, J. Marius

doi:10.1109/simpar.2016.7862386

Cited by 50 publications

(64 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, studies have investigated self-driving cars based on driving commands and captured images [12][13][14]. While being driven, the car learns a model for self-driving based on the images and the driving commands.…”

Section: Research On Driving Methods Of Carsmentioning

confidence: 99%

“…Recently, studies on end-to-end control-based self-driving have been actively conducted to control vehicles using images captured by one or multiple cameras attached to them as input [12][13][14]. In general, the entire process of enabling a vehicle to drive autonomously consists of the following steps: recognizing obstacles, deciding on a driving direction based on recognized obstacles, and controlling the vehicle based on the decided driving direction.…”

Section: Introductionmentioning

confidence: 99%

“…Various deep learning techniques have been applied to learning for end-to-end control. Vehicles operate autonomously by learning from images of the road entered into convolutional neural networks (CNNs) [12][13][14]. Studies have been conducted on autonomous driving using deep neural networks (DNNs) [15] and on predicting the driving distance of vehicles using recurrent convolutional neural networks (RCNNs) [16].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

et al. 2018

View full text Add to dashboard Cite

Recent research on deep learning has been applied to a diversity of fields. In particular, numerous studies have been conducted on self-driving vehicles using end-to-end approaches based on images captured by a single camera. End-to-end controls learn the output vectors of output devices directly from the input vectors of available input devices. In other words, an end-to-end approach learns not by analyzing the meaning of input vectors, but by extracting optimal output vectors based on input vectors. Generally, when end-to-end control is applied to self-driving vehicles, the steering wheel and pedals are controlled autonomously by learning from the images captured by a camera. However, high-resolution images captured from a car cannot be directly used as inputs to Convolutional Neural Networks (CNNs) owing to memory limitations; the image size needs to be efficiently reduced. Therefore, it is necessary to extract features from captured images automatically and to generate input images by merging the parts of the images that contain the extracted features. This paper proposes a learning method for end-to-end control that generates input images for CNNs by extracting road parts from input images, identifying the edges of the extracted road parts, and merging the parts of the images that contain the detected edges. In addition, a CNN model for end-to-end control is introduced. Experiments involving the Open Racing Car Simulator (TORCS), a sustainable computing environment for cars, confirmed the effectiveness of the proposed method for self-driving by comparing the accumulated difference in the angle of the steering wheel in the images generated by it with those of resized images containing the entire captured area and cropped images containing only a part of the captured area. The results showed that the proposed method reduced the accumulated difference by 0.839% and 0.850% compared to those yielded by the resized images and cropped images, respectively.

show abstract

Section: Research On Driving Methods Of Carsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Afterwards, the confusion matrix elements were averaged across all frames, which gave mean values of 10,612, 23,904, 4,890, and 3,794 true positive, true negative, false positive, and false negative event counts, respectively. Similar metrics could be computed with different DVS models 2,7 in this scenario to compare their fidelity.…”

Section: Dvs Modelmentioning

confidence: 99%

“…Additional parameters can also be controlled through biasing circuitry, such as the time scale of DVS events. While this model is straightforward to understand and has been previously demonstrated 2 , it is not ideal, due to nonlinearities in actual DVS behavior and practicalities in acquiring conventional video. The performance of this model will be explored in Section 2.…”

Section: Introductionmentioning

confidence: 99%

Tracking from a moving platform with the Dynamic Vision Sensor

Cox

Morley

2019

Computational Imaging IV

View full text Add to dashboard Cite

The Dynamic Vision Sensor (DVS) is an imaging sensor that processes the incident irradiance image and outputs temporal log irradiance changes in the image, such as those generated by moving target(s) and/or the moving sensor platform. From a static platform, this enables the DVS to cancel out background clutter and greatly decrease the sensor bandwidth required to track temporal changes in a scene. However, the sensor bandwidth advantage is lost when imaging a scene from a moving platform due to platform motion causing optical flow in the background. Imaging from a moving platform has been utilized in many recently reported applications of this sensor. However, this approach inherently outputs background clutter generated from optical flow, and as such this approach has limited spatio-temporal resolution and is of limited utility for target tracking applications. In this work we present a new approach to moving target tracking applications with the DVS. Essentially, we propose modifying the incident image to cancel out optical flow due to platform motion, thereby removing background clutter and recovering the bandwidth performance advantage of the DVS. We propose that such improved performance can be accomplished by integrating a hardware tracking and stabilization subsystem with the DVS. Representative simulation scenarios are used to quantify the performance of the proposed approach to clutter cancellation and improved sensor bandwidth.

show abstract

ROSS-LAN: RObotic Sensing Simulation Scheme for Bioinspired Robotic Bird LANding

Eguíluz

Dios

Ollero

2019

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Aerial robotics is evolving towards the design of bioinspired platforms able to mimic the behavior of birds and insects during flight. The design offlapping-wing platforms involves research from different scientific areas such as aerodynamics, mechanics, electronics and computer science. The development of perception algorithms for autonomous navigation of ornithopters requires sensor data information to study, understand and solve the limitations present during the flight of these type of robots. However, the payload constraints and hardware complexity of these type of aerial platforms hamper sensor data acquisition. This paper focuses on the development of a multi-sensor simulator able to retrieve the sensor information captured during the landing of birds. The landing trajectory is computed by using a bioinspired trajectory generator relying on tau theory. Another contribution of this paper is the online 1 publication of a dataset with the sensor measurements obtained from several trajectories simulated in different scenarios.

show abstract

Towards a framework for end-to-end control of a simulated vehicle with spiking neural networks

Cited by 50 publications

References 25 publications

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing

Tracking from a moving platform with the Dynamic Vision Sensor

ROSS-LAN: RObotic Sensing Simulation Scheme for Bioinspired Robotic Bird LANding

Contact Info

Product

Resources

About