Brain-Inspired Cognitive Model With Attention for Self-Driving Cars

Chen, Shitao; Zhang, Songyi; Shang, Jinghao; Chen, Badong; Zheng, Nanning

doi:10.1109/tcds.2017.2717451

Cited by 100 publications

(41 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…But neural networks are notoriously cryptic -both network architecture and hidden layer activations may have no obvious relation to the function being estimated by the network. An exception to the rule is visual attention networks [26,21,7]. These networks provide spatial attention maps -areas of the image that the network attends to -that can be displayed in a way that is easy for users to interpret.…”

Section: Introductionmentioning

confidence: 99%

Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention

Kim

Canny

2017

2017 IEEE International Conference on Computer Vision (ICCV)

284

246

View full text Add to dashboard Cite

Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable -they should provide easy-tointerpret rationales for their behavior -so that passengers, insurance companies, law enforcement, developers etc., can understand what triggered a particular behavior. Here we explore the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network's output (steering control). Our approach is two-stage. In the first stage, we use a visual attention model to train a convolution network endto-end from images to steering angle. The attention model highlights image regions that potentially influence the network's output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network's behavior. We demonstrate the effectiveness of our model on three datasets totaling 16 hours of driving. We first show that training with attention does not degrade the performance of the end-to-end network. Then we show that the network causally cues on a variety of features that are used by humans while driving.

show abstract

Section: Introductionmentioning

confidence: 99%

Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention

Kim

Canny

2017

2017 IEEE International Conference on Computer Vision (ICCV)

284

246

View full text Add to dashboard Cite

show abstract

“…Autonomous driving has seen dramatic advances in recent years, for instance for road scene parsing [23,67,79,24], lane following [46,37,17], path planning [12,18,62,63], and end-to-end driving models [77,22,21,56]. By now, autonomous vehicles have driven many thousands of miles and companies aspire to sell such vehicles in a few years.…”

Section: Introductionmentioning

confidence: 99%

End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners

Hecker

Dai

Gool

2018

Lecture Notes in Computer Science

166

182

View full text Add to dashboard Cite

For human drivers, having rear and side-view mirrors is vital for safe driving. They deliver a more complete view of what is happening around the car. Human drivers also heavily exploit their mental map for navigation. Nonetheless, several methods have been published that learn driving models with only a front-facing camera and without a route planner. This lack of information renders the self-driving task quite intractable. We investigate the problem in a more realistic setting, which consists of a surround-view camera system with eight cameras, a route planner, and a CAN bus reader. In particular, we develop a sensor setup that provides data for a 360-degree view of the area surrounding the vehicle, the driving route to the destination, and low-level driving maneuvers (e.g. steering angle and speed) by human drivers. With such a sensor setup we collect a new driving dataset, covering diverse driving scenarios and varying weather/illumination conditions. Finally, we learn a novel driving model by integrating information from the surround-view cameras and the route planner. Two route planners are exploited: 1) by representing the planned routes on Open-StreetMap as a stack of GPS coordinates, and 2) by rendering the planned routes on TomTom Go Mobile and recording the progression into a video. Our experiments show that: 1) 360-degree surround-view cameras help avoid failures made with a single front-view camera, in particular for city driving and intersection scenarios; and 2) route planners help the driving task significantly, especially for steering angle prediction. Code, data and more visual results will be made available at http://www.vision.ee.ethz.ch/˜heckers/Drive360. Keywords: Autonomous driving · end-to-end learning of driving · route planning for driving · surround-view cameras · driving dataset

show abstract

“…Simulation tools can be used to both generate training data on specific conditions or to train end-to-end driving systems [32], [33]. Using virtual data during training can enhance the performance of detection models on real environments.…”

Section: Datasetsmentioning

confidence: 99%