DeepPicar: A Low-Cost Deep Neural Network-Based Autonomous Car

Abstract

We present DeepPicar, a low-cost deep neural network based autonomous car platform. DeepPicar is a small scale replication of a real self-driving car called DAVE-2 by NVIDIA. DAVE-2 uses a deep convolutional neural network (CNN), which takes images from a front-facing camera as input and produces car steering angles as output. DeepPicar uses the same network architecture-9 layers, 27 million connections and 250K parameters-and can drive itself in real-time using a web camera and a Raspberry Pi 3 quad-core platform. Using DeepPicar, we analyze the Pi 3's computing capabilities to support end-to-end deep learning based real-time control of autonomous vehicles. We also systematically compare other contemporary embedded computing platforms using the DeepPicar's CNN-based real-time control workload.We find that all tested platforms, including the Pi 3, are capable of supporting the CNN-based real-time control, from 20 Hz up to 100 Hz, depending on hardware platform. However, we find that shared resource contention remains an important issue that must be considered in applying CNN models on shared memory based embedded computing platforms; we observe up to 11.6X execution time increase in the CNN based control loop due to shared resource contention. To protect the CNN workload, we also evaluate state-of-the-art cache partitioning and memory bandwidth throttling techniques on the Pi 3. We find that cache partitioning is ineffective, while memory bandwidth throttling is an effective solution.

Denial-of-Service Attacks on Shared Cache in Multicore: Analysis and Prevention

“…Specifically, it uses the LLC miss counter to calculate the amount of memory bandwidth consumed by each core. Prior studies show the effectiveness of memory bandwidth throttling in protecting real-time tasks [3], [7], [31], [32].…”

A survey of deep learning techniques for autonomous driving

“…End2End control papers mainly employ either deep neural networks trained offline on real‐world and/or synthetic data (Bechtel et al, ; Bojarski et al, ; C. Chen, Seff, Kornhauser, & Xiao, ; Eraqi et al, ; Fridman et al, ; Hecker et al, ; Rausch et al, ; Xu et al, ; S. Yang et al, ), or DRL systems trained and evaluated in simulation (Jaritz et al, ; Perot, Jaritz, Toromanoff, & Charette, ; Sallab et al, 2017b). Methods for porting simulation trained DRL models to real‐world driving have also been reported (Wayve, 2018), as well as DRL systems trained directly on real‐world image data (Pan et al, , ).…”

“…End2End architectures similar to PilotNet, which map visual data to steering commands, have been reported in Rausch et al (), Bechtel et al (), and C. Chen et al (). In Xu et al (), autonomous driving is formulated as a future ego‐motion prediction problem.…”

“…The backpropagation algorithm for gradient estimation in deep networks is now efficiently implemented on parallel graphic processing units (GPUs). This kind of processing allows the training of large and complex network architectures, which in turn require huge amounts of training samples (see Section 8).End2End control papers mainly employ either deep neural networks trained offline on real-world and/or synthetic data(Bechtel et al, 2018;Bojarski et al, 2016;C. Chen, Seff, Kornhauser, & Xiao, 2015;Eraqi et al, 2017;Fridman et al, 2017;Hecker et al, 2018;Rausch et al, 2017;Xu et al, 2017;S.…”

A Physical Testbed for Intelligent Transportation Systems

“…2, consists of a LaTrax Desert Prerunner 1/18-scale radio-controlled (RC) car chassis, an electronic speed controller (ESC), a Raspberry Pi 3 Model B+, and a Raspberry Pi Camera Module v2. It is an adaptation of the DeepPicar [14] but with a focus on integration into a vehicular network rather than evaluating the viability of embedded systems in autonomous vehicles. In addition, our In the autonomous control mode, the control flow consists of three main steps: front-view image streaming to a remote convolutional neural network (CNN) service, CNNbased image processing, and generated turn-angle and speed value streaming to the vehicle.…”