Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

Bojarski, Mariusz; Yeres, Philip; Choromanska, Anna; Choromański, Krzysztof; Firner, Bernhard; Jackel, L. D.; Müller, Urs

doi:10.48550/arxiv.1704.07911

Cited by 87 publications

(122 citation statements)

References 0 publications

Supporting

Mentioning

121

Contrasting

Order By: Relevance

“…Since ResNet-110 is a small DNN, distributing the computation across more chiplets results in a sub-optimal configuration. A similar trend is shown in SIMBA for a small DNN, DriveNet [5].…”

Section: Calibration With Simbasupporting

confidence: 77%

SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks

Krishnan,

Mandal,

Pannala

et al. 2021

Preprint

View full text Add to dashboard Cite

In-memory computing (IMC) on a monolithic chip for deep learning faces dramatic challenges on area, yield, and on-chip interconnection cost due to the ever-increasing model sizes. 2.5D integration or chiplet-based architectures interconnect multiple small chips (i.e., chiplets) to form a large computing system, presenting a feasible solution beyond a monolithic IMC architecture to accelerate large deep learning models. This paper presents a new benchmarking simulator, SIAM, to evaluate the performance of chiplet-based IMC architectures and explore the potential of such a paradigm shift in IMC architecture design. SIAM integrates device, circuit, architecture, network-on-chip (NoC), network-on-package (NoP), and DRAM access models to realize an end-to-end system. SIAM is scalable in its support of a wide range of deep neural networks (DNNs), customizable to various network structures and configurations, and capable of efficient design space exploration. We demonstrate the flexibility, scalability, and simulation speed of SIAM by benchmarking different state-of-the-art DNNs with CIFAR-10, CIFAR-100, and ImageNet datasets. We further calibrate the simulation results with a published silicon result, SIMBA. The chiplet-based IMC architecture obtained through SIAM shows 130× and 72× improvement in energy-efficiency for ResNet-50 on the ImageNet dataset compared to Nvidia V100 and T4 GPUs. CCS Concepts: • Hardware → Analysis and Design of Emerging Devices and Systems; On-chip Resource Management; Emerging Architectures; Interconnect; System-on-a-chip.

show abstract

“…Since ResNet-110 is a small DNN, distributing the computation across more chiplets results in a sub-optimal configuration. A similar trend is shown in SIMBA for a small DNN, DriveNet [5].…”

Section: Calibration With Simbasupporting

confidence: 77%

SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks

Krishnan,

Mandal,

Pannala

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…ALVINN [24] was the first imitation learning application of autonomous driving, predicting the steering angle from data from active and passive sensors. Deep learning advances have reignited interest in conditional imitation learning for autonomous driving [51]. ALVINN utilized a conditional order to display an E2E network for lanes following vacant highways, which monitored the steering angle from a single camera [52].…”

Section: Related Workmentioning

confidence: 99%

Navigating an Automated Driving Vehicle via the Early Fusion of Multi-Modality

Haris

Głowacz

2022

Sensors

View full text Add to dashboard Cite

The ability of artificial intelligence to drive toward an intended destination is a key component of an autonomous vehicle. Different paradigms are now being employed to address artificial intelligence advancement. On the one hand, modular pipelines break down the driving model into submodels, such as perception, maneuver planning and control. On the other hand, we used the end-to-end driving method to assign raw sensor data directly to vehicle control signals. The latter is less well-studied but is becoming more popular since it is easier to use. This article focuses on end-to-end autonomous driving, using RGB pictures as the primary sensor input data. The autonomous vehicle is equipped with a camera and active sensors, such as LiDAR and Radar, for safe navigation. Active sensors (e.g., LiDAR) provide more accurate depth information than passive sensors. As a result, this paper examines whether combining the RGB from the camera and active depth information from LiDAR has better results in end-to-end artificial driving than using only a single modality. This paper focuses on the early fusion of multi-modality and demonstrates how it outperforms a single modality using the CARLA simulator.

show abstract

“…Recently, ML models have started to be deployed into highimpact, real-world decision-making settings such as medicine [16], self-driving cars [12], and college admissions [40]. However, this has led to problems: many of these settings have key constraints that ML models were not originally designed to handle.…”

Section: Accuracymentioning

confidence: 99%

MPC-Friendly Commitments for Publicly Verifiable Covert Security

Agrawal¹,

Bell²,

Gascón³

et al. 2021

Preprint

View full text Add to dashboard Cite

We address the problem of efficiently verifying a commitment in a two-party computation. This addresses the scenario where a party P1 commits to a value 𝑥 to be used in a subsequent secure computation with another party P2 that wants to receive assurance that P1 did not cheat, i.e. that 𝑥 was indeed the value inputted into the secure computation. Our constructions operate in the publicly verifiable covert (PVC) security model, which is a relaxation of the malicious model of MPC appropriate in settings where P1 faces a reputational harm if caught cheating.We introduce the notion of PVC commitment scheme and indexed hash functions to build commitments schemes tailored to the PVC framework, and propose constructions for both arithmetic and Boolean circuits that result in very efficient circuits. From a practical standpoint, our constructions for Boolean circuits are 60× faster to evaluate securely, and use 36× less communication than baseline methods based on hashing. Moreover, we show that our constructions are tight in terms of required non-linear operations, by proving lower bounds on the nonlinear gate count of commitment verification circuits. Finally, we present a technique to amplify the security properties our constructions that allows to efficiently recover malicious guarantees with statistical security.

show abstract

Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

Cited by 87 publications

References 0 publications

SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks

SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks

Navigating an Automated Driving Vehicle via the Early Fusion of Multi-Modality

MPC-Friendly Commitments for Publicly Verifiable Covert Security

Contact Info

Product

Resources

About