Learning by Watching

Zhang, Jimuyang; Ohn-Bar, Eshed

doi:10.1109/cvpr46437.2021.01252

Cited by 23 publications

(7 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our idea of training the ego motion planner using data from all vehicles is closely related to Filos et al [18] and Zhang and Ohn-Bar [49]. Filos et al [18] extends offline reinforcement learning to learn from other agents' behaviors.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Learning from All Vehicles

Chen¹

2022

Preprint

View full text Add to dashboard Cite

In this paper, we present a system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes. This system uses the behaviors of other agents to create more diverse driving scenarios without collecting additional data. The main difficulty in learning from other vehicles is that there is no sensor information. We use a set of supervisory tasks to learn an intermediate representation that is invariant to the viewpoint of the controlling vehicle. This not only provides a richer signal at training time but also allows more complex reasoning during inference. Learning how all vehicles drive helps predict their behavior at test time and can avoid collisions. We evaluate this system in closed-loop driving simulations. Our system outperforms all prior methods on the public CARLA Leaderboard by a wide margin, improving driving score by 25 and route completion rate by 24 points.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Filos et al [18] extends offline reinforcement learning to learn from other agents' behaviors. Zhang and Ohn-Bar [49] train a privileged imitation learning policy that learns from other vehicles in a scene. Their policy…”

Section: Related Workmentioning

confidence: 99%

Learning from All Vehicles

Chen¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Observational Imitation Learning: Our key idea is to leverage the scale and diversity of easily available online ego-centric navigation data to learn a robust conditional imitation learning policy [13,17]. While learning from labeled demonstrations can significantly simplify the challenging vision-based policy learning task [3,6,11,12,31,32,41,44,49,50,54,58,60,80,83,84], observed images in our settings are not labeled with corresponding actions of a demonstrator. We therefore work to generalize current conditional imitation learning (CIL) approaches [13,17,18] to learn, from unlabeled image observations, an agent that can navigate in complex urban scenarios.…”

Section: Related Workmentioning

confidence: 99%

SelfD: Self-Learning Large-Scale Driving Policies From the Web

Zhang¹,

Zhu²,

Ohn-Bar³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…While there is a large body of literature on end-to-end RL on perception data for urban driving (Codevilla et al, 2018;Ohn-Bar et al, 2020;Codevilla et al, 2019;Chen et al, 2020;Zhang and Ohn-Bar, 2021;Prakash et al, 2021;Zhang and Ohn-Bar, 2021), there are significantly fewer works on the same topic for high-speed racing. We hypothesize this may partly be attributed to the lack of open-source, high-fidelity simulation environments for racing, in comparison to the ubiquity of CARLA simulator Dosovitskiy et al (2017a) for urban driving research.…”

Section: Related Workmentioning

confidence: 99%

Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing

Francis¹,

Chen²,

Ganju³

et al. 2022

Preprint

View full text Add to dashboard Cite

We present the results of our autonomous racing virtual challenge, based on the newly-released Learn-to-Race (L2R) simulation framework, which seeks to encourage interdisciplinary research in autonomous driving and to help advance the state of the art on a realistic benchmark. Analogous to racing being used to test cutting-edge vehicles, we envision autonomous racing to serve as a particularly challenging proving ground for autonomous agents as: (i) they need to make sub-second, safety-critical decisions in a complex, fast-changing environment; and (ii) both perception and control must be robust to distribution shifts, novel road features, and unseen obstacles. Thus, the main goal of the challenge is to evaluate the joint safety, performance, and generalisation capabilities of reinforcement learning agents on multi-modal perception, through a two-stage process. In the first stage of the challenge, we evaluate an autonomous agent's ability to drive as fast as possible, while adhering to safety constraints. In the second stage, we additionally require the agent to adapt to an unseen racetrack through safe exploration. In this paper, we describe the new L2R Task 2.0 benchmark, with refined metrics and baseline approaches. We also provide an overview of deployment, evaluation, and rankings for the inaugural instance of the L2R Autonomous Racing Virtual Challenge (supported by Carnegie Mellon University, Arrival Ltd., AICrowd, Amazon Web Services, and Honda Research), which officially used the new L2R Task 2.0 benchmark and received over 20,100 views, 437 active participants, 46 teams, and 733 model submissions-from 88+ unique institutions, in 58+ different countries. Finally, we release leaderboard results from the challenge and provide description of the two top-ranking approaches in cross-domain model transfer, across multiple sensor configurations and simulated races.

show abstract

Learning by Watching

Cited by 23 publications

References 41 publications

Learning from All Vehicles

Learning from All Vehicles

SelfD: Self-Learning Large-Scale Driving Policies From the Web

Learn-to-Race Challenge 2022: Benchmarking Safe Learning and Cross-domain Generalisation in Autonomous Racing

Contact Info

Product

Resources

About