SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments

Arec, Jamgochian,; Etienne, Buehrle,; Fischer, Johannes; Kochenderfer, Mykel J.

doi:10.1109/icra48891.2023.10161449

Cited by 8 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This modeling insight has been formalized using the options framework (Sutton, Precup, and Singh 1999;Daniel et al 2016) and offers an effective strategy to learning tasks by breaking them down into manageable sub-tasks, learning to accomplish them individually, and subsequently combining them to achieve the overarching task objective. While initially proposed for RL, this hierarchical approach has also been utilized for IL (Ranchod, Rosman, and Konidaris 2015; Le et al 2018;Unhelkar and Shah 2019;Gupta et al 2019;Jing et al 2021;Orlov-Savko et al 2022;Jamgochian et al 2023;Nasiriany et al 2023;Gao, Jiang, and Chen 2023;Chen, Lan, and Aggarwal 2023). Unsupervised methodologies, such as InfoGAIL and Directed InfoGAIL, leverage information-theoretic measures to initially discover latent options and subsequently imitate the expert (Li, Song, and Ermon 2017;Sharma et al 2018).…”

Section: Il Via Stationary Distribution Correction Estimationmentioning

confidence: 99%

GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation

Jain,

Unhelkar

2024

AAAI

View full text Add to dashboard Cite

Offline imitation learning (IL) refers to learning expert behavior solely from demonstrations, without any additional interaction with the environment. Despite significant advances in offline IL, existing techniques find it challenging to learn policies for long-horizon tasks and require significant re-training when task specifications change. Towards addressing these limitations, we present GO-DICE an offline IL technique for goal-conditioned long-horizon sequential tasks. GO-DICE discerns a hierarchy of sub-tasks from demonstrations and uses these to learn separate policies for sub-task transitions and action execution, respectively; this hierarchical policy learning facilitates long-horizon reasoning.Inspired by the expansive DICE-family of techniques, policy learning at both the levels transpires within the space of stationary distributions. Further, both policies are learnt with goal conditioning to minimize need for retraining when task goals change. Experimental results substantiate that GO-DICE outperforms recent baselines, as evidenced by a marked improvement in the completion rate of increasingly challenging pick-and-place Mujoco robotic tasks. GO-DICE is also capable of leveraging imperfect demonstration and partial task segmentation when available, both of which boost task performance relative to learning from expert demonstrations alone.

show abstract

Section: Il Via Stationary Distribution Correction Estimationmentioning

confidence: 99%

GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation

Jain,

Unhelkar

2024

AAAI

View full text Add to dashboard Cite

show abstract

A Survey of Artificial Intelligence Approaches to Safety and Mission-Critical Systems

Thames,

Sun

2024

2024 Integrated Communications, Navigation and Surveillance Conference (ICNS)

View full text Add to dashboard Cite

Guided Hierarchical Reinforcement Learning for Safe Urban Driving

Albilani,

Bouzeghoub

2023

2023 IEEE 35th International Conference on Tools With Artificial Intelligence (ICTAI)

View full text Add to dashboard Cite

Designing a safe decision-making system for end-toend urban driving is still challenging. Numerous contributions based on Deep Reinforcement Learning (DRL) were developed. However, they all suffer from the cold start issue and require extensive convergence training. Recent solutions for urban driving have emerged based on both Hierarchical Reinforcement Learning (HRL) and imitation learning to overcome these limitations. Nevertheless, they do not guarantee a safe exploration for an autonomous vehicle. In the literature, rule-based systems played a pivotal role in ensuring the safety of self-driving cars, but they require manual rule encoding. This paper introduces GHRL, a decision-making framework for vision-based urban driving that benefits from HRL, and a rule-based system for safe urban driving. The HRL algorithm learns the high-level policies, whereas the low-level policies are guided by the expert demonstration rules modeled with the Answer Set Programming (ASP) formalism. When a critical situation occurs, the system will shift to rely on ASP rules. The state of each policy includes visual features extracted by a convolutional neural network from a monocular camera, information on localization, and waypoints. GHRL is evaluated on the Carla NoCrash benchmark. The results show that by incorporating logical rules, GHRL achieved better performance over state-of-the-art HRL algorithms.

show abstract

SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments

Cited by 8 publications

References 14 publications

GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation

GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation

A Survey of Artificial Intelligence Approaches to Safety and Mission-Critical Systems

Guided Hierarchical Reinforcement Learning for Safe Urban Driving

Contact Info

Product

Resources

About