Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

Cong, Fengyu; Wang, Bin; Zhuang, Yuzheng; Zhang, Zongzhang; Hao, Jianye; Ji, Xuewu; Liu, Wulong

doi:10.24963/ijcai.2020/405

Cited by 19 publications

(12 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proximal policy optimization was applied to control autonomous driving and subsequently to actual vehicles [31]. The other studies [32][33][34] introduced important research aspects pertaining to DRL in autonomous driving.…”

Section: Related Workmentioning

confidence: 99%

A Deep Coordination Graph Convolution Reinforcement Learning for Multi-Intelligent Vehicle Driving Policy

Tan

Zuo

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the growing up of Internet of Things technology, the application of Internet of Things has been popularized in the field of intelligent vehicles. Therefore, more artificial intelligence algorithms, especially DRL methods, are more widely used in autonomous driving. A large number of deep reinforcement learning (RL) technologies are continuously applied to the behavior planning module of single-vehicle autonomous driving in early. However, autonomous driving is an environment where multi-intelligent vehicles coexist, interact with each other, and dynamically change. In this environment, multiagent RL technology is one of the most promising technologies for solving the coordination behavior planning problem of multivehicles. However, the research related to this topic is rare. This paper introduces a dynamic coordination graph (CG) convolution technology for the cooperative learning of multi-intelligent vehicles. This method dynamically constructs a CG model among multiple vehicles, effectively reducing the impact of unrelated intelligent vehicles and simplifying the learning process. The relationship between intelligent vehicles is refined using the attention mechanism, and the graph convolution RL technology is used to simulate the message-passing aggregation algorithm to maximize the local utility and obtain the maximum joint utility to guide coordination learning. Driving samples are used as training data, and the model guided by reward shaping is combined with the model of the free graph convolution RL method, which enables our proposed method to achieve high gradualness and improve its learning efficiency. In addition, as the graph convolutional RL algorithm shares parameters between agents, it can easily build scales that are suitable for large-scale multiagent systems, such as traffic environments. Finally, the proposed algorithm is tested and verified for the multivehicle cooperative lane-changing problem in the simulation environment of autonomous driving. Experimental results show that our proposed method has better value function representation in that it can learn better coordination driving policies than traditional dynamic coordination algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

A Deep Coordination Graph Convolution Reinforcement Learning for Multi-Intelligent Vehicle Driving Policy

Tan

Zuo

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Deep IRL GAIL GAIL [81] & extensions ALVINN [10] PilotNet [63], DAgger [68], SafeDAgger [69] Maximum Margin IRL [73,74] Maximum Entropy IRL [75] Deep MaxEnt IRL [79], GCL [80] InfoGAIL [202], CGAIL [205], Triple-GAIL [207], AIRL [84]…”

Section: Traditional Irlmentioning

confidence: 99%

“…The trained models rely on manually specified labels to output actions; hence, they cannot select modes adaptively according to environmental scenarios. Recently, Fei et al [207] proposed Triple-GAIL, which can learn adaptive skill selection and imitation jointly from expert demonstrations, and generated experiences by introducing an auxiliary skill selector.…”

Section: Uncertainty-aware Drl/dil For Admentioning

confidence: 99%

A Survey of Deep RL and IL for Autonomous Driving Policy Learning

Zhu,

Zhao

2021

Preprint

View full text Add to dashboard Cite

Autonomous driving (AD) agents generate driving policies based on online perception results, which are obtained at multiple levels of abstraction, e.g., behavior planning, motion planning and control. Driving policies are crucial to the realization of safe, efficient and harmonious driving behaviors, where AD agents still face substantial challenges in complex scenarios. Due to their successful application in fields such as robotics and video games, the use of deep reinforcement learning (DRL) and deep imitation learning (DIL) techniques to derive AD policies have witnessed vast research efforts in recent years. This paper is a comprehensive survey of this body of work, which is conducted at three levels: First, a taxonomy of the literature studies is constructed from the system perspective, among which five modes of integration of DRL/DIL models into an AD architecture are identified. Second, the formulations of DRL/DIL models for conducting specified AD tasks are comprehensively reviewed, where various designs on the model state and action spaces and the reinforcement learning rewards are covered. Finally, an in-depth review is conducted on how the critical issues of AD applications regarding driving safety, interaction with other traffic participants and uncertainty of the environment are addressed by the DRL/DIL models. To the best of our knowledge, this is the first survey to focus on AD policy learning using DRL/DIL, which is addressed simultaneously from the system, task-driven and problem-driven perspectives. We share and discuss findings, which may lead to the investigation of various topics in the future.

show abstract

“…Such a two-level decision-making process is usually formu- lated by an Option model (Sutton et al, 1999) or goal-based framework (Le et al, 2018). Although some works (Fei et al, 2020) have assumed the help of sub-task segmentation annotations, this paper mainly focuses on learning from unsegmented demonstrations to allow more practicability.…”

Section: Introductionmentioning

confidence: 99%

Adversarial Option-Aware Hierarchical Imitation Learning

Jing¹,

Huang²,

Sun³

et al. 2021

Preprint

View full text Add to dashboard Cite

It has been a challenge to learning skills for an agent from long-horizon unannotated demonstrations. Existing approaches like Hierarchical Imitation Learning(HIL) are prone to compounding errors or suboptimal solutions. In this paper, we propose Option-GAIL, a novel method to learn skills at long horizon. The key idea of Option-GAIL is modeling the task hierarchy by options and train the policy via generative adversarial optimization. In particular, we propose an Expectation-Maximization(EM)-style algorithm: an E-step that samples the options of expert conditioned on the current learned policy, and an M-step that updates the low-and high-level policies of agent simultaneously to minimize the newly proposed option-occupancy measurement between the expert and the agent. We theoretically prove the convergence of the proposed algorithm. Experiments show that Option-GAIL outperforms other counterparts consistently across a variety of tasks.

show abstract

Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets

Cited by 19 publications

References 2 publications

A Deep Coordination Graph Convolution Reinforcement Learning for Multi-Intelligent Vehicle Driving Policy

A Deep Coordination Graph Convolution Reinforcement Learning for Multi-Intelligent Vehicle Driving Policy

A Survey of Deep RL and IL for Autonomous Driving Policy Learning

Adversarial Option-Aware Hierarchical Imitation Learning

Contact Info

Product

Resources

About