IADRL: Imitation Augmented Deep Reinforcement Learning Enabled UGV-UAV Coalition for Tasking in Complex Environments

Abstract

Recent developments in Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) have made them highly useful for various tasks. However, they both have their respective constraints that make them incapable of completing intricate tasks alone in many scenarios. For example, a UGV is unable to reach high places, while a UAV is limited by its power supply and payload capacity. In this paper, we propose an Imitation Augmented Deep Reinforcement Learning (IADRL) model that enables a UGV and UAV to form a coalition that is complementary and cooperative for completing tasks that they are incapable of achieving alone. IADRL learns the underlying complementary behaviors of UGVs and UAVs from a demonstration dataset that is collected from some simple scenarios with non-optimized strategies. Based on observations from the UGV and UAV, IADRL provides an optimized policy for the UGV-UAV coalition to work in an complementary way while minimizing the cost. We evaluate the IADRL approach in an visual game-based simulation platform, and conduct experiments that show how it effectively enables the coalition to cooperatively and cost-effectively accomplish tasks. INDEX TERMS Unmanned aerial vehicle (UAV), unmanned ground vehicle (UGV), coalition, deep reinforcement learning (DRL), imitation learning. JIAN ZHANG (Member, IEEE) received the B.Sc. and M.Sc. degrees in applied physics from Sichuan University, Chengdu, China, in 2001 and 2008, respectively, and the Ph.D. degree in electrical and computer engineering from Auburn University, Auburn, AL, USA, in 2016. He is currently an Assistant Research Professor with the RFID Laboratory, Auburn University. His main research interests include RFID technologies and applications, the Internet of Things, indoor localization, UAV, and collaborative robotics. His work focuses on improving the efficiency of supply chain management for industry and business.

Motion Planning for Mobile Robots—Focusing on Deep Reinforcement Learning: A Systematic Review

“…As shown in FIGURE 24(a), this method can automatically learn a navigation strategy, imitate data by experts, and apply to multi-robot system. As shown in FIGURE 24(b), Zhang et al [123] proposed an imitation learning model (IADRL) to drive UAV to cooperate with each other to complete more complex tasks. The model provides a strategy for multi-UAV system, which makes multi-UAV complete the task with the minimum system cost.…”

Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning

“…An agent interfaces with all levels of control. This methodology constrains a trained agent to the specific vehicle it was trained on as it has learned not only how to avoid obstacles and adapt to a changing environment, but also a vehicle’s unique specific dynamics ( Cheng and Zhang, 2018 ; Codevilla et al, 2018 ; Zhang et al, 2020 ; Zhou et al, 2020 ). These implementations would likely struggle generalizing in any cross-vehicle or cross-domain implementation without re-training of the agent.…”

A RFID-Integrated Framework for Tag Anti-Collision in UAV-Aided VANETs

“…There have been some works about RFID-integrated UAV-aided networks [11][12][13][14][15][16][17][18][19][20][21][22][23]. Currently, Buffi et al in [11] presented a drone-based UHF-RFID tag identification scheme, which can be applied in indoor scenarios and VANET environments.…”

“…In addition, Won et al in [17] and Zhang et al in [18] introduced machine learning and deep reinforcement learning to RFID-integrated UAV-aided networks and investigated the tag estimation scheme. In addition, indoor positioning [19], resource allocation [20], tag localization [21], remote sensing [22], and electromagnetic modeling [23] have also been investigated.…”