Keeping in Touch with Collaborative UAVs: A Deep Reinforcement Learning Approach

Yang, Bo; Liu, Min

doi:10.24963/ijcai.2018/78

Cited by 17 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pham et al [116] proposed a MARL algorithm for a team of UAVs, where the suggested MARL algorithm utilizes UAV agents to learn concurrently to supply full coverage of an unknown field of concern, thus reducing the overlapping sections among their field of views. Yang and Liu [117] studied the UAVs' optimal link discovery and selection problem using a multiagent deep RL (MADRL) framework with a fractional slicing technique to reduce computational complexity. Their studies showed that a MADRL-based approach can be useful for the task of ensuring continuous network connection of multiple UAVs.…”

Section: B Microagent Behavioral Interventionmentioning

confidence: 99%

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Zhang

Valencia

Chang

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Agent-based modeling (ABM) involves developing models in which agents make adaptive decisions in a changing environment. Machine-learning (ML) based inference models can improve sequential decision-making by learning agents' behavioral patterns. With the aid of ML, this emerging area can extend traditional agent-based schemes that hardcode agents' behavioral rules into an adaptive model. Even though there are plenty of studies that apply ML in ABMs, the generalized applicable scenarios, frameworks, and procedures for implementations are not well addressed. In this article, we provide a comprehensive review of applying ML in ABM based on four major scenarios, i.e., microagent-level situational awareness learning, microagent-level behavior intervention, macro-ABM-level emulator, and sequential decision-making. For these four scenarios, the related algorithms, frameworks, procedures of implementations, and multidisciplinary applications are thoroughly investigated. We also discuss how ML can improve prediction in ABMs by trading off the variance and bias and how ML can improve the sequential decision-making of microagent and macrolevel policymakers via a mechanism of reinforced behavioral intervention. At the end of this article, future perspectives of applying ML in ABMs are discussed with respect to data acquisition and quality issues, the possible solution of solving the convergence problem of reinforcement learning, interpretable ML applications, and bounded rationality of ABM.

show abstract

Section: B Microagent Behavioral Interventionmentioning

confidence: 99%

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Zhang

Valencia

Chang

2023

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…Examples of the scenarios include robot team navigation (Corke et al, 2005), smart grid operation (Dall'Anese et al, 2013), and control of mobile sensor networks (Cortes et al, 2004). Here we choose unmanned aerial vehicles (UAVs) (Yang and Liu, 2018;Pham et al, 2018;Tožička et al, 2018;Shamsoshoara et al, 2019;Cui et al, 2019;Qie et al, 2019), a recently surging application scenario of multi-agent autonomous systems, as one representative example. Specifically, a team of UAVs are deployed to accomplish a cooperation task, usually without the coordination of any central controller, i.e., in a decentralized fashion.…”

Section: Cooperative Setting Unmanned Aerial Vehiclesmentioning

confidence: 99%

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Zhang¹,

Yang²,

Başar³

2019

Preprint

154

View full text Add to dashboard Cite

Recent years have witnessed significant advances in reinforcement learning (RL), which has registered great success in solving various sequential decision-making problems in machine learning. Most of the successful RL applications, e.g., the games of Go and Poker, robotics, and autonomous driving, involve the participation of more than one single agent, which naturally fall into the realm of multi-agent RL (MARL), a domain with a relatively long history, and has recently re-emerged due to advances in single-agent RL techniques. Though empirically successful, theoretical foundations for MARL are relatively lacking in the literature. In this chapter, we provide a selective overview of MARL, with focus on algorithms backed by theoretical analysis. More specifically, we review the theoretical results of MARL algorithms mainly within two representative frameworks, Markov/stochastic games and extensive-form games, in accordance with the types of tasks they address, i.e., fully cooperative, fully competitive, and a mix of the two. We also introduce several significant but challenging applications of these algorithms. Orthogonal to the existing reviews on MARL, we highlight several new angles and taxonomies of MARL theory, including learning in extensive-form games, decentralized MARL with networked agents, MARL in the mean-field regime, (non-)convergence of policy-based methods for learning in games, etc. Some of the new angles extrapolate from our own research endeavors and interests. Our overall goal with this chapter is, beyond providing an assessment of the current state of the field on the mark, to identify fruitful future research directions on theoretical studies of MARL. We expect this chapter to serve as continuing stimulus for researchers interested in working on this exciting while challenging topic.Recent years have witnessed sensational advances of reinforcement learning (RL) in many prominent sequential decision-making problems, such as playing the game of Go (Silver

show abstract

“…In [8], the motion planning problem for a pick-up and place drone has been studied. The information exchange problem between different UAVs and its effect on the performance of a multi-UAV system has been investigated in [9]. In [2], a personal single drone delivery system has been implemented to deliver packages to destinations within 5km of the pickup location.…”

Section: Related Work and Backgroundmentioning

confidence: 99%

Dynamic Resource Management for Providing QoS in Drone Delivery Systems

Khamidehi¹,

Raeis²,

Sousa³

2021

Preprint

View full text Add to dashboard Cite

Drones have been considered as an alternative means of package delivery to reduce the delivery cost and time. Due to the battery limitations, the drones are best suited for lastmile delivery, i.e., the delivery from the package distribution centers (PDCs) to the customers. Since a typical delivery system consists of multiple PDCs, each having random and time-varying demands, the dynamic drone-to-PDC allocation would be of great importance in meeting the demand in an efficient manner. In this paper, we study the dynamic UAV assignment problem for a drone delivery system with the goal of providing measurable Quality of Service (QoS) guarantees. We adopt a queueing theoretic approach to model the customerservice nature of the problem. Furthermore, we take a deep reinforcement learning approach to obtain a dynamic policy for the re-allocation of the UAVs. This policy guarantees a probabilistic upper-bound on the queue length of the packages waiting in each PDC, which is beneficial from both the service provider's and the customers' viewpoints. We evaluate the performance of our proposed algorithm by considering three broad arrival classes, including Bernoulli, Time-Varying Bernoulli, and Markov-Modulated Bernoulli arrivals. Our results show that the proposed method outperforms the baselines, particularly in scenarios with Time-Varying and Markov-Modulated Bernoulli arrivals, which are more representative of real-world demand patterns. Moreover, our algorithm satisfies the QoS constraints in all the studied scenarios while minimizing the average number of UAVs in use.

show abstract

Keeping in Touch with Collaborative UAVs: A Deep Reinforcement Learning Approach

Cited by 17 publications

References 0 publications

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Synergistic Integration Between Machine Learning and Agent-Based Modeling: A Multidisciplinary Review

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Dynamic Resource Management for Providing QoS in Drone Delivery Systems

Contact Info

Product

Resources

About