Nowadays, Wireless Sensor Networks (WSNs) are playing a vital and sustainable role in many verticals touching different aspects of our lives including civil, public, and military applications. WSNs majorly consist of a few to several sensor nodes, that are connected to each other via wireless communication links and require real-time or delayed data transfer. In this paper, we propose an autonomous Unmanned Aerial Vehicle (UAV)-enabled data gathering mechanism for delay-tolerant WSN applications. The objective is to employ a self-trained UAV as a flying mobile unit collecting data from ground sensor nodes spatially distributed in a given geographical area during a predefined period of time. In this approach, two Reinforcement Learning (RL) approaches, specifically Deep Deterministic Gradient Decent (DDPG) and Q-learning (QL) algorithms, are jointly employed to train the UAV to understand the environment and provide effective scheduling to accomplish its data collection mission. The DDPG is used to autonomously decide the best trajectory to adopt in an obstacle-constrained environment, while the QL is developed to determine the order of nodes to visit such that the data collection time is minimized. The schedule is obtained while considering the limited battery capacity of the flying unit, its need to return the charging station, the time windows of data acquisition, and the priority of certain sensor nodes. Customized reward functions are designed for each RL model and, through numerical simulations, we investigate their training performances. We also analyze the behavior of the autonomous UAV for different selected scenarios and corroborate the ability of the proposed approach in performing effective data collection. A comparison with the deterministic optimal solution is provided to validate the performance of the learning-based approach. INDEX TERMS Internet-of-things, data gathering, reinforcement learning, scheduling, unmanned areal vehicles
Considerable attention has been given to leverage a variety of smart city applications using unmanned aerial vehicles (UAVs). The rapid advances in artificial intelligence can empower UAVs with autonomous capabilities allowing them to learn from their surrounding environment and act accordingly without human intervention. In this paper, we propose a spatiotemporal scheduling framework for autonomous UAVs using reinforcement learning. The framework enables UAVs to autonomously determine their schedules to cover the maximum of pre-scheduled events spatially and temporally distributed in a given geographical area and over a predetermined time horizon. The designed framework has the ability to update the planned schedules in case of unexpected emergency events. The UAVs are trained using the Q-learning (QL) algorithm to find effective scheduling plan. A customized reward function is developed to consider several constraints especially the limited battery capacity of the flying units, the time windows of events, and the delays caused by the UAV navigation between events. Numerical simulations show the behavior of the autonomous UAVs for various scenarios and corroborate the ability of QL to handle complex vehicle routing problems with several constraints. A comparison with an optimal deterministic solution is also provided to validate the performance of the learning-based solution.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.