Energy Management Strategy based on Deep Q-network in the Solar-powered UAV Communications System

Cong, Jiayi; Bin, Li; Guo, Xijing; Zhang, Ruonan

doi:10.1109/iccworkshops50388.2021.9473509

Cited by 8 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem was first modeled as a distributed multi-agent control problem, then a deep RL (DRL) framework was developed to control each UAV while minimizing the total energy consumption of the UAVs and ensuring minimum coverage requirements and geographical fairness are achieved. An energy management strategy for solar-powered UAV-BSs based on deep Q-networks was proposed in [224], wherein the total energy consumption of the UAV-BSs as well as the 3D flight trajectory were jointly optimized to enhance their communication capacity. The authors in [225] studied the problem of 3D trajectory design and frequency allocation while considering the energy consumption of the UAV-BSs and the fairness of user coverage.…”

Section: Machine Learning Approachesmentioning

confidence: 99%

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Abubakar

Ahmad

Omeke

et al. 2023

Drones

View full text Add to dashboard Cite

Wireless communication networks have been witnessing unprecedented demand due to the increasing number of connected devices and emerging bandwidth-hungry applications. Although there are many competent technologies for capacity enhancement purposes, such as millimeter wave communications and network densification, there is still room and need for further capacity enhancement in wireless communication networks, especially for the cases of unusual people gatherings, such as sport competitions, musical concerts, etc. Unmanned aerial vehicles (UAVs) have been identified as one of the promising options to enhance capacity due to their easy implementation, pop-up fashion operation, and cost-effective nature. The main idea is to deploy base stations on UAVs and operate them as flying base stations, thereby bringing additional capacity where it is needed. However, UAVs mostly have limited energy storage, hence, their energy consumption must be optimized to increase flight time. In this survey, we investigate different energy optimization techniques with a top-level classification in terms of the optimization algorithm employed—conventional and machine learning (ML). Such classification helps understand the state-of-the-art and the current trend in terms of methodology. In this regard, various optimization techniques are identified from the related literature, and they are presented under the above-mentioned classes of employed optimization methods. In addition, for the purpose of completeness, we include a brief tutorial on the optimization methods and power supply and charging mechanisms of UAVs. Moreover, novel concepts, such as reflective intelligent surfaces and landing spot optimization, are also covered to capture the latest trends in the literature.

show abstract

Section: Machine Learning Approachesmentioning

confidence: 99%

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Abubakar

Ahmad

Omeke

et al. 2023

Drones

View full text Add to dashboard Cite

show abstract

“…However, the work did not show the increase in flying time based on different heights and ground user count. Also, in [14] , the author used a deep Q network to optimize the UAVs' capacity to support the ground users. Although they used solar powered UAVs, they have not included a scheduling mechanism to support future users in the desired UAV range.…”

Section: Related Workmentioning

confidence: 99%

Journal of Smart Environments and Green Computing

Prakash¹,

Wang²,

Scott³

2022

J Smart Environ Green Comput

View full text Add to dashboard Cite

Aim: The rapid growth in the number of ground users over recent years has introduced the issues for a base station of providing more reliable connectivity and guaranteeing the reasonable quality of service (QoS). Thanks to the unique features of unmanned aerial vehicles (UAVs), such as flexibility in deployment, large coverage range and lower cost, UAVs can help the base station to provide wireless connectivity to the ground users, e.g., in rural and remote areas. As the energy limitation is the main concern for UAVs, the motivation is to provide uninterrupted connection to ground users in the next generation wireless networks using solar powered UAV-assisted air networks. Methods: The research uses global horizontal irradiance (GHI) data from the National Renewable Energy Laboratory, small cell power ratings for communication, and UAV parameters. In addition, the TensorFlow library and Python programming language were also used to develop machine learning models and simulate the UAV flying time. Results: In this paper, we develop a novel resource management system for UAVs, which consists of an energy harvesting deep learning model to predict the future power harvested from the solar panel and a consumption model which determines user arrival rate. With energy consumption and harvesting predictions, the resource management system adaptively switches the power consumed by a UAV for communication. In addition, based on the future energy availability and user's arrival rate, the resource management system communicates with other UAVs and enables energy coordinating scheduling among multiple UAVs to support user communications. The experiment results demonstrate that by using adaptive energy scheduling among UAVs, the flying time of the UAVs is improved by 40% during nighttime and by 37% when performing energy coordination among multiple UAVs. Conclusion: In this work, the UAV based communications have been researched. To understand more about UAVs and air segments, some literature review has been done based on previous works. Finally, alteration of the transmission power using several methodologies has been accomplished to increase the flying time of the UAV.

show abstract

“…The existing literature has studied a number of problems related to the energy management of UAVs for wireless communication systems, such as [ 5 , 6 , 7 , 8 , 9 , 10 ]. The authors in [ 5 ] derived a theoretical model on the propulsion energy consumption of UAVs, which first correlated the UAVs’ energy consumption with the varying flying speed, direction, and acceleration in UAV communications.…”

Section: Introductionmentioning

confidence: 99%

“…The works in [ 5 , 6 , 7 , 8 ] only consider the optimization of UAV energy consumption to save energy, but ignore the energy supplementary which can also extend the working time of a UAV to serve more users. The work in [ 9 ] studied the energy of solar-powered UAVs and considered the solar energy harvesting during the UAV deployment, which enhances the UAV communication capacity. The authors in [ 10 ] introduced ground solar panels to recharge UAVs and discussed the relationship between UAV battery level and UAV coverage.…”

Section: Introductionmentioning

confidence: 99%

“…With the ground solar panels as supplementary, the mission duration of UAVs can be extended. However, most of the existing works such as [ 5 , 6 , 7 , 8 , 9 , 10 ] solves the UAV energy management problems with optimization methods, which takes too much time for UAVs to obtain the optimal policies to execute in practical environments.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Biological Intelligence Inspired Trajectory Design for Energy Harvesting UAV Networks

Liu

Wang²,

Yin³

2023

Sensors

View full text Add to dashboard Cite

In this paper, the problem of trajectory design for energy harvesting unmanned aerial vehicles (UAVs) is studied. In the considered model, the UAV acts as a moving base station to serve the ground users, while collecting energy from the charging stations located at the center of a user group. For this purpose, the UAV must be examined and repaired regularly. In consequence, it is necessary to optimize the trajectory design of the UAV while jointly considering the maintenance costs, the reward of serving users, the energy management, and the user service time. To capture the relationship among these factors, we first model the completion of service and the harvested energy as the reward, and the energy consumption during the deployment as the cost. Then, the deployment profitability is defined as the ratio of the reward to the cost of the UAV trajectory. Based on this definition, the trajectory design problem is formulated as an optimization problem whose goal is to maximize the deployment profitability of the UAV. To solve this problem, a foraging-based algorithm is proposed to find the optimal trajectory so as to maximize the deployment profitability and minimize the average user service time. The proposed algorithm can find the optimal trajectory for the UAV with low time complexity at the level of polynomial. Fundamental analysis shows that the proposed algorithm achieves the maximal deployment profitability. Simulation results show that, compared to Q-learning algorithm, the proposed algorithm effectively reduces the operation time and the average user service time while achieving the maximal deployment profitability.

show abstract

Energy Management Strategy based on Deep Q-network in the Solar-powered UAV Communications System

Cited by 8 publications

References 15 publications

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

A Survey on Energy Optimization Techniques in UAV-Based Cellular Networks: From Conventional to Machine Learning Approaches

Journal of Smart Environments and Green Computing

Biological Intelligence Inspired Trajectory Design for Energy Harvesting UAV Networks

Contact Info

Product

Resources

About