Study on the Glider Soaring Strategy in Random Location Thermal Updraft via Reinforcement Learning

Abstract

Soaring birds can use thermal updrafts in natural environments to fly for long periods or distances. The flight strategy of soaring birds can be implemented to gliders to increase their flight time. Currently, studies on soaring flight strategies focus on the turbulent nature of updrafts while neglecting the random characteristics of its generation and disappearance. In addition, most flight strategies only focus on utilizing updrafts while neglecting how to explore it. Therefore, in this paper, a complete flight strategy that seeks and uses random location thermal updrafts is mainly emphasized and developed. Moreover, through the derivation of flight dynamics and related formulas, the principle of gliders acquiring energy from thermal updrafts is explained through energy concepts. This concept lays a theoretical foundation for research on soaring flight strategies. Furthermore, the method of reinforcement learning is adopted, and a perception strategy suitable for gliders that considers the vertical ground speed, vertical ground speed change rate, heading angle, and heading angle change as the main perception factors is developed. Meanwhile, an area exploring strategy was trained by reinforcement learning, and the two strategies were combined into a complete flight strategy that seeks and uses updrafts. Finally, based on the guidance of the soaring strategy, the flight of the glider in the simulation environment is tested. The soaring strategy is verified to significantly improve the flight time lengths of gliders.

Using Explainable Artificial Intelligence (XAI) to Predict the Influence of Weather on the Thermal Soaring Capabilities of Sailplanes for Smart City Applications

“…As the velocity of the rising air tends to be strongest in the centre of a thermal [6], early research therefore focused on methods to centre gliders in these thermals [24]. In fact, most of the research until 2012 was focused on utilising thermals efficiently, as opposed to finding them [28]. Thanks to the increasing computational power of onboard computers, several autonomous soaring algorithms have been proposed and developed [6].…”

“…Thanks to the increasing computational power of onboard computers, several autonomous soaring algorithms have been proposed and developed [6]. Some initial research has been undertaken on finding thermals; for example, Cui et al [28] used reinforcement learning to develop an 'area exploring strategy' to seek and benefit from thermal updrafts. They tested their strategy in a simulated environment with random thermals.…”

Revealing principles of autonomous thermal soaring in windy conditions using vulture-inspired deep reinforcement-learning

“…Such strategy should balance two, often conflicting objectives: locally, it should exploit thermal updrafts to gain altitude and, globally, it should race between designated way-points 9 , 35 . The exploration-exploitation problem in thermal soaring was also addressed by Cui et al, who solved it in a simulated environment using deep-RL and combining new energy considerations 36 .…”