Learning efficient navigation in vortical flow fields

Gunnarson, Peter; Mandralis, Ioannis; Novati, Guido; Koumoutsakos, Petros; Dabiri, John O.

doi:10.1038/s41467-021-27015-y

“…RL can also be used to station a stratospheric Loon superpressure balloon at multiple locations using a RL controller [23]. In addition, a RL algorithm has been trained to efficiently navigate in vortical flow fields [24]. Finally, an actor-critic architecture has also been used to track a ground target by an unmanned autonomous vehicle [25], where the RL network is able to control an agent to avoid collisions and reach the target using range and angle information by using Recurrent Neural Networks (RNN).…”

Section: Related Workmentioning

confidence: 99%

A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

Masmitja,

Martin,

Katija

et al. 2023

Preprint

0

View full text Add to dashboard Cite

Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target's localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets could be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenarios.

show abstract

“…The goal is for eventual robotic swarms to behave autonomously. Applications of emerging techniques in artificial intelligence such as reinforcement learning (Gunnarson et al, 2021) can enable robust control with limited onboard computational processing, and physical tests can be performed in a threestory tall tank with controllable upwelling and downwelling water currents at the California Institute of Technology (Figure 8a,b).…”

Section: The Future Of Bio-inspired Ocean Explorationmentioning

confidence: 99%

Bio-Inspired Ocean Exploration

Xu¹,

Dabiri²

2022

Oceanog

View full text Add to dashboard Cite

Substantial efforts have been made to expand our knowledge of the physics, biology, chemistry, and geography of the ocean using state-of-the-art measurement tools. With new global projects and technological advances, the collaborative efforts of the Ocean Decade (2021–2030) are well on the way to revolutionizing our knowledge of ocean sciences and sustainability. Yet even today, over three-quarters of the seafloor is still unmapped, more than 90% of marine life still awaits discovery and classification, and the number of ocean sensors required to study global phenomena at sufficient temporal and spatial resolutions is seemingly intractable. To address this challenge, new approaches such as bio-inspired robotics can expand our existing toolbox and bridge this knowledge gap. The concept of biology-inspired engineering has emerged as a powerful tool to complement traditional engineering approaches to technology development. For example, specific swimming features of jellyfish and fish have been applied to a variety of fields, from vehicular propulsion to wind energy to medical diagnostics. In particular, jellyfish are advantageous model organisms because of their energy efficiency, with the lowest known cost of transport compared to other animals, as well as their ubiquity and survivability in various ocean environments. In this article we highlight the evolution of research into jellyfish-inspired robotic constructs and their potential applications in ocean exploration. After initial projects using entirely engineered materials (i.e., jellyfish-inspired submarine propellers) and tissue engineering methods (i.e., rat cardiac cells seeded on flexible films), recent work to integrate microelectronic systems onto live jellyfish demonstrates that their swimming speeds can be increased (up to three times compared to their baselines) and their energy efficiency can be improved (up to four times compared to their baselines). This shows promise for the robotic control of jellyfish in real-world oceanic environments, where the animals are already distributed globally. Future work can improve the maneuverability of these bio-hybrid jellyfish robots, incorporate miniaturized sensors to profile regions of interest, and ultimately deploy swarms of these low-power, low-cost robots to obtain high-resolution data and improve ocean climate models. The synergy of bio-inspired technologies with existing ocean measurement tools holds promise to push the frontiers of ocean exploration and stewardship.

show abstract

“…Or to station a stratospheric Loon superpressure balloon at multiple locations using a RL controller [23]. In addition, a RL algorithm has been trained to efficiently navigate in vortical flow fields [24]. Finally, an actor-critic architecture has also been used to track a ground target by an unmanned autonomous vehicle [25], where the RL network is able to control an agent to avoid collisions and reach the target using range and angle information by using Recurrent Neural Networks (RNN).…”

Section: Related Workmentioning

confidence: 99%

A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

Masmitja¹,

Martín

²

,

Katija³

et al. 2022

2022 IEEE 18th International Conference on Automation Science and Engineering (CASE)

View full text Add to dashboard Cite

Underwater target localization using range-only and single-beacon (ROSB) techniques with autonomous vehicles has been used recently to improve the limitations of more complex methods, such as long baseline and ultra-short baseline systems. Nonetheless, in ROSB target localization methods, the trajectory of the tracking vehicle near the localized target plays an important role in obtaining the best accuracy of the predicted target position. Here, we investigate a Reinforcement Learning (RL) approach to find the optimal path that an autonomous vehicle should follow in order to increase and optimize the overall accuracy of the predicted target localization, while reducing time and power consumption. To accomplish this objective, different experimental tests have been designed using state-of-the-art deep RL algorithms. Our study also compares the results obtained with the analytical Fisher information matrix approach used in previous studies. The results revealed that the policy learned by the RL agent outperforms trajectories based on these analytical solutions, e.g. the median predicted error at the beginning of the target's localisation is 17% less. These findings suggest that using deep RL for localizing acoustic targets can be successfully applied to in-water applications that include tracking of acoustically tagged marine animals by autonomous underwater vehicles. This is envisioned as a first necessary step to validate the use of RL to tackle such problems, which could be used later on in a more complex scenarios I. INTRODUCTIONOne of the main challenges in marine research lies in underwater positioning of underwater features or assets (e.g., marine species [1] or underwater vehicles [2]). Due to the large attenuation of radio waves in water [3], Global Positioning System (GPS) signals are not suitable for positioning underwater targets. Nonetheless, acoustic signals can fill the underwater communications gap left by radio waves. Acoustic signals have much greater underwater propagation capabilities [4], and therefore, a network of nodes or beacons can be deployed and used to localize underwater targets, which may include Autonomous Underwater Vehicles (AUV), benthic rovers, or acoustically tagged organisms.Unfortunately, underwater acoustic deployments are often complex and highly economically and logistically expensive

show abstract

Learning efficient navigation in vortical flow fields

Cited by 44 publications

References 32 publications

A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

Bio-Inspired Ocean Exploration

A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles

Contact Info

Product

Resources

About