Learning swimming escape patterns for larval fish under energy constraints

Mandralis, Ioannis; Weber, Pascal; Novati, Guido; Koumoutsakos, Petros

doi:10.1103/physrevfluids.6.093101

Cited by 7 publications

(11 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10(e) and (f), respectively. This style of 'burst-and-coast' swimming is frequently observed in fish 72 , and is consistent with other results in the RL swimming literature 29 .…”

Section: Results -Velocity and Path Trackingsupporting

confidence: 92%

“…While reinforcement learning is being widely applied in many areas of robotics [17][18][19][20] and fluid control [24][25][26] , swimming robots have for the most part not received much attention. A few notable recent exceptions employed reinforcement learning for tasks such as predicting efficient schooling configurations for pairs of swimmers 27 or larger schools 28 , or efficient start and escape patterns 29 . However, important problems related to mobile robotics such as station keeping, velocity tracking under disturbances or path tracking have not been addressed in the area of fish-like swimming robots due to the computational challenges of running a large number of high fidelity simulations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Physics-Informed Reinforcement Learning for Motion Control of a Fish-like Swimming Robot

Rodwell

Tallapragada

2022

Preprint

View full text Add to dashboard Cite

Motion control of fish-like swimming robots presents many challenges due to the unstructured environment and un-modelled governing physics of the fluid-robot interaction. Commonly used low-fidelity control models using simplified formulas for drag and lift force do not capture key physics that can play an important role in the dynamics of small-sized robots with limited actuation. Deep Reinforcement Learning (DRL) holds considerable promise for motion control of robots with complex dynamics. Reinforcement learning methods require large amounts of data exploring a large subset of the relevant state space, which can be expensive, time consuming or unsafe to obtain. Data from simulations can be used in the initial stages of DRL, but in the case of swimming robots, large numbers of simulations are also infeasible from a time and computational resources perspective. Surrogate models that capture the primary physics of the system can be a useful starting point for training a DRL agent which is subsequently transferred to train with a higher fidelity simulation. We demonstrate the utility of such physics-informed reinforcement learning to train a policy that can enable velocity and path tracking for a planar swimming (fish-like) rigid Joukowski hydrofoil. This is done through a curriculum where the DRL agent is first trained to track limit cycles in a velocity space for a representative nonholonomic system and then transferred to to train on a small simulation data set of the swimmer. The results show the utility of physics-informed reinforcement learning for the control of fish-like swimming robots.

show abstract

“…10(e) and (f), respectively. This style of 'burst-and-coast' swimming is frequently observed in fish 72 , and is consistent with other results in the RL swimming literature 29 .…”

Section: Results -Velocity and Path Trackingsupporting

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Physics-Informed Reinforcement Learning for Motion Control of a Fish-like Swimming Robot

Rodwell

Tallapragada

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We also showed that these changing energetics were reflected in the selection probabilities of different movements, such that there was a negative correlation between the increase in bout-type energy cost and increase in bout-type probability. This extends previous work investigating the energetics of burst-andcoast versus cyclical swimming between 1-5 dpf, where it was theoretically predicted [72] and experimentally demonstrated [14,48] that fluid-regime alterations of drag profiles for differing swim styles could explain the change from continuous to burst-and-coast swimming. Our study shows that this adaptation mechanism is not limited to early development, and may be maintained until at least juvenile stages.…”

Section: Connection Between Energy Change and Behavioral Changesupporting

confidence: 86%

Behavioral adaptation to changing energy constraints via altered frequency of movement selection

Darveniza,

Zhu,

Pujic

et al. 2023

Preprint

View full text Add to dashboard Cite

Animal behavior is strongly constrained by energy consumption. A natural manipulation which provides insight into this constraint is development, where an animal must adapt its movement to a changing energy landscape as its body grows. Unlike many other animals, for fish it is relatively easy to estimate the energy consumed by their movements via fluid mechanics. Here we simulated the fluid mechanics of>100,000 experimentally-recorded movement bouts from larval zebrafish across different ages and fluid conditions as they huntedParamecia. We find that these fish adapt to their changing relationship with the fluid environment as they grow by adjusting the frequency with which they select different types of movements, so that more expensive movements are chosen less often. This strategy was preserved when fish were raised in an unnaturally viscous environment. This work suggests a general principle by which animals could minimize energy consumption in the face of changing energy costs over development.

show abstract

“…Interesting behaviors from fishes have since been unearthed such as anti-predatory responses (Rieucau et al, 2014), encounters of fish with nets (Jones et al, 2008;Rudstam et al, 2011), differences in swimming speed (He, 1993;Breen et al, 2004;Spangler andCollins, 2011), avoidance (de Robertis andHandegard, 2013), exhaustion (Krag et al, 2009), orientation (Odling-Smee andBraithwaite, 2003;Holbrook and Perera, 2009;Haro et al, 2020), escapement (Glass et al, 1993;Mandralis et al, 2021), herding behavior (Ryer et al, 2006), and unique social behaviors (Anders et al, 2017a) from which selectivity studies in gears are based on. Knowledge of fish reaction and escape behavior has thus grown, leading to the development of novel gears with more open meshes, careful placement of sorting grids, and other devices to improve both size and species selectivity (Stewart, 2001;Watson and Kerstetter, 2006;Vogel, 2016;O'Neill et al, 2019).…”

Section: Observations Of Fish Behavior In Fishing Gearsmentioning

confidence: 99%

“…This behavior can be utilized to modify mechanical structures and panels in gears. Physical stimuli can play an important role for allowing fishes to escape (Mandralis et al, 2021) or be sorted (Larsen and Larsen, 1993;Brinkhof et al, 2020). These are usually installed in or on the gears after a series of behavioral trials on fish responses to different configurations (Santos et al, 2016).…”

Section: Responses To Physical Stimulimentioning

confidence: 99%

Artificial intelligence for fish behavior recognition may unlock fishing gear selectivity

2023

View full text Add to dashboard Cite

Through the advancement of observation systems, our vision has far extended its reach into the world of fishes, and how they interact with fishing gears—breaking through physical boundaries and visually adapting to challenging conditions in marine environments. As marine sciences step into the era of artificial intelligence (AI), deep learning models now provide tools for researchers to process a large amount of imagery data (i.e., image sequence, video) on fish behavior in a more time-efficient and cost-effective manner. The latest AI models to detect fish and categorize species are now reaching human-like accuracy. Nevertheless, robust tools to track fish movements in situ are under development and primarily focused on tropical species. Data to accurately interpret fish interactions with fishing gears is still lacking, especially for temperate fishes. At the same time, this is an essential step for selectivity studies to advance and integrate AI methods in assessing the effectiveness of modified gears. We here conduct a bibliometric analysis to review the recent advances and applications of AI in automated tools for fish tracking, classification, and behavior recognition, highlighting how they may ultimately help improve gear selectivity. We further show how transforming external stimuli that influence fish behavior, such as sensory cues and gears as background, into interpretable features that models learn to distinguish remains challenging. By presenting the recent advances in AI on fish behavior applied to fishing gear improvements (e.g., Long Short-Term Memory (LSTM), Generative Adversarial Network (GAN), coupled networks), we discuss the advances, potential and limits of AI to help meet the demands of fishing policies and sustainable goals, as scientists and developers continue to collaborate in building the database needed to train deep learning models.

show abstract

Learning swimming escape patterns for larval fish under energy constraints

Cited by 7 publications

References 62 publications

Physics-Informed Reinforcement Learning for Motion Control of a Fish-like Swimming Robot

Physics-Informed Reinforcement Learning for Motion Control of a Fish-like Swimming Robot

Behavioral adaptation to changing energy constraints via altered frequency of movement selection

Artificial intelligence for fish behavior recognition may unlock fishing gear selectivity

Contact Info

Product

Resources

About