Optimal steering of a smart active particle

Abstract

We formulate the theory for steering an active particle with optimal travel time between two locations and apply it to the Mexican hat potential without brim. For small heights the particle can cross the potential barrier, while for large heights it has to move around it. Thermal fluctuations in the orientation strongly affect the path over the barrier. Then we consider a smart active particle and apply reinforcement learning. We show how the active particle learns in repeating episodes to move optimally. The optimal steering is stored in the optimized action-value function, which is able to rectify thermal fluctuations.

Machine learning for active matter

“…A recent proof-of-principle study demonstrated how reinforcement learning finds good strategies for navigation in a steady flow 62 (Fig. 4b), and for point-to-point navigation in complex fluid environments [63][64][65] . Reinforcement learning can be used to find strategies for marine probes to target certain oceanic regions of interest 64 , and also yields fundamental insight into how birds soar in thermal updrafts guided by cues from the turbulent air flow 66 , enabling gliders to soar in such updrafts 67 (Fig.…”

Hydrodynamics can determine the optimal route for microswimmer navigation

“…Likewise, navigation and search strategies are frequently encountered in a plethora of biological systems, including the foraging of animals for food 2 , or of T cells searching for targets to mount an immune response 3 . Very recently there is a growing interest also in optimal navigation problems and search strategies [4][5][6][7][8][9] of microswimmers [10][11][12][13] and "dry" active Brownian particles [14][15][16][17][18] . These active agents can self-propel in a low-Reynoldsnumber solvent, and might play a key role in tomorrow's nanomedicine as recently popularized, e.g.…”

“…on space vehicles. Recent work has explored optimal navigation problems of dry active particles (and particles in external flow fields) accounting for (i) and partly also for (ii): Specifically, the very recent works 4,5,8,9,[25][26][27][28][29][30][31][32][33] have pioneered the usage of reinforcement learning [34][35][36] , e.g. to determine optimal steering strategies of active particles to optimally navigate toward a target position 4,5,8,9 or to exploit external flow fields to avoid getting trapped in certain flow structures by learning smart gravitaxis 25 .…”

“…5,6,37 have used (deep) reinforcement learning to explore microswimmer navigation problems in mazes and obstacle arrays assuming global 5 or only local 6 knowledge of the environment. Very recent analytical approaches 7,8 to optimal active particle navigation complement these works and allow testing machine-learned results 8,9 . (In addition, note that a significant knowledge exists on the complementary problem of optimizing body-shape deformation of deformable swimmers with optimal control theory; see e.g.…”

Why Do Animals Swirl and How Do They Group?

“…Once we accept such a hypothesis we immediately recognize that the trial-and-error method, supplemented by a reward and penalty is in the heart of the reinforcement learning (RL) -one of the most powerful tools of machine learning (ML) techniques. This method has been successfully exploited for various transport problems of active particles [10,32,36]. The application of ML to communication problems, including animal communications has also demonstrated its efficiency [17,19,23,37,40,42].…”