Predicting the long-term collective behaviour of fish pairs with deep learning

Abstract

Modern computing has enhanced our understanding of how social interactions shape collective behaviour in animal societies. Although analytical models dominate in studying collective behaviour, this study introduces a deep learning model to assess social interactions in the fish species Hemigrammus rhodostomus . We compare the results of our deep learning approach with experiments and with the results of a state-of-the-art analytical model. To that end, we propose a systematic methodology to assess the faithfulness of a collective motion model, exploiting a set of stringent individual and collective spatio-temporal observables. We demonstrate that machine learning (ML) models of social interactions can directly compete with their analytical counterparts in reproducing subtle experimental observables. Moreover, this work emphasizes the need for consistent validation across different timescales, and identifies key design aspects that enable our deep learning approach to capture both short- and long-term dynamics. We also show that our approach can be extended to larger groups without any retraining, and to other fish species, while retaining the same architecture of the deep learning network. Finally, we discuss the added value of ML in the context of the study of collective motion in animal groups and its potential as a complementary approach to analytical models.

Quantifying the biomimicry gap in biohybrid robot-fish pairs

“…We use a pretrained version of the Deep Learning Interaction (DLI) model [36], to generate real-time goal positions for the LureBot [35]. The DLI consists of 7 layers (see figure 2(b): 1st and 4th are LSTM layers [24]; the remaining are densely connected layers; ReLU activations are used for all layers except for the last one which is linear.…”

“…As a matter of fact, many fish-robot systems have been proposed to investigate various aspects of fish behavior, employing behavioral models with diverse degrees of detail and realistic features, and typically relying on analytical modeling approaches based on observation of fish interaction [6, 7, 15-17, 19, 21, 29, 30, 34, 39, 42, 44, 47]. Concurrently, machine learning-based modeling approaches have gained a growing interest [13,14,20,23,36], but only a handful have been tested in real-time with a robotic device [13]. These machine learning approaches are usually intended to study collective behavior by predicting motion in simulations alone [13,20,23], while the studies that exploit 1) The modeling phase may introduce a first source of discrepancy between the effect of social interactions on the swimming patterns in the model and the ones observed in real fish.…”

“…A few flocking models for fish behavior, analytical or machine learning, have been evaluated in extended simulations to study long-term emergent collective behavior [12,19,36]. However, these models have not been tested and validated in biohybrid groups.…”

“…Consequently, none of these models have been stringently benchmarked on both shortand long-term timescales within both simulation and fish-robot biohybrid experiments. In fact, previous research indicates that certain models may yield satisfactory biomimetic outcomes in the short term while failing to reproduce emergent dynamics accurately on longer time scales [36].…”

“…In this study, we investigate this notion by employing the (pretrained) machine learning model presented in [36]. We implement this model on a robotic system which is shown to achieve unprecedented levels of biomimicry, the LureBot [35].…”