Beyond tracking: using deep learning to discover novel interactions in biological swarms

Abstract

Most deep-learning frameworks for understanding biological swarms are designed to fit perceptive models of group behavior to individual-level data (e.g., spatial coordinates of identified features of individuals) that have been separately gathered from video observations. Despite considerable advances in automated tracking, these methods are still very expensive or unreliable when tracking large numbers of animals simultaneously. Moreover, this approach assumes that the human-chosen features include sufficient features to explain important patterns in collective behavior. To address these issues, we propose training deep network models to predict system-level states directly from generic graphical features from the entire view, which can be relatively inexpensive to gather in a completely automated fashion. Because the resulting predictive models are not based on human-understood predictors, we use explanatory modules (e.g., Grad-CAM) that combine information hidden in the latent variables of the deep-network model with the video data itself to communicate to a human observer which aspects of observed individual behaviors are most informative in predicting group behavior. This represents an example of augmented intelligence in behavioral ecology -knowledge co-creation in a human-AI team. As proof of concept, we utilize a 20-day video recording of a colony of over 50 Harpegnathos saltator ants to showcase that, without any individual annotations provided, a trained model can generate an "importance map" across the video frames to highlight regions of important behaviors, such as dueling (which the AI has no a priori knowledge of), that play a role in the resolution of reproductive-hierarchy re-formation. Based on the empirical results, we also discuss the potential use and current challenges to further develop the proposed framework as a tool to discover behaviors that have not yet been considered crucial to understand complex social dynamics within biological collectives.

Dynamic pathogen detection and social feedback shape collective hygiene in ants

“…The identified rules contribute to preferential grooming of the most-infectious colony members by the individuals with the lowest own infectiousness, identified by epidemiological modeling as important to reduce disease risk for the colony 33 . Such immediate functional benefits – and thus evolutionary relevance – of emergent behaviors can rarely be pinpointed 57 , despite recent technical advances that spurred a multitude of studies on collective actions across the social, physical, and life sciences 58 , 59 . Our results showcase the importance of collective behaviors for the evolution of social insects, whose fitness – similar to other cooperative entities like cells in a body – inherently depends on group-level performance 60 – 62 .…”

Rural Tourism Marketing Strategy under Natural Protection Environment Based on Deep Learning

“…Due to the large and fast changing demand of China's rural tourism market, marketing needs to constantly adjust strategies to meet market demand. And with the upgrading of consumer demand for services, the consumer demand also shows a diversified trend [15,16]. The demand of Chinese tourists for rural tourism is diversified, with few personalized products and high demand diversity.…”