Tracking All Members of a Honey Bee Colony Over Their Lifetime Using Learned Models of Correspondence

Abstract

Computational approaches to the analysis of collective behavior in social insects increasingly rely on motion paths as an intermediate data layer from which one can infer individual behaviors or social interactions. Honey bees are a popular model for learning and memory. Previous experience has been shown to affect and modulate future social interactions. So far, no lifetime history observations have been reported for all bees of a colony. In a previous work we introduced a recording setup customized to track up to 4,000 marked bees over several weeks. Due to detection and decoding errors of the bee markers, linking the correct correspondences through time is non-trivial. In this contribution we present an in-depth description of the underlying multi-step algorithm which produces motion paths, and also improves the marker decoding accuracy significantly. The proposed solution employs two classifiers to predict the correspondence of two consecutive detections in the first step, and two tracklets in the second. We automatically tracked ∼2,000 marked honey bees over 10 weeks with inexpensive recording hardware using markers without any error correction bits. We found that the proposed two-step tracking reduced incorrect ID decodings from initially ∼13% to around 2% post-tracking. Alongside this paper, we publish the first trajectory dataset for all bees in a colony, extracted from ∼3 million images covering 3 days. We invite researchers to join the collective scientific effort to investigate this intriguing animal system. All components of our system are open-source.

Machine learning for active matter

“…Data acquisition and analysis. The most important and common use of machine learning in active-matter research is in the analysis and classification of experimental data using supervised learning models (usually neural nets) [33][34][35][36][37] (Box 1). In fact, most active-matter experiments are performed using video microscopy, which provides large, high-quality training datasets that also cover less likely experimental conditions, which are ideally suited to image analysis with supervised machine-learning methods 38 .…”

Markerless tracking of an entire insect colony

“…Our markerless detection and tracking techniques offer new possibilities for the quantitative study of honey bee colonies on the collective scale at single-organism resolution and are complementary to existing approaches 11 . Both the brood and adult populations change over time (Fig.…”

“…These factors present substantial difficulties for automated image analysis for which a common solution is to apply physical tags to some 8 or most 9,10 of the colony members. Barcoded tags allow for the distinct marking of a sufficiently large number of individuals to track a naturally-sized colony and have been exploited to unravel important aspects of bee communication 8,11 and information spread 10 . However, the burden of manually tagging hundreds or thousands of small insects, without harm or inhibition to their motion, carries severe limitations.…”

“…However, the burden of manually tagging hundreds or thousands of small insects, without harm or inhibition to their motion, carries severe limitations. For example, marking newly hatched bees requires either opening the hive or introducing marked newborns without letting any hatch inside, both of which disrupt the colony 9,11 . Moreover, the recognition of markers fails when the tag is partially occluded or blurred.…”

Data-Driven Modeling of Resource Distribution in Honeybee Swarms

“…Tracking the unmarked bees is a major challenge, however, despite recent improvements in tracking techniques (Boenisch et al, 2018;Bozek et al, 2018), because of the occlusions in the clusters, which makes it all but impossible to identify individuals. As a way around this problem, we converted the videos of each experiment to a sequence of average density maps, constructed using Otsu's method (Otsu, 1979).…”