idtracker.ai: tracking all individuals in small or large collectives of unmarked animals

Romero-Ferrero, Francisco; Bergomi, Mattia G.; Hinz, Robert C.; Heras, Francisco J. H.; Polavieja, Gonzalo G. de

doi:10.1038/s41592-018-0295-5

Cited by 246 publications

(256 citation statements)

References 12 publications

Supporting

Mentioning

255

Contrasting

Order By: Relevance

“…Recent machine vision advances in the precise posture tracking of individual animals 17,18,52 as well as of the positions of highly-similar organisms in groups 46 are enabling new quantitative studies of behavior 53,54 . In collective behavior specifically, the use of CNNs for the pixel-based identification of individual organisms has significantly advanced markerless, long-time tracking in 2D, from more modest assemblies (~10 individuals) 34 to large groups (~100 individuals) 46 . However, while networklearned identities can resolve confounding occlusions and overlaps, a principal challenge of individualresolution group tracking, there must also be enough isolated instances to train the identification network.…”

Section: Discussionmentioning

confidence: 99%

“…Previous studies 34,35,46 have demonstrated that objects indistinguishable to human eye might nonetheless carry unique visual signatures, also termed 'pixel personality', that can be quantified and can importantly aid the task of simultaneous tracking of multiple highly similar objects. Notably, the rise of deep learning has enabled new ways of extracting such visual signatures from images.…”

Section: Colony-wide Tracking At Single-organism Resolutionmentioning

confidence: 99%

“…However, using CNNs for the quantification of pixel identities requires a set of instances of each object that can serve as a training set. For example, in tracking up to 100 fish such a set could be constructed if a segment in a video existed where no fish collisions were observed during 3,000 video frames 46 . Notably in the much denser environment of ~1,000 individuals inside of a beehive, such collision-free segments are rare or nonexistent.…”

Section: Colony-wide Tracking At Single-organism Resolutionmentioning

confidence: 99%

See 2 more Smart Citations

Markerless tracking of an entire insect colony

Bożek

Hebert

Portugal

et al. 2020

Preprint

View full text Add to dashboard Cite

We present a comprehensive, computational method for tracking an entire colony of the honey bee Apis mellifera using high-resolution video on a natural honeycomb background. We adapt a convolutional neural network (CNN) segmentation architecture to automatically identify bee and brood cell positions, body orientations and within-cell states. We achieve high accuracy (~10% body width error in position, ~10° error in orientation, and true positive rate > 90%) and demonstrate months-long monitoring of sociometric colony fluctuations. We combine extracted positions with rich visual features of organismcentered images to track individuals over time and through challenging occluding events, recovering ~79% of bee trajectories from five observation hives over a span of 5 minutes. The resulting trajectories reveal important behaviors, including fast motion, comb-cell activity, and waggle dances. Our results provide new opportunities for the quantitative study of collective bee behavior and for advancing tracking techniques of crowded systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Colony-wide Tracking At Single-organism Resolutionmentioning

confidence: 99%

Section: Colony-wide Tracking At Single-organism Resolutionmentioning

confidence: 99%

See 1 more Smart Citation

Markerless tracking of an entire insect colony

Bożek

Hebert

Portugal

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This bias could be overcome by the tracking of individuals over multiple images (e.g. see Romerro-Ferrero, Bergomi, Hinz, Heras, & Polavieja, 2019). (Tresson et al, 2019).…”

Section: Further Improvementsmentioning

confidence: 99%

CORIGAN: Assessing multiple species and interactions within images

et al. 2019

View full text Add to dashboard Cite

Images are resourceful data for ecologists and can provide a more complete information than other methods to study biodiversity and the interactions between species. Automated image analysis however often relies on extensive datasets, not implementable by small research teams. We are here proposing an object detection method that allows the analysis of high‐resolution images containing many animals interacting in a small dataset. We developed an image analysis pipeline named ‘CORIGAN’ to extract the characteristics of animal communities. CORIGAN is based on the YOLOv3 model as the core of object detection. To illustrate potential applications, we use images collected during a sentinel prey experiment. Our pipeline can be used to detect, count and study the physical interactions between various animals. On our example dataset, the model reaches 86.6% precision and 88.9% recall at the species level or even at the caste level for ants. The training set required fewer than 10 hr of labelling. Based on the pipeline output, it was possible to build the trophic and non‐trophic interactions network describing the studied community. CORIGAN relies on generic properties of the detected animals and can be used for a wide range of studies and supports. Here, we study invertebrates on high‐resolution images, but the same processing can be transferred for the study of larger animals on satellite or aircraft images.

show abstract

“…Trackers can assist in a wide range of experimental tasks such as monitoring activity, measuring response to stimuli 1;2 , and locating body parts over time 3;4 . Some trackers are designed to track and maintain identities of multiple individuals occupying the same arena [5][6][7][8] while others measure the collective activity of groups without maintaining identities or rely on physical segregation of animals to ensure trajectories never collide [9][10][11][12] . But few of these trackers are designed as platforms for high throughput, hardware control, and flexible experimental reconfiguration.…”

Section: Introductionmentioning

confidence: 99%

MARGO (Massively Automated Real-time GUI for Object-tracking), a platform for high-throughput ethology

Werkhoven

Rohrsen

Qin

et al. 2019

Preprint

View full text Add to dashboard Cite

Fast object tracking in real time allows convenient tracking of very large numbers of animals and closed-loop experiments that control stimuli for multiple animals in parallel. We developed MARGO, a real-time animal tracking suite for custom behavioral experiments. We demonstrated that MARGO can rapidly and accurately track large numbers of animals in parallel over very long timescales. We incorporated control of peripheral hardware, and implemented a flexible software architecture for defining new experimental routines. These features enable closed-loop delivery of stimuli to many individuals simultaneously. We highlight MARGO's ability to coordinate tracking and hardware control with two custom behavioral assays (measuring phototaxis and optomotor response) and one optogenetic operant conditioning assay. There are currently several open source animal trackers. MARGO's strengths are 1) robustness, 2) high throughput, 3) flexible control of hardware and 4) real-time closed-loop control of sensory and optogenetic stimuli, all of which are optimized for large-scale experimentation.

show abstract

idtracker.ai: tracking all individuals in small or large collectives of unmarked animals

Cited by 246 publications

References 12 publications

Markerless tracking of an entire insect colony

Markerless tracking of an entire insect colony

CORIGAN: Assessing multiple species and interactions within images

MARGO (Massively Automated Real-time GUI for Object-tracking), a platform for high-throughput ethology

Contact Info

Product

Resources

About