Publisher Correction: SLEAP: A deep learning system for multi-animal pose tracking

Pereira, Talmo; Tabris, Nathaniel; Matsliah, Arie; Turner, D. T.; Li, Junyu; Ravindranath, Shruthi; Papadoyannis, Eleni S.; Normand, Edna; Deutsch, David; Wang, Z. Yan; McKenzie‐Smith, Grace C.; Mitelut, Catalin; Castro, Marielisa Diez; D’Uva, John; Kislin, Mikhail; Sanes, Dan H.; Kocher, Sarah D.; Wang, Samuel S.‐H.; Falkner, Annegret L.; Shaevitz, Joshua W.; Murthy, Mala

doi:10.1038/s41592-022-01495-2

Cited by 12 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tracking objects in images and videos has undergone a revolution with deep learning and neural network frame-works, where the tracking and reconstruction of complex animal postures is possible after training networks on only a few example images ( Mathis et al, 2018 ; Pereira et al, 2022 ). However, such approaches are computationally intensive and generally require dedicated and GPU hardware beyond the capabilities of the standard Raspberry Pi, making them incompatible with our project goals.…”

Section: Resultsmentioning

confidence: 99%

pi_tailtrack: A compact, inexpensive, and open-source behaviour-tracking system for head-restrained zebrafish

Randlett

2023

Preprint

View full text Add to dashboard Cite

Quantifying animal behavior during microscopy is crucial to associate optically recorded neural activity with behavioural outputs and states. Here I describe an imaging and tracking system for head-restrained larval zebrafish compatible with functional microscopy. This system is based on the Raspberry Pi computer, Pi NoIR camera, and open-source software for the real-time tail segmentation and skeletonization of the zebrafish tail at over 100hz. This allows for precise and long-term analyses of swimming behaviour, that can be related to functional signals recorded in individual neurons. This system offers a simple but performant solution for quantifying the behavior of head-restrained larval zebrafish, which can be built for 340 euro.

show abstract

Section: Resultsmentioning

confidence: 99%

pi_tailtrack: A compact, inexpensive, and open-source behaviour-tracking system for head-restrained zebrafish

Randlett

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In the field of behavior modeling, there exist three major groups of methods, supervised, unsupervised, and semi-supervised. The supervised methods consist of methods such as DeepLabCut (DLC) [7], LEAP [6], AlphaTracker [5], amongst others. Although these methods capture the positions of the subjects, they lack the ability to model smaller movements and unlabeled behavior, and necessitate tedious labeling.…”

Section: Discussionmentioning

confidence: 99%

“…Pose estimation tools such as DeepLabCut (DLC) and LEAP have been broadly applied to neuroscience experiments to track the body parts of animals performing different tasks, including in the social setting [3, 4, 5, 6, 7]. These are typically supervised techniques that require extensive manual labeling.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Disentangled multi-subject and social behavioral representations through a constrained subspace variational autoencoder (CS-VAE)

Musall

Churchland

et al. 2022

Preprint

View full text Add to dashboard Cite

Effectively modeling and quantifying behavior is essential for our understanding of the brain. Modeling behavior in naturalistic settings in social and multi-subject tasks in a unified manner remains a significant challenge. Modeling the behavior of different subjects performing the same task requires partitioning the behavioral data into features that are common across subjects, and others that are distinct to each subject. Modeling social interactions between multiple individuals in a freely-moving setting requires disentangling effects due to the individual as compared to social investigations. To achieve flexible disentanglement of behavior into interpretable latent variables with individual and across-subject or social components, we build on a semi-supervised approach to partition the behavioral subspace, and propose a novel regularization based on the Cauchy-Schwarz divergence to the model. Our model, known as the constrained subspace variational autoencoder (CS-VAE), successfully models distinct features of the behavioral videos across subjects, as well as continuously varying differences in social behavior. Our approach vastly facilitates the analysis of the resulting latent variables in downstream tasks such as uncovering disentangled behavioral motifs and the efficient decoding of a novel subject's behavior.

show abstract

“…Rich behavioral datasets representing a large swath of a species' ethogram have now been collected and can be deployed to benchmark performance on species-specific embodied Turing tests. Furthermore, these datasets are being rapidly expanded given new tools in 3D videography [54][55][56][57] . Additionally, detailed biomechanical measurements support highly realistic animal body models, complete with skeletal constraints, muscles, tendons, and paw features 58 .…”

Section: A Roadmap For Solving the Embodied Turing Testmentioning

confidence: 99%

Catalyzing next-generation Artificial Intelligence through NeuroAI

et al. 2023

View full text Add to dashboard Cite

Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities – inherited from over 500 million years of evolution – that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.

show abstract

Publisher Correction: SLEAP: A deep learning system for multi-animal pose tracking

Abstract: In the version of this article initially published, there were errors in the y-axis labels in Fig. 3j. From top down, the first two labels, now reading "Python" and "R", were transposed. The correction has been made in the HTML and PDF versions of the article.

Cited by 12 publications

References 0 publications

pi_tailtrack: A compact, inexpensive, and open-source behaviour-tracking system for head-restrained zebrafish

pi_tailtrack: A compact, inexpensive, and open-source behaviour-tracking system for head-restrained zebrafish

Disentangled multi-subject and social behavioral representations through a constrained subspace variational autoencoder (CS-VAE)

Catalyzing next-generation Artificial Intelligence through NeuroAI

Contact Info

Product

Resources

About