Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in three-dimensional (3D) space. Deep neural networks can estimate two-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here, we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.
Studying how neural circuits orchestrate limbed behaviors requires the precise 10 measurement of pose-the positions of each appendage-in 3-dimensional (3D) space. Recent 11 advances in computer vision and machine learning have made it possible to use deep neural 12 networks to estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, 13the unique challenges associated with transforming these measurements into reliable and precise 14 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. 15 Here we present DeepFly3D, a computational pipeline for inferring the 3D pose of tethered, adult 16 Drosophila using multiple camera images. First, we introduce an approach for multiple-camera 17 calibration using the animal itself rather than the typical checkerboard or similar external 18 apparatus. Second, we present an iterative approach that robustly infers 3D pose using graphical 19 models and deep-learning based 2D predictions from multiple cameras. False predictions are 20 rejected using an optimization scheme based on dynamic programming and belief propagation. To 21 close the loop, the corrected poses are used to retrain the 2D pose deep estimation network and to 22 improve 3D pose estimation. Finally, we provide a graphical user interface (GUI) and active learning 23 policy for interacting with, annotating, and correcting 3D pose data. Emphasizing the importance of 24 our tool, we demonstrate that unsupervised behavioral embedding of 3D joint angles yields more 25 accurate behavioral maps than those generated with 2D pose data because the latter are highly 26 perspective-dependent. We provide our DeepFly3D deep network and weights, training data, 27 computational pipeline, and nearly one million images of tethered, behaving flies along with 28 corresponding 3D joint positions 1 . These tools make it possible to acquire high-fidelity behavioral 29 measurements at an unprecedented level of precision and resolution for a variety of biological 30 applications. 31 32 35 exoskeletons, these measurements are naturally made with reference to 3D joint and appendage 36 locations. Paired with modern approaches to simultaneously record the activity of neural popu-37 lations in tethered, behaving animals (Dombeck et al., 2007; Seelig et al., 2010; Chen et al., 2018), 38 3D joint and appendage tracking promises to accelerate the dissection of neural control principles, 39 1 https://github.com/NeLy-EPFL/DeepFly3D particularly in the genetically tractable and numerically simple nervous system of the fly, Drosophila 40 melanogaster. 41 However, algorithms for reliably estimating 3D pose in such small animals have not yet been 42 developed. Instead, multiple alternative approaches have been taken. For example, one can affix 43 and use small markers-reflective, colored, or fluorescent particles-to identify and reconstruct 44 keypoints from video data (Bender et al., 2010; Kain et al., 2013; Todd et al., 2017). Although this 45 approach works well on h...
During neural circuit assembly, axonal growth cones are exposed to multiple guidance signals at trajectory choice points. While axonal responses to individual guidance cues have been extensively studied, less is known about responses to combination of signals and underlying molecular mechanisms. Here, we studied the convergence of signals directing trajectory selection of spinal motor axons entering the limb. We first demonstrate that Netrin-1 attracts and repels distinct motor axon populations, according to their expression of Netrin receptors. Quantitative in vitro assays demonstrate that motor axons synergistically integrate both attractive or repulsive Netrin-1 signals together with repulsive ephrin signals. Our investigations of the mechanism of ephrin-B2 and Netrin-1 integration demonstrate that the Netrin receptor Unc5c and the ephrin receptor EphB2 can form a complex in a ligand-dependent manner and that Netrin–ephrin synergistic growth cones responses involve the potentiation of Src family kinase signaling, a common effector of both pathways.DOI: http://dx.doi.org/10.7554/eLife.10841.001
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.