The need for automated and efficient systems for tracking full animal pose has increased with the complexity of behavioral data and analyses. Here we introduce LEAP (LEAP estimates animal pose), a deep-learning-based method for predicting the positions of animal body parts. This framework consists of a graphical interface for labeling of body parts and training the network. LEAP offers fast prediction on new data, and training with as few as 100 frames results in 95% of peak performance. We validated LEAP using videos of freely behaving fruit flies and tracked 32 distinct points to describe the pose of the head, body, wings and legs, with an error rate of <3% of body length. We recapitulated reported findings on insect gait dynamics and demonstrated LEAP’s applicability for unsupervised behavioral classification. Finally, we extended the method to more challenging imaging situations and videos of freely moving mice.

The desire to understand how the brain generates and patterns behavior has driven rapid methodological innovation in tools to quantify natural animal behavior. While advances in deep learning and computer vision have enabled markerless pose estimation in individual animals, extending these to multiple animals presents unique challenges for studies of social behaviors or animals in their natural environments. Here we present Social LEAP Estimates Animal Poses (SLEAP), a machine learning system for multi-animal pose tracking. This system enables versatile workflows for data labeling, model training and inference on previously unseen data. SLEAP features an accessible graphical user interface, a standardized data model, a reproducible configuration system, over 30 model architectures, two approaches to part grouping and two approaches to identity tracking. We applied SLEAP to seven datasets across flies, bees, mice and gerbils to systematically evaluate each approach and architecture, and we compare it with other existing approaches. SLEAP achieves greater accuracy and speeds of more than 800 frames per second, with latencies of less than 3.5 ms at full 1,024 × 1,024 image resolution. This makes SLEAP usable for real-time applications, which we demonstrate by controlling the behavior of one animal on the basis of the tracking and detection of social interactions with another animal.

Recent work quantifying postural dynamics has attempted to define the repertoire of behaviors across the entire data set, we use a technique we refer to as cluster sampling. A simple random 117 subset of the movie images are grouped via k-means clustering and then these images are 118 sampled uniformly across groups for labeling. The grouping is based on linear correlations 119 between pixel intensities in the images as a proxy measure for similarity in body pose. The 120 diversity of poses represented using this method can be observed in the centroids of each of the 121 clusters identified (Supplementary Fig. 2). 122 123 Poses in each training image are labeled using a custom GUI with draggable body part markers 124 that form a skeleton (Fig. 1b). For the fruit fly, we track four points on each of the six legs, two 125 points on the wing tips, three points on the thorax and abdomen, and three points on the head 126 for a total of 32 points in every frame. These points were chosen to align with known Drosophila 127 body joints (Supplementary Fig. 3). For every training image, the user drags each skeleton 128 point to the appropriate body part and the program saves the label positions into a self-129 contained file. To enhance the size of the training image set further without the need for hand 130 labeling more frames, we augment the dataset by applying small random rotations and body-131 axis reflections to generate new samples from the labeled data. As the neural network 132 processes the raw images, the rotated and reflected images add new information that the 133 network can use during training. 134 135

Mitochondrial DNA (mtDNA) mutations manifest with vast clinical heterogeneity. The molecular basis of this variability is mostly unknown because the lack of model systems has hampered mechanistic studies. We generated induced pluripotent stem cells from patients carrying the most common human disease mutation in mtDNA, m.3243A>G, underlying mitochondrial encephalomyopathy, lactic acidosis, and stroke-like episodes (MELAS) syndrome. During reprogramming, heteroplasmic mtDNA showed bimodal segregation toward homoplasmy, with concomitant changes in mtDNA organization, mimicking mtDNA bottleneck during epiblast specification. Induced pluripotent stem cell-derived neurons and various tissues derived from teratomas manifested cell-type specific respiratory chain (RC) deficiency patterns. Similar to MELAS patient tissues, complex I defect predominated. Upon neuronal differentiation, complex I specifically was sequestered in perinuclear PTEN-induced putative kinase 1 (PINK1) and Parkin-positive autophagosomes, suggesting active degradation through mitophagy. Other RC enzymes showed normal mitochondrial network distribution. Our data show that cellular context actively modifies RC deficiency manifestation in MELAS and that autophagy is a significant component of neuronal MELAS pathogenesis. mitochondria | disease modeling