The integration of sensorimotor signals to internally estimate self-movement is critical for spatial perception and motor control. However, which neural circuits accurately track body motion and how these circuits control movement remain unknown. We found that a population of Drosophila neurons that were sensitive to visual flow patterns typically generated during locomotion, the horizontal system (HS) cells, encoded unambiguous quantitative information about the fly's walking behavior independently of vision. Angular and translational velocity signals were integrated with a behavioral-state signal and generated direction-selective and speed-sensitive graded changes in the membrane potential of these non-spiking cells. The nonvisual direction selectivity of HS cells cooperated with their visual selectivity only when the visual input matched that expected from the fly's movements, thereby revealing a circuit for internally monitoring voluntary walking. Furthermore, given that HS cells promoted leg-based turning, the activity of these cells could be used to control forward walking.

Videos of animal behavior are used to quantify researcher-defined behaviors-of-interest to study neural function, gene mutations, and pharmacological therapies. Behaviors-of-interest are often scored manually, which is time-consuming, limited to few behaviors, and variable across researchers. We created DeepEthogram: software that uses supervised machine learning to convert raw video pixels into an ethogram, the behaviors-of-interest present in each video frame. DeepEthogram is designed to be general-purpose and applicable across species, behaviors, and video-recording hardware. It uses convolutional neural networks to compute motion, extract features from motion and images, and classify features into behaviors. Behaviors are classified with above 90% accuracy on single frames in videos of mice and flies, matching expert-level human performance. DeepEthogram accurately predicts rare behaviors, requires little training data, and generalizes across subjects. A graphical interface allows beginning-to-end analysis without end-user programming. DeepEthogram's rapid, automatic, and reproducible labeling of researcher-defined behaviors-of-interest may accelerate and enhance supervised behavior analysis.

Researchers commonly acquire videos of animal behavior and quantify the prevalence of behaviors of interest to study nervous system function, the effects of gene mutations, and the efficacy of pharmacological therapies. This analysis is typically performed manually and is therefore immensely time consuming, often limited to a small number of behaviors, and variable across researchers. Here, we created DeepEthogram: software that takes raw pixel values of videos as input and uses machine learning to output an ethogram, the set of user-defined behaviors of interest present in each frame of a video. We used convolutional neural network models that compute motion in a video, extract features from motion and single frames, and classify these features into behaviors. These models classified behaviors with greater than 90% accuracy on single frames in videos of flies and mice, matching expert-level human performance. The models accurately predicted even extremely rare behaviors, required little training data, and generalized to new videos and subjects. DeepEthogram runs rapidly on common scientific computer hardware and has a graphical user interface that does not require programming by the end-user. We anticipate DeepEthogram will enable the rapid, automated, and reproducible assignment of behavior labels to every frame of a video, thus accelerating all those studies that quantify behaviors of interest.Code is available at: https://github.com/jbohnslav/deepethogram

The lack of sensitive and robust behavioral assessments of pain in preclinical models has been a major limitation for both pain research and the development of novel analgesics. Here, we demonstrate a novel data acquisition and analysis platform that provides automated, quantitative, and objective measures of naturalistic rodent behavior in an observer-independent and unbiased fashion. The technology records freely behaving mice, in the dark, over extended periods for continuous acquisition of 2 parallel video data streams: (1) near-infrared frustrated total internal reflection for detecting the degree, force, and timing of surface contact and (2) simultaneous ongoing video graphing of whole-body pose. Using machine vision and machine learning, we automatically extract and quantify behavioral features from these data to reveal moment-by-moment changes that capture the internal pain state of rodents in multiple pain models. We show that these voluntary pain-related behaviors are reversible by analgesics and that analgesia can be automatically and objectively differentiated from sedation. Finally, we used this approach to generate a paw luminance ratio measure that is sensitive in capturing dynamic mechanical hypersensitivity over a period and scalable for highthroughput preclinical analgesic efficacy assessment.

No abstract

Characterizing animal behavior requires methods to distill 3D movements from video data. Though keypoint tracking has emerged as a widely used solution to this problem, it only provides a limited view of pose, reducing the body of an animal to a sparse set of experimenter-defined points. To more completely capture 3D pose, recent studies have fit 3D mesh models to subjects in image and video data. However, despite the importance of mice as a model organism in neuroscience research, these methods have not been applied to the 3D reconstruction of mouse behavior. Here, we present ArMo, an articulated mesh model of the laboratory mouse, and demonstrate its application to multi-camera recordings of head-fixed mice running on a spherical treadmill. Using an end-to-end gradient based optimization procedure, we fit the shape and pose of a dense 3D mouse model to data-derived keypoint and point cloud observations. The resulting reconstructions capture the shape of the animal's surface while compactly summarizing its movements as a time series of 3D skeletal joint angles. ArMo therefore provides a novel alternative to the sparse representations of pose more commonly used in neuroscience research.

No abstract