Estimation of skeletal kinematics in freely moving rodents

Abstract

Forming a complete picture of the relationship between neural activity and skeletal kinematics requires quantification of skeletal joint biomechanics during free behavior; however, without detailed knowledge of the underlying skeletal motion, inferring limb kinematics using surface-tracking approaches is difficult, especially for animals where the relationship between the surface and underlying skeleton changes during motion. Here we developed a videography-based method enabling detailed three-dimensional kinematic quantification of an anatomically defined skeleton in untethered freely behaving rats and mice. This skeleton-based model was constrained using anatomical principles and joint motion limits and provided skeletal pose estimates for a range of body sizes, even when limbs were occluded. Model-inferred limb positions and joint kinematics during gait and gap-crossing behaviors were verified by direct measurement of either limb placement or limb kinematics using inertial measurement units. Together we show that complex decision-making behaviors can be accurately reconstructed at the level of skeletal kinematics using our anatomically constrained model.

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

“…V1, shows that the network confuses between the left and right top front paws. This example is in line with the recent observation that network predictions need to be extensively postprocessed to better match ground-truth kinematics (Karashchuk et al, 2021;Monsees et al, 2022). Simpler post-processing approaches drop low-confidence frames and interpolate over them with a polynomial (The International Brain Laboratory, 2023) or an autoregressive integrated moving average (ARIMA) model (Nath et al, 2019).…”

“…Simpler post-processing approaches drop low-confidence frames and interpolate over them with a polynomial (The International Brain Laboratory, 2023) or an autoregressive integrated moving average (ARIMA) model (Nath et al, 2019). More complex post-processing schemes rely on specific body models and require expensive optimization or sampling techniques (Biderman et al, 2020;Joska et al, 2021;Zhang et al, 2021;Monsees et al, 2022), limiting their general applicability. We discuss these further in 5.9.7.…”

“…To start, the labeling process can be laborious, especially when labeling complicated skeletons on multiple views. Even with large labeled datasets, trained networks are often unreliable: they output "glitchy" predictions that require further manipulation before downstream analyses (Karashchuk et al, 2021;Monsees et al, 2022), and struggle to generalize to animals and sessions that were not represented in their labeled training set (Gonschorek et al, 2021). Even well-trained networks that achieve low pixel error on a small number of labeled test frames can still produce a sufficient fraction of error frames that hinder downstream scientific tasks.…”

ArMo: An Articulated Mesh Approach for Mouse 3D Reconstruction

“…For example, GIMBAL instantiates spatiotemporal priors on the overall pose of the animal via a hierarchical von Mises-Fisher-Gaussian model, which results in better inference of 3D keypoint positions than naive triangulation (Zhang et al 2021). Another approach called the ACM (anatomically constrained model) uses Kalman smoothing and pose priors to accurately reconstruct 3D skeletal kinematics in freely behaving rodents over a wide range of sizes (Monsees et al 2022). Other methods have fit simplified body models to rodents.…”

Chasing cortical behavior: designing multiphoton microscopes for imaging neuronal populations in freely moving rodents

“…SiPMs are intrinsically resilient to ambient light exposure, unlike with the PMTs that are commonly used on conventional multiphoton microscopes that can be permanently damaged from ambient light exposure. SiPMs can be used in behavioral arenas requiring ambient light 32 for behavioral pose tracking, 40 eye tracking 3 and visually based prey capture behaviors. In addition, their fast response currents also allow for fast temporal gating, clocked at the excitation laser pulse rate, for optical noise rejection [Fig.…”