NeuroMechFly, a neuromechanical model of adult Drosophila melanogaster

Abstract

Animal behavior emerges from a seamless interaction between neural network dynamics, musculoskeletal properties, and the physical environment. Accessing and understanding the interplay between these intertwined elements requires the development of integrative and morphologically realistic neuromechanical simulations. Until now, there has been no such simulation framework for the widely studied model organism, Drosophila melanogaster. Here we present NeuroMech-Fly, a data-driven model of the adult female fly within a physics-based simulation environment.NeuroMechFly combines a series of independent computational modules including a biomechanical exoskeleton with articulating body parts-legs, halteres, wings, abdominal segments, head, proboscis, and antennae-muscle models, and neural network controllers. To enable illustrative use cases, we first define minimal leg degrees-of-freedom by analyzing real 3D kinematic measurements during real Drosophila walking and grooming. Then, we show how, by replaying these behaviors using NeuroMechFly's biomechanical exoskeleton in its physics-based simulation environment, one can predict otherwise unmeasured torques and contact reaction forces. Finally, we leverage NeuroMechFly's full neuromechanical capacity to discover neural networks and muscle parameters that enable locomotor gaits optimized for speed and stability. Thus, NeuroMech-Fly represents a powerful testbed for building an understanding of how behaviors emerge from interactions between complex neuromechanical systems and their physical surroundings.

Descending neuron population dynamics during odor-evoked and spontaneous limb-dependent behaviors

“…Joint angles from all time points and flies were pooled to generate kernel density estimates. The angles are defined as described in Lobato-Rios et al, 2022 . …”

Descending neuron population dynamics during odor-evoked and spontaneous limb-dependent behaviors

“…A seventh, front-facing camera recorded spherical treadmill rotations that could be converted into fictive locomotor trajectories using FicTrac [34] (Figure 1m). Multi-view camera images (Figure 1n) were post-processed using DeepFly3D [33] to estimate 3D joint positions [40] (Figure 1o). These data were used to train a dilated temporal convolutional neural network (DTCN) [41] that could accurately classify epochs of walking, resting, head (eye and antennal) grooming, front leg rubbing, and posterior (hindleg and abdominal) movements (Figure 1p, Figure S1g).…”

“…Specifically, while analyzing head grooming DNs, we noticed a large pair of ventral neurons (Figure 5a) that sometimes exhibited asymmetric activity (Figure 5b, gray arrowheads) when flies appeared to touch one rather than both antennae (Video 5). To quantify this observation, we replayed limb 3D kinematics in NeuroMechFly, a biomechanical simulation of Drosophila [40] (Figure 5c). By detecting leg-antennal collisions as a proxy for antenna deflection, we found that occasional asymmetries (Figure 5d) did coincide with asymmetric activity in corresponding neural data (Figure 5b, purple traces).…”

“…Briefly, we detected outliers based on changes in leg segment length and chose the pair of cameras with minimal reprojection error for triangulation. After outlier correction, the data were aligned and joint angles were computed using published code [40]: https://github.com/NeLy-EPFL/df3dPostProcessing/tree/outlier_ correction.…”

“…To infer limb-antennal collisions, we performed kinematic replay using the NeuroMechFly physics simulation framework as described in [40]. We used joint angles to replay real limb movements in the simulation.…”