Three-dimensional tracking and behaviour monitoring of multiple fruit flies

Abstract

The increasing interest in the investigation of social behaviours of a group of animals has heightened the need for developing tools that provide robust quantitative data. Drosophila melanogaster has emerged as an attractive model for behavioural analysis; however, there are still limited ways to monitor fly behaviour in a quantitative manner. To study social behaviour of a group of flies, acquiring the position of each individual over time is crucial. There are several studies that have tried to solve this problem and make this data acquisition automated. However, none of these studies has addressed the problem of keeping track of flies for a long period of time in three-dimensional space. Recently, we have developed an approach that enables us to detect and keep track of multiple flies in a three-dimensional arena for a long period of time, using multiple synchronized and calibrated cameras. After detecting flies in each view, correspondence between views is established using a novel approach we call the 'sequential Hungarian algorithm'. Subsequently, the three-dimensional positions of flies in space are reconstructed. We use the Hungarian algorithm and Kalman filter together for data association and tracking. We evaluated rigorously the system's performance for tracking and behaviour detection in multiple experiments, using from one to seven flies. Overall, this system presents a powerful new method for studying complex social interactions in a three-dimensional environment.

Machine vision methods for analyzing social interactions

“…In such cases, the ambiguity caused by the occlusions can be resolved with tracking software that uses innate or artificial identifiers to maintain identity at least before and after such events (see 'Marking individuals' and 'Tracking' sections below). Three-dimensional (3D) tracking can also disambiguate occlusions by providing multiple views (Ardekani et al, 2013;Hong et al, 2015), but is a more technically challenging solution requiring additional equipment, synchronization and registration of cameras, and 3D tracking software.…”

Targeted manipulation of neuronal activity in behaving adult flies

“…A chamber with a low ceiling, narrow walls, and a "dead end" has been designed that prevents flies from turning around, thus forcing them to walk backwards (Bidaye et al 2014). In contrast, chambers can be designed with high ceilings to permit flight (Reynolds and Frye 2007;Straw et al 2011;Ardekani et al 2013;van Breugel and Dickinson 2014).…”

“…Chamber height determines the vertical space in which flies can move, and different heights require different methods to ensure that flies are visible at all times. For chambers with high ceilings, methods have been developed for tracking flies in threedimensional space through the use of multiple cameras and computational reconstruction of flight trajectories (Straw et al 2011;Ardekani et al 2013;van Breugel and Dickinson 2014). For non-flight behaviors, one camera can be used in conjunction with a low ceiling to restrict flies to the focal length of the camera lens.…”

“…Therefore, machine vision-based behavioral tracking algorithms have been developed to improve annotation consistency, throughput rate, and quantitative analysis of behavior (Anderson and Perona 2014;Egnor and Branson 2016). Such algorithms track multiple statistics of the trajectories of flies and/or their body parts through time (e.g., translational speed, angular speed, or distance to another fly), which can be used to define classifiers for particular behaviors (Dankert et al 2009;Branson et al 2009;Robie et al 2010;Straw et al 2011;Tsai and Huang 2012;Donelson et al 2012;Schusterreiter and Grossmann 2013;Ardekani et al 2013;Bidaye et al 2014;Dell et al 2014;Berman et al 2014). These statistics can also be fed into supervised machine-learning algorithms, such as the Janelia Automatic Animal Behavior Annotator (JAABA), where the researcher can train new behavior classifiers by manually annotating a small amount of video based on their own intuition about the behavior (Branson et al 2009;Kabra et al 2013).…”

Determination of the Spontaneous Locomotor Activity in <em>Drosophila melanogaster</em>

“…Flies in this protocol tend to exhibit cylindrical flight patterns, which is potentially due to the shape of Drosophila culture vials 10 . This method was improved by using a dome that allows measuring spontaneous movement of two flies 11 . A highthroughput method that uses a camera to automatically monitor and quantify the individual and social behavior of Drosophila has been also described 12 .…”