Wide-Angle, Monocular Head Tracking using Passive Markers

Abstract

Camera images can encode large amounts of visual information of an animal and its environment, enabling high fidelity 3D reconstruction of the animal and its environment using computer vision methods. Most systems, both markerless (e.g. deep learning based) and marker-based, require multiple cameras to track features across multiple points of view to enable such 3D reconstruction. However, such systems can be expensive and are challenging to set up in small animal research apparatuses. We present an open-source, marker-based system for tracking the head of a rodent for behavioral research that requires only a single camera with a potentially wide field of view. The system features a lightweight visual target and computer vision algorithms that together enable high-accuracy tracking of the six-degree-of-freedom position and orientation of the animal's head. The system, which only requires a single camera positioned above the behavioral arena, robustly reconstructs the pose over a wide range of head angles (360 degrees in yaw, and approximately +/-120 degrees in roll and pitch). Experiments with live animals demonstrate that the system can reliably identifyrat head position and orientation. Evaluations using a commercial optical tracker device show that the system achieves accuracy that rivals commercial multi-camera systems. Our solution significantly improves upon existing monocular marker-based tracking methods, both in accuracy and in allowable range of motion. The proposed system enables the study of complex behaviors by providing robust, fine-scale measurements of rodent head motions in a wide range of orientations.

Closed-loop control and recalibration of place cells by optic flow

“…For experimental sessions, instead of the single large marker attached to the body harness, a set of smaller (3mm, 4 mm) markers were placed in a rigid arrangement around the recording headstage. This allowed our custom algorithm to track the 3D position and orientation of the constellation of markers with higher accuracy and robustness 49 . Thus, the rat did not need to wear the harness during sessions.…”

“…A near-infrared camera (GS3-U3-41C6NIR-C, FLIR, OR, USA; 2048x2048 px, 45 fps) with a wide-field lens (NMV-6M1, 6mm, F1.8, Navitar, NY, USA) provided an overhead view of the experiment. The 3D position and orientation of the head of the rat were detected in real-time (45 fps) through single-camera tracking 49 of a set of markers mounted on the rat's neural recording implant. The central rotating pillar of the dome was mounted on bearings.…”

The Dome: A virtual reality apparatus for freely locomoting rodents

“…In order to prevent this, we hide the camera from the rat by placing it behind the ‘brim’ of the top hat. In the Current version, we track the head position of the rat in real-time using the camera [ 24 ]. To enable this, the neural recording headstage attached to the head of the rat supports a lightweight three-dimensional pattern of retro-reflective markers (the ‘crown’).…”

“…Arbitrary shapes can be displayed through OpenGL at high frame rates by the use of pre-compiled display lists. Computer 3 (Ubuntu 18.04) receives the camera signal through a USB3.0 interface and runs a custom-developed ROS node that tracks the position and orientation of the animal’s head [ 24 ]. In addition, it also records high-resolution video at full frame rate (45–90 fps) for post-experiment processing.…”

“…In the Current version, the three-dimensional head position of the rat was tracked in real time by a custom single-camera-based tracking algorithm. The tracking accuracy of the algorithm was validated with a industry-validated visual tracking system and reported in [ 24 ] (a full report of this tracking system is in preparation). Traditionally, camera-based tracking systems are slower than tracking systems based on sensors such as optical encoders.…”