A Kinect-based system for automatic recording of some pigeon behaviors

Abstract

Contact switches and touch screens are the state of the art for recording pigeons' pecking behavior. Recording other behavior, however, requires a different sensor for each behavior, and some behaviors cannot easily be recorded. We present a flexible and inexpensive image-based approach to detecting and counting pigeon behaviors that is based on the Kinect sensor from Microsoft. Although the system is as easy to set up and use as the standard approaches, it is more flexible because it can record behaviors in addition to key pecking. In this article, we show how both the fast, fine motion of key pecking and the gross body activity of feeding can be measured. Five pigeons were trained to peck at a lighted contact switch, a pigeon key, to obtain food reward. The timing of the pecks and the food reward signals were recorded in a log file using standard equipment. The Kinect-based system, called BehaviorWatch, also measured the pecking and feeding behavior and generated a different log file. For key pecking, BehaviorWatch had an average sensitivity of 95 % and a precision of 91 %, which were very similar to the pecking measurements from the standard equipment. For detecting feeding activity, BehaviorWatch had a sensitivity of 95 % and a precision of 97 %. These results allow us to demonstrate that an advantage of the Kinect-based approach is that it can also be reliably used to measure activity other than key pecking.Keywords Kinect . Key pecking . Vision detecting behavior Recording pigeons' key pecking using a contact switch, a pigeon key, is common in studying animal learning and behavior. Such measurements only record whether or not a peck occurred at a specific time and at a general location. More recently, touch sensitive screens, which can provide a precise location of the peck, have become increasingly popular. Additional information could be collected if a video were used, including:1. the initiation time, speed of motion, head pose at start and end of pecks, pecks on the wall adjacent to key, and pecks initiated but not completed. 2. the global body and head pose: head position and orientation, body position, and foot position. 3. the subject behaviors relative to the stimulus area, turning around, flapping wings, and so forth.

Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning

“…We developed customized software to acquire raw depth images from both sensors. Although the Kinect sensor has been recently used for behavioral tracking in rats (31), pigeons (33), pigs (34), and human (35), we found that its spatial resolution was not sufficient for resolving pairs of mice, which are considerably smaller; in contrast, the Senz3D sensor's higher 3D resolution made it better suited for this application (Fig. S1).…”

Development of a new 3D tracking system for multiple marmosets under free-moving conditions

“…The animals were healthy throughout the experiment, proving the safety of our system. A similar 3D tracking system using Kinect senser or cameras has been reported for various animal species, including marmosets 1, 20, 23, 24, 30, 31 . However, none of the existing 3D tracking systems have achieved such a long-term analysis, whereas FulMAI is the only behavioral analysis system for longitudinal observation over each marmoset’s lifetime in the family.…”

“…A similar system, the Kinect system, has been reported for animals similar in size to marmosets, such as rats 30 and pigeons 31 , but the resolution of the Kinect system is not sufficient nor suitable for tracking fast-moving multiple animals 21 . When the Kinect system is used on multiple animals, a supplementary RFID is necessary 32 .…”

“…A similar 3D tracking system, the Kinect system, has been reported for marmosets 26 and similarly sized animals, such as rats 31 and pigeons 32 , but its resolution is neither sufficient nor suitable for tracking fast-moving 22 or multiple animals. Another animal behavioral analysis system, DeepLabCut, is a powerful tool for estimating the posture of an animal using only a camera, but its accuracy depends on the shutter speed of the camera 30, 33 . Although the images in this study were captured at 30 Hz, the subjects were blurred because the marmosets were very active and moved quickly.…”

Development of a new 3D tracking system for multiple marmosets under free-moving conditions

“…A similar 3D tracking system, the Kinect system, has been reported for marmosets 26 and similarly sized animals, such as rats 31 and pigeons 32 , but its resolution is neither su cient nor suitable for tracking fastmoving 22 or multiple animals. Another animal behavioral analysis system, DeepLabCut, is a powerful tool for estimating the posture of an animal using only a camera, but its accuracy depends on the shutter speed of the camera 30,33 . Although the images in this study were captured at 30 Hz, the subjects were blurred because the marmosets were very active and moved quickly.…”