Multimodal system for recording individual-level behaviors in songbird groups

Abstract

In longitudinal observations of animal groups, the goal is to identify individuals and to reliably detect their interactive behaviors including their vocalizations. However, to reliably extract individual vocalizations from their mixtures and other environmental sounds remains a serious challenge. Promising approaches are multi-modal systems that make use of animal-borne wireless sensors and that exploit the inherent signal redundancy. In this vein, we designed a modular recording system (BirdPark) that yields synchronized data streams and contains a custom software-defined radio receiver. We record pairs of songbirds with multiple cameras and microphones and record their body vibrations with custom low-power frequency-modulated (FM) radio transmitters. Our custom multi-antenna radio demodulation technique increases the signal-to-noise ratio of the received radio signals by 6 dB and reduces the signal loss rate by a factor of 87 to only 0.03% of the recording time compared to standard single-antenna demodulation techniques. Nevertheless, neither a single vibration channel nor a single sound channel is sufficient by itself to signal the complete vocal output of an individual, with each sensor modality missing on average about 3.7% of vocalizations. Our work emphasizes the need for high-quality recording systems and for multi-modal analysis of social behavior.

Balanced Deep CCA for Bird Vocalization Detection

“…All methods were carried out in accordance with relevant guidelines and regulations (Swiss Animal Welfare Act and Ordinance, TSchG, TSchV, TVV). Vibration transmitter devices were mounted on the back of the birds with a rubber band harness [13]. The vibration transducer (Knowles Bu-21771-000 accelerometer) on the transmitter functioned as a contact microphone and selectively recorded vocalizations of the bird that carried the transmitter.…”

“…Additionally the transmitter devices picks up movement signals from e.g. wing flaps and radio noises [13]. An additional, modified transmitter device, where the vibration transducer was replaced by a Knowles FG-23329-D65 microphone, was mounted on the wall of the homecage, it delivered the microphone signal.…”

TinyBird-ML: An ultra-low Power Smart Sensor Node for Bird Vocalization Analysis and Syllable Classification

“…Traditionally, animal-borne sensors are based on frequency modulation (FM) to transmit data [15], [16]. Although this approach can achieve minimal power consumption [17], there are well-known issues in terms of signal quality and interference when more than a single transmitter is in the same area [18]. Moreover, FM transmitters allow only one-way communication from the sensor to the receiver, and there is no way to send information back to the sensor node on the bird [16].…”