Automated bird sound classifications of long-duration recordings produce occupancy model outputs similar to manually annotated data

Abstract

Occupancy modeling is used to evaluate avian distributions and habitat associations, yet it typically requires extensive survey effort because a minimum of 3 repeat samples are required for accurate parameter estimation. Autonomous recording units (ARUs) can reduce the need for surveyors on-site, yet their utility was limited by hardware costs and the time required to manually annotate recordings. Software that identifies bird vocalizations may reduce the expert time needed if classification is sufficiently accurate. We assessed the performance of BirdNET—an automated classifier capable of identifying vocalizations from >900 North American and European bird species—by comparing automated to manual annotations of recordings of 13 breeding bird species collected in northwestern California. We compared the parameter estimates of occupancy models evaluating habitat associations supplied with manually annotated data (9-min recording segments) to output from models supplied with BirdNET detections. We used 3 sets of BirdNET output to evaluate the duration of automatic annotation needed to approach manually annotated model parameter estimates: 9-min, 87-min, and 87-min of high-confidence detections. We incorporated 100 3-s manually validated BirdNET detections per species to estimate true and false positive rates within an occupancy model. BirdNET correctly identified 90% and 65% of the bird species a human detected when data were restricted to detections exceeding a low or high confidence score threshold, respectively. Occupancy estimates, including habitat associations, were similar regardless of method. Precision (proportion of true positives to all detections) was >0.70 for 9 of 13 species, and a low of 0.29. However, processing of longer recordings was needed to rival manually annotated data. We conclude that BirdNET is suitable for annotating multispecies recordings for occupancy modeling when extended recording durations are used. Together, ARUs and BirdNET may benefit monitoring and, ultimately, conservation of bird populations by greatly increasing monitoring opportunities.

Low-cost open-source recorders and ready-to-use machine learning approaches provide effective monitoring of threatened species

“…However, the recall rate obtained using BirdNET is among the highest values ever published using this technique. For example, Cole et al (2022) recently evaluated BirdNET for detecting 13 bird species in North America and found that the recall rate ranged from 9% to 68%. Similarly, the recall rate obtained by Tolkova et al (2021) ranged between 11% and 71% for three common bird species.…”

“…Some of the state-of-the-art techniques for handling big datasets, such as deep learning and convolutional neural networks (Stowell, 2022;Stowell et al, 2019), can be difficult to run for ornithologists, managers, and researchers without bioacoustics or engineering backgrounds. However, user-friendly and ready-to-use machine learning approaches have recently been developed and are increasingly accessible to respond to real-life monitoring challenges and the general public (Cole et al, 2022). Among these approaches is Bird-NET, a research project between The Cornell Lab of Ornithology and the Chemnitz University of Technology.…”

“…Likewise, BirdNET's ability to identify bird vocalizations largely differs among species. For example, Cole et al (2022) found that the proportion of annotated calls correctly identified by BirdNET may differ from 9% for the Mourning dove (Zenaida macroura) to 68% for the California Quail (Callipepla californica). There are also off-the-shelf commercially available programs like Kaleidoscope Pro (300$ annual license, Wildlife Acoustics, USA) that can be used to detect and to group candidate sounds into clusters using Hidden Markov Models (Abrahams and Geary, 2020;.…”

BirdNET: applications, performance, pitfalls and future opportunities

“…In addition to choosing the confidence score threshold, BirdNET also allows users to adjust the overlap of prediction segments (overlapping segments will increase the detection resolution but also the processing time; Kahl et al 2021) and the sensitivity parameter (higher values may work better in speciesrich soundscapes; Kahl et al 2021), as well as to apply spatial and temporal filters (e.g. classify sounds only for species on eBird checklists for a certain location and/or period; Cole et al 2022) or specify a custom list to classify sounds only for target species (see Manzano et al 2022).…”

“…Indeed, several recent studies have employed BirdNET automatically to classify bird vocalizations and draw ecological or conservation inferences from the data collected (e.g. Cole et al 2022, Wood et al 2022. Bird-NET precision (% detections correctly classified) was claimed to be about 0.8-0.9 in focal recordings (Arif et al 2020 but it seems to decrease considerably for sound recordings collected using omnidirectional microphones (Wood et al 2021).…”