Using intrinsic and contextual information associated with automated signal detections to improve call recognizer performance: A case study using the cryptic and critically endangered Night Parrot Pezoporus occidentalis

Abstract

1. Rapid expansion in the collection of large acoustic datasets to answer ecological questions has generated a parallel requirement for techniques that streamline analysis of these datasets. In many cases, automated signal recognition algorithms, often termed 'call recognizers', are the only feasible option for doing this. To date, most research has focused on what types of recognizers perform best, and how to train these recognizers to optimize performance. 2. We demonstrate that once recognizer construction is complete and the data processed, further improvements are possible using intrinsic and contextual information associated with each detection. We initially construct a call recognizer for the Night Parrot Pezoporus occidentalis using the r package monitor, and scan a test dataset. We then examine a number of intrinsic variables associated with each detection generated by the recognizer, and several contextual variables associated with the species' environment and ecology, to determine if they might help predict whether a given detection is a true positive (target signal) or false positive (nontarget signal). We test several logistic regression models incorporating different combinations of intrinsic and contextual variables, selecting the best-performing model for application. We train the model, using it to calculate the probability each detection is a true or false positive. 3. Substituting this model-derived probability for raw recognizer score improved the recognizer's performance, reducing the number of detections requiring proofing by 60% to achieve a recall of 90%, and by 76% to achieve a recall of 75%. 4. This technique is applicable to any recognizer output, regardless of the underlying algorithm. Application requires an understanding of how the recognizer algorithm determines matches, and knowledge of a species' ecology and environment. Because advanced programming skills and expertise are not required to apply this technique, it will be particularly relevant to field ecologists for whom building and operating call recognizers is an element of their research toolbox, but not necessarily a focus. | 1521 Methods in Ecology and Evoluঞon LESEBERG Et aL. How to cite this article: Leseberg NP, Venables WN, Murphy SA, Watson JEM. Using intrinsic and contextual information associated with automated signal detections to improve call recognizer performance: A case study using the cryptic and critically endangered Night Parrot Pezoporus occidentalis.

Accounting for both automated recording unit detection space and signal recognition performance in acoustic surveys: A protocol applied to the cryptic and critically endangered Night Parrot (Pezoporus occidentalis)

“…To assess whether the relationships are similar, we grouped the different call types into three broad categories of call type: 'bell-like', 'short whistle' and 'long whistle'. Previous research has demonstrated consistent relationships between score and call type within these groups (Leseberg et al 2020). The call type data set contained 406 bell-like calls, 96 long whistle calls and 483 short whistle calls.…”

“…Furthermore, the large data sets collected require a call recogniser to extract potential Night Parrot calls. While statistics around the precision and recall of one Night Parrot call recogniser have been established (Leseberg et al 2020), how these relate to the detection radius of the ARUs has not been investigated.…”

“…This process resulted in 31 different templates (Table 1), which were combined into a template list (Katz et al 2016) and represent the final 'recogniser'. This recogniser was able to achieve maximum recall of 0.94, with an area under the curve (AUC) of 0.76 (Leseberg et al 2020).…”

An evaluation of acoustic field recorders paired with automated call recognition as a monitoring tool for the Mallee Emu-wren Stipiturus mallee

“…Context is important when evaluating recogniser performance (Knight et al 2017;Leseberg et al 2020). Performance metrics should be considered reliable only under the environmental conditions in which they were F-score is a performance metric for automated signal detection that allows the user to prioritise precision and/or recall.…”

A Convolutional Neural Network Bird Species Recognizer Built From Little Data by Iteratively Training, Detecting, and Labeling

“…While this decreases the total work somewhat, there is still a large time burden involved with listening to the audio. Passive acoustic monitoring is increasingly being applied to monitor Australian birds particularly in conservation contexts (e.g., Leseberg et al, 2020;Teixeira et al, 2021).…”