Open-Source Practices for Music Signal Processing Research: Recommendations for Transparent, Sustainable, and Reproducible Audio Research

McFee, Brian; Kim, Jong Wook; Cartwright, Mark; Salamon, Justin; Bittner, Rachel M.; Bello, Juan Pablo

doi:10.1109/msp.2018.2875349

Cited by 34 publications

(26 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Software based automation in acoustic analysis is an important step in ensuring reproducible results, which in turns spurs the growth of the research field (McFee et al 2018;Baker and Vincent 2019). The itsfm package written in the Python (Van Rossum and Drake Jr 1995) language is an open-sourced method which may be used in the analysis of animal vocalisations such as CF-FM bat calls, and other vocalisations.…”

Section: Discussionmentioning

confidence: 99%

“…While code descriptions help explaining the principles behind design, it is not sufficient to ensure uniformity or correctness in implementation. Differences in implementation may lead to differences in scientific results (Baker and Vincent 2019; McFee et al 2018). Publishing code as publicly available packages allows for external code inspection and improvements.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

itsfm, an open-source package to reliably segment and measure sounds by frequency modulation

Beleyur

2021

Preprint

View full text Add to dashboard Cite

Analysing animal vocalisations in detail provides insights into the biomechanics, decision making and sensory processes behind their behaviours. Echolocating bats, and in particular, the CF-FM calls of high-duty cycle bats serve as a convenient model system to illustrate this point. The CF component in the CF-FM call is used for prey detection and the FM component is used in target ranging. According to the behavioural context at hand such as flight with conspecifics or prey capture, bats choose to increase the duration, intensity or spectral range of the components differently. Studying the call component alterations requires an objective methodology that first segments the components and then allows measurements on them. Studies till now have segmented the call components manually, or automatically using what I term the ‘peak-frequency’ method. Manual segmentation is error prone, while the ‘peak-frequency’ method requires on-axis recordings for good results. Despite multiple papers using a peak-frequency based segmentation, there remain no publicly available software implementations. itsfm is an open-source package that fills this gap with two implemntations that can segment CF-FM calls, one of them being an implementation of the peak-percentage method. itsfm additionally introduces the ‘pseudo-Wigner-Ville distribution’ (PWVD) method for call segmentation, thus allowing the segmentation of calls captured under a wider variety of recording conditions. I create a synthetic dataset and assess the performance of the PWVD method and the ‘peak-frequency’ method. The PWVD performs consistently well in call component segmentation in comparison to the peak-percentage method. I also discuss the supporting methods in the itsfm package that can help the further automatic segmentation, measurement and analysis of sounds. Though originally developed for the segmentation and measurement of CF-FM bat calls, the methods in itsfm are speciesagnostic, and may be used for vocalisations of any type.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

itsfm, an open-source package to reliably segment and measure sounds by frequency modulation

Beleyur

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…This interface is directed towards the avian bioacoustics community, in order to allow the large-scale deployment of autonomous recording units for flight call monitoring. In an effort of conducting transparent, sustainable, and reproducible audio research [71], BirdVoxDetect also comprises documentation, a test suite, a Python package indexation, and an interoperable application programming interface (API).…”

Section: Contributionsmentioning

confidence: 99%

Robust sound event detection in bioacoustic sensor networks

et al. 2019

Self Cite

View full text Add to dashboard Cite

Bioacoustic sensors, sometimes known as autonomous recording units (ARUs), can record sounds of wildlife over long periods of time in scalable and minimally invasive ways. Deriving per-species abundance estimates from these sensors requires detection, classification, and quantification of animal vocalizations as individual acoustic events. Yet, variability in ambient noise, both over time and across sensors, hinders the reliability of current automated systems for sound event detection (SED), such as convolutional neural networks (CNN) in the time-frequency domain. In this article, we develop, benchmark, and combine several machine listening techniques to improve the generalizability of SED models across heterogeneous acoustic environments. As a case study, we consider the problem of detecting avian flight calls from a ten-hour recording of nocturnal bird migration, recorded by a network of six ARUs in the presence of heterogeneous background noise. Starting from a CNN yielding state-of-the-art accuracy on this task, we introduce two noise adaptation techniques, respectively integrating short-term (60 ms) and long-term (30 min) context. First, we apply per-channel energy normalization (PCEN) in the time-frequency domain, which applies short-term automatic gain control to every subband in the mel-frequency spectrogram. Secondly, we replace the last dense layer in the network by a context-adaptive neural network (CA-NN) layer, i.e. an affine layer whose weights are dynamically adapted at prediction time by an auxiliary network taking long-term summary statistics of spectrotemporal features as input. We show that PCEN reduces temporal overfitting across dawn vs. dusk audio clips whereas context adaptation on PCEN-based summary statistics reduces spatial overfitting across sensor locations. Moreover, combining them yields state-of-the-art results that are unmatched by artificial data augmentation alone. We release a pre-trained version of our best performing system under the name of BirdVoxDetect, a ready-to-use detector of avian flight calls in field recordings.

show abstract

“…We apply log amplitude scaling and normalize resultant mel spectrograms to have 120dB dynamic range. Precisely recreating this representation [14] is simple in our codebase. 3…”

Section: Extractionmentioning

confidence: 99%

Expediting TTS Synthesis with Adversarial Vocoding

et al. 2019

View full text Add to dashboard Cite

Recent approaches in text-to-speech (TTS) synthesis employ neural network strategies to vocode perceptually-informed spectrogram representations directly into listenable waveforms. Such vocoding procedures create a computational bottleneck in modern TTS pipelines. We propose an alternative approach which utilizes generative adversarial networks (GANs) to learn mappings from perceptually-informed spectrograms to simple magnitude spectrograms which can be heuristically vocoded. Through a user study, we show that our approach significantly outperforms naïve vocoding strategies while being hundreds of times faster than neural network vocoders used in state-of-theart TTS systems. We also show that our method can be used to achieve state-of-the-art results in unsupervised synthesis of individual words of speech.

show abstract

Open-Source Practices for Music Signal Processing Research: Recommendations for Transparent, Sustainable, and Reproducible Audio Research

Cited by 34 publications

References 14 publications

itsfm, an open-source package to reliably segment and measure sounds by frequency modulation

itsfm, an open-source package to reliably segment and measure sounds by frequency modulation

Robust sound event detection in bioacoustic sensor networks

Expediting TTS Synthesis with Adversarial Vocoding

Contact Info

Product

Resources

About