An Integrated Framework for Field Recording, Localization, Classification and Annotation of Birdsongs Using Robot Audition Techniques — Harkbird 2.0

Sumitani, Shinji; Suzuki, Reiji; Chiba, Naoaki; Matsubayashi, Shiho; Arita, Takaya; Nakadai, Kazuhiro; Okuno, Hiroshi G.

doi:10.1109/icassp.2019.8683743

Cited by 9 publications

(12 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This runs on Ubuntu Linux in which HARK, HARK-Python, PySide, etc. are installed (see Nakadai et al (2017); Suzuki et al 2017; Sumitani et al (2019). The codes and documents are available from websites (http:// www.alife.cs.i.nagoya-u.ac.jp/˜reiji/HARKBird/) and GitHub (https://github.com/HARKBird-project/HARKBird/).…”

Section: Doa Estimation and Sound Source Separation Using Harkbirdmentioning

confidence: 99%

“…However, there have been few attempts so far. We are developing a portable system, HARKBird, to localize birdsongs Sumitani et al 2019). HARKBird can extract acoustic events in a recording, and we can obtain the begin and end timings, the spatial information (e.g., position or direction from the microphone array), and the spectrogram of the sound separated from the original recording.…”

Section: Introductionmentioning

confidence: 99%

“…More recently, we proposed a real-time 2D bird song localization system with two microphone arrays and showed that online localization was useful because appropriate locations to place microphone arrays based on the real-time localization results can be found (Sumitani et al 2018). An off-line analysis with manual annotations of vocalization types based on t-SNE (t-distributed Stochastic Neighbor Embedding (Van der Maaten and Hinton 2008)) (Sumitani et al 2019) could extract the spatial distribution of bird songs using a short recording in a preliminary playback experiment on a single individual of songbird (Sumitani et al 2018). However, it is still unclear whether such a portable and easily deployable system can be used to obtain sufficient information to observe the spatial dynamics of bird behaviors depending on ecological and biological conditions such as habitat and territorial structures.…”

Section: Introductionmentioning

confidence: 99%

“…An off‐line analysis with manual annotations of vocalization types based on t‐SNE (t‐distributed Stochastic Neighbor Embedding (Van der Maaten and Hinton 2008)) (Sumitani et al. 2019) could extract the spatial distribution of bird songs using a short recording in a preliminary playback experiment on a single individual of songbird (Sumitani et al. 2018).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Fine‐scale observations of spatio‐spectro‐temporal dynamics of bird vocalizations using robot audition techniques

Sumitani

Suzuki

Matsubayashi

et al. 2020

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

Ecoacoustics needs sophisticated acoustic monitoring tools to extract a wide level of features from an observed mixture of sounds. We have developed a portable acoustic monitoring system called 'HARKBird' which consists of a laptop PC and an inexpensive commercial microphone array with the robot audition software HARK. HARKBird can extract acoustic events in a recording, and we can obtain the begin and end timings, the spatial information (e.g., position or direction from the microphone array), and the spectrogram of the sound separated from the original recording. In this study, we report how robot audition techniques contribute to monitoring spatio-spectro-temporal dynamics of bird behaviors, using an extended and minimal system based on multiple microphone arrays. The dimension reduction of separated sounds is important to integrate the information from multiple microphone arrays. As a dimension reduction algorithm, we use t-SNE to help manual annotation of each sound and to generate the vocalization distribution automatically. We conduct playback experiments to Spotted Towhee (Pipilo maculatus) to simulate different cases of territorial intrusions (song/call/no playback). Our hypothesis in playback experiments is that playback of conspecific vocalizations would invoke aggressive responses of males against song playbacks and the effects would be more prominent than those of call playbacks. Our primary aim is to test whether our system can extract the necessary information on the aggressiveness of target individuals to examine our hypothesis. We show the system with manual annotation of vocalizations can extract their different spatio-spectro-temporal dynamics in different conditions, which supported our hypothesis. We also consider the spectral affinity-based automatic matching of localized sounds from different microphone arrays. The relative number of localized songs depending on the playback conditions reflected a similar trend to those in the manual approach, implying that we can grasp the long-term dynamics of vocalizations without costly annotations.

show abstract

Section: Doa Estimation and Sound Source Separation Using Harkbirdmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fine‐scale observations of spatio‐spectro‐temporal dynamics of bird vocalizations using robot audition techniques

Sumitani

Suzuki

Matsubayashi

et al. 2020

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

show abstract

“…The associate editor coordinating the review of this manuscript and approving it for publication was Jingchang Huang . signal-to-noise ratio of those collected recordings become a crucial issue, especially when building an automated robust bird sound classification system [14], [26], [29].…”

Section: Introductionmentioning

confidence: 99%

Investigation of Different CNN-Based Models for Improved Bird Sound Classification

Xie

Zhu

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Automatic bird sound classification plays an important role in monitoring and further protecting biodiversity. Recent advances in acoustic sensor networks and deep learning techniques provide a novel way for continuously monitoring birds. Previous studies have proposed various deep learning based classification frameworks for recognizing and classifying birds. In this study, we compare different classification models and selectively fuse them to further improve bird sound classification performance. Specifically, we not only use the same deep learning architecture with different inputs but also employ two different deep learning architectures for constructing the fused model. Three types of time-frequency representations (TFRs) of bird sounds are investigated aiming to characterize different acoustic components of birds: Mel-spectrogram, harmonic-component based spectrogram, and percussive-component based spectrogram. In addition to different TFRs, a different deep learning architecture, SubSpectralNet, is employed to classify bird sounds. Experimental results on classifying 43 bird species show that fusing selected deep learning models can effectively increase the classification performance. Our best fused model can achieve a balanced accuracy of 86.31% and a weighted F1-score of 93.31%. INDEX TERMS Bird sound classification, deep learning, class-based late fusion, time-frequency representation.

show abstract

Complex systems approaches to temporal soundspace partitioning in bird communities as a self-organizing phenomenon based on behavioral plasticity

Suzuki

Cody

2019

Artif Life Robotics

View full text Add to dashboard Cite

An Integrated Framework for Field Recording, Localization, Classification and Annotation of Birdsongs Using Robot Audition Techniques — Harkbird 2.0

Cited by 9 publications

References 14 publications

Fine‐scale observations of spatio‐spectro‐temporal dynamics of bird vocalizations using robot audition techniques

Fine‐scale observations of spatio‐spectro‐temporal dynamics of bird vocalizations using robot audition techniques

Investigation of Different CNN-Based Models for Improved Bird Sound Classification

Complex systems approaches to temporal soundspace partitioning in bird communities as a self-organizing phenomenon based on behavioral plasticity

Contact Info

Product

Resources

About