Context-based environmental audio event recognition for scene understanding

Abstract

To the best of our knowledge, this is the first work that models event correlations as scene context for robust audio event detection from complex and noisy environments. Note that according to the recent report, the mean accuracy for the acoustic scene classification task by human listeners is only around 71 % on the data collected in office environments from the DCASE dataset. None of the existing methods performs well on all scene categories and the average accuracy of the best performances of the recent 11 methods is 53.8 %. The proposed method averagely achieves an accuracy of 62.3 % on the same dataset. Additionally, we create a 10-CASE dataset by manually collecting 5,250 audio clips of 10 scene types and 21 event categories. Our experimental results on 10-CASE show that the proposed method averagely achieves the enhanced performance of 78.3 %, and the average accuracy of audio event recognition can be effectively improved by capturing dominant audio sources and reasoning non-dominant events from the dominant ones through acoustic context modeling. In the future work, exploring the interactions between acoustic scene recognition and audio event detection, and incorporating other modalities to improve the accuracy are required to further advance the proposed framework.

Automatic Audio Captioning using Attention weighted Event based Embeddings

“…Automatic Audio Captioning (AAC) is an inter-modal translation task, where the objective is to generate a textual description for a corresponding input audio signal [1]. Audio captioning is a critical step towards machine intelligence and many applications in daily scenarios, such as audio retrieval [2], scene understanding [3] [4], applications for the hearing impaired patients [5], detailed audio surveillance etc. Unlike an Automatic Speech Recognition (ASR) task, the output is a description rather than a transcription of the contents within the audio sample.…”

Text-to-Audio Grounding Based Novel Metric for Evaluating Audio Caption Similarity

“…Automatic Audio Captioning (AAC) is an inter-modal translation task, where the objective is to generate a textual description for a corresponding input audio signal [2]. Audio captioning is a critical step towards machine intelligence with multiple applications in daily scenarios, ranging from audio retrieval [3], scene understanding [4,5] to assist the hearing impaired [6] and audio surveillance. Unlike an Automatic Speech Recognition (ASR) task, the output is a description rather than a transcription of the linguistic content in the audio sample.…”