Multi-Level Adaptive Speech Activity Detector for Speech in Naturalistic Environments

Abstract

Speech activity detection (SAD) is a part of many speech processing applications. The traditional SAD approaches use signal energy as the evidence to identify the speech regions. However, such methods perform poorly under uncontrolled environments. In this work, we propose a novel SAD approach using a multi-level decision with signal knowledge in an adaptive manner. The multi-level evidence considered are modulation spectrum and smoothed Hilbert envelope of linear prediction (LP) residual. Modulation spectrum has compelling parallels to the dynamics of speech production and captures information only for the speech component. Contrarily, Hilbert envelope of LP residual captures excitation source aspect of speech. Under uncontrolled scenario, these evidence are found to be robust towards the signal distortions and thus expected to work well. In view of different levels of interference present in the signal, we propose to use a quality factor to control the speech/nonspeech decision in an adaptive manner. We refer this method as multi-level adaptive SAD and evaluate on Fearless Steps corpus that is collected during Apollo-11 Mission in naturalistic environments. We achieve a detection cost function of 7.35% with the proposed multi-level adaptive SAD on the evaluation set of Fearless Steps 2019 challenge corpus.

Circumspection in using automated measures: Talker gender and addressee affect error rates for adult speech detection in the Language ENvironment Analysis (LENA) system

“…Clearly, lower DCF indicates better classification performance. Two recent studies found DCF values of 7.4% (Sharma, Das, & Li, 2019) and 11.7% (Hansen, Joglekar, Shekhar, Kothapally, Yu, Kaushik, & Sangwan, 2019) (using a = 0.25 and b = 0.75) in S/VAD for recordings from the Apollo 11 space travel mission (Kaushik et al, 2018). Another study (Dubey, Sangwan, & Hansen, 2018) evaluated algorithms using a corpus of noisy recordings from degraded military communication channels and reported DCF values (also with a = 0.25, b = 0.75) ranging from 4.3% to 8.9% (mean = 6.1%) across five novel algorithms (averaging across degraded channel conditions), where this constituted comparable performance in relation to baseline algorithms (e.g., Sholokhov, Sahidullah, & Kinnunen, 2018).…”

FEARLESS STEPS Challenge (FS-2): Supervised Learning with Massive Naturalistic Apollo Data

“…While this was similar to the 116 system submissions received for FS-1 challenge, participation for both tracks of SD and ASR tasks was noticeably higher. The systems developed for Figure 4: rVad-SincNet based SID baseline system [34,35] FS-2 also exhibited vast improvements in performance compared to the best systems developed for FS-1 challenge [2,11,12,13,15], as seen in Table-5. We observed relative improvements of 67%, 57%, and 62% for SAD, Speaker Diarization from scratch, and Speech Recognition from audio streams tasks respectively.…”

“…This began with the Inaugural FEARLESS STEPS Challenge: Massive Naturalistic Audio (FS-1). The first edition of this challenge encouraged the development of core unsupervised/semi-supervised speech and language systems for single-channel data with low resource availability, serving as the First Step towards extracting high-level information from such massive unlabeled corpora [11,12,13,14,15]. As a natural progression following the successful inaugural FS-1 challenge, the FEARLESS STEPS Challenge Phase-2 (FS-2) focuses on the development of single-channel supervised learning strategies.…”

SLoClas: A Database for Joint Sound Localization and Classification

“…Sound Localization and Classification (SLC) refers to estimating the spatial location of a sound source and identifying the type of a sound event through a unified framework. A SLC method enables the autonomous robots to determine sound location and detect sound events for navigation and interaction with the surroundings [1,2]. Thus, SLC is useful in smart-city and smart-home applications to automatically specify social or human activities, and assist the hearing impaired to visualize and realize sounds [3,4,5,6].…”