Joint Optimization of Deep Neural Network-Based Dereverberation and Beamforming for Sound Event Detection in Multi-Channel Environments

Noh, Kyoung Jin; Chang, Joon‐Hyuk

doi:10.3390/s20071883

Cited by 12 publications

(10 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, several sound events of short duration (e.g., "object impact" and "keyboard typing") have more than one audio pattern, such as attack, decay, and release parts. To control the training weight of the sound event model in accordance with the ease/difficulty of model training, the use of focal loss has been proposed [19,20]. In this paper, we newly introduce the following asymmetric focal loss (AFL), which enables the control of the focusing factor of active and inactive frames separately.…”

Section: Asymmetric Focal Loss (Afl)mentioning

confidence: 99%

Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance

Imoto

Mishima²,

Arai³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In many methods of sound event detection (SED), a segmented time frame is regarded as one data sample to model training. The durations of sound events greatly depend on the sound event class, e.g., the sound event "fan" has a long duration, whereas the sound event "mouse clicking" is instantaneous. Thus, the difference in the duration between sound event classes results in a serious data imbalance in SED. Moreover, most sound events tend to occur occasionally; therefore, there are many more inactive time frames of sound events than active frames. This also causes a severe data imbalance between active and inactive frames. In this paper, we investigate the impact of sound duration and inactive frames on SED performance by introducing four loss functions, such as simple reweighting loss, inverse frequency loss, asymmetric focal loss, and focal batch Tversky loss. Then, we provide insights into how we tackle this imbalance problem.

show abstract

Section: Asymmetric Focal Loss (Afl)mentioning

confidence: 99%

Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance

Imoto

Mishima²,

Arai³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…However, the performances of TDNNs have not been investigated using sounds in different acoustic environments. To enhance the performance of sound event detection, other DNN-based acoustic techniques such as noise reduction [ 43 ], and dereverberation and beamforming [ 44 ] have been investigated. In our study for AER and ICL, the effectiveness of transfer learning was demonstrated using TDNNs pre-trained on the sounds of AudioSet to extract the embedding of acoustic events.…”

Section: Introductionmentioning

confidence: 99%

An Incremental Class-Learning Approach with Acoustic Novelty Detection for Acoustic Event Recognition

Bayram

İnce

2021

Sensors

View full text Add to dashboard Cite

Acoustic scene analysis (ASA) relies on the dynamic sensing and understanding of stationary and non-stationary sounds from various events, background noises and human actions with objects. However, the spatio-temporal nature of the sound signals may not be stationary, and novel events may exist that eventually deteriorate the performance of the analysis. In this study, a self-learning-based ASA for acoustic event recognition (AER) is presented to detect and incrementally learn novel acoustic events by tackling catastrophic forgetting. The proposed ASA framework comprises six elements: (1) raw acoustic signal pre-processing, (2) low-level and deep audio feature extraction, (3) acoustic novelty detection (AND), (4) acoustic signal augmentations, (5) incremental class-learning (ICL) (of the audio features of the novel events) and (6) AER. The self-learning on different types of audio features extracted from the acoustic signals of various events occurs without human supervision. For the extraction of deep audio representations, in addition to visual geometry group (VGG) and residual neural network (ResNet), time-delay neural network (TDNN) and TDNN based long short-term memory (TDNN–LSTM) networks are pre-trained using a large-scale audio dataset, Google AudioSet. The performances of ICL with AND using Mel-spectrograms, and deep features with TDNNs, VGG, and ResNet from the Mel-spectrograms are validated on benchmark audio datasets such as ESC-10, ESC-50, UrbanSound8K (US8K), and an audio dataset collected by the authors in a real domestic environment.

show abstract

“…Both approaches are mainly based on estimating the time difference of arrival (TDOA) obtained by using various configurations of microphone arrays, such as linear array [12], circular array [13], or distributed array [14] and different cross-correlation algorithm to estimate time lag between microphones. The first approach aims to maximize the steered response power (SRP) of the output of a delay and sum beamformer [15]. This direct approach performs an exhaustive search in the whole SRP space to find a sound location, which is found to be computationally expensive.…”

Section: Introductionmentioning

confidence: 99%

Free-Field TDOA-AOA Sound Source Localization Using Three Soundfield Microphones

et al. 2020

View full text Add to dashboard Cite

Accurate source localization is an important problem in many research areas as well as practical applications in wireless communications and acoustic signal processing. This paper presents a passive three-dimensional sound source localization (SSL) method that employs a geometric configuration of three soundfield microphones. Two methods for estimating the angle of arrival (AOA) and time difference of arrival (TDOA) are proposed based on Ambisonics A and B format signals. The closed-form solution for sound source location estimation based on two TDOAs and three AOAs is derived. The proposed method is evaluated by simulations and physical experiments in our anechoic chamber. Simulations demonstrate that the estimation method can theoretically obtain Cramér-Rao lower bound for a small Gaussian noise present in AOA and TDOA observations. Investigation on the uncertainty of TDOA and AOA measurements depending on the length of measurement interval is also conducted. Experimental results in terms of RMSE indicate that the proposed solution can be used to accurately find a 3D position of the sound source in free-field environment. Performance evaluation regarding the number of estimation steps shows that higher accuracy can be achieved by longer observations of stationary sound source. INDEX TERMS 3D sound source localization, angle of arrival, cramér-rao bound, soundfiled microphone, time difference of arrival.

show abstract

Joint Optimization of Deep Neural Network-Based Dereverberation and Beamforming for Sound Event Detection in Multi-Channel Environments

Cited by 12 publications

References 34 publications

Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance

Impact of Sound Duration and Inactive Frames on Sound Event Detection Performance

An Incremental Class-Learning Approach with Acoustic Novelty Detection for Acoustic Event Recognition

Free-Field TDOA-AOA Sound Source Localization Using Three Soundfield Microphones

Contact Info

Product

Resources

About