Exploiting Parallel Audio Recordings to Enforce Device Invariance in CNN-based Acoustic Scene Classification

Abstract

Distribution mismatches between the data seen at training and at application time remain a major challenge in all application areas of machine learning. We study this problem in the context of machine listening (Task 1b of the DCASE 2019 Challenge). We propose a novel approach to learn domain-invariant classifiers in an end-to-end fashion by enforcing equal hidden layer representations for domain-parallel samples, i.e. time-aligned recordings from different recording devices. No classification labels are needed for our domain adaptation (DA) method, which makes the data collection process cheaper. We show that our method improves the target domain accuracy for both a toy dataset and an urban acoustic scenes dataset. We further compare our method to Maximum Mean Discrepancy-based DA and find it more robust to the choice of DA parameters. Our submission, based on this method, to DCASE 2019 Task 1b gave us the 4th place in the team ranking.

STAViS: Spatio-Temporal AudioVisual Saliency Network

“…The huge amounts of video data that are becoming more and more available through online sources, enable and at the same time require continuously better performance in tasks like activity recognition, video saliency, scene analysis or video summarization, imposing the need to exploit not only spatial information, but also temporal [7,47,27]. Similar advances have also been achieved in audio processing areas, such as acoustic event detection [44], speech recognition [17,9], sound localiza-tion [56], by using deep learning techniques.…”

Relational Teacher Student Learning with Neural Label Embedding for Device Adaptation in Acoustic Scene Classification

“…In particular, spectrum correction [23] and channel conversion [24] build a front-end module to convert speech features from the source domain to target domain before feeding them to the back-end classifier. Besides front-end features, mid-level feature based transfer systems, which uses bottleneck features [25] or hidden layer representations [26] are adopted to transfer knowledge from source to target domain. Adversarial training methods in [27,28] leverage an extra domain discriminator to solve the device mismatch problem although the key focus is on lack of labeled target data.…”