Multi-channel Acoustic Modeling using Mixed Bitrate OPUS Compression

Abstract

Recent literature has shown that a learned front end with multi-channel audio input can outperform traditional beamforming algorithms for automatic speech recognition (ASR). In this paper, we present our study on multi-channel acoustic modeling using OPUS compression with different bitrates for the different channels. We analyze the degradation in word error rate (WER) as a function of the audio encoding bitrate and show that the WER degrades by 12.6% relative with 16kpbs as compared to uncompressed audio. We show that its always preferable to have a multi-channel audio input over a single channel audio input given limited bandwidth. Our results show that for the best WER, when one of the two channels can be encoded with a bitrate higher than 32kbps, its optimal to encode the other channel with the highest bitrate possible. For bitrates lower than that, its preferable to distribute the bitrate equally between the two channels. We further show that by training the acoustic model on mixed bitrate input, upto 50% of the degradation can be recovered using a single model.

Multi-Channel Opus Compression for Far-Field Automatic Speech Recognition with a Fixed Bitrate Budget

“…While Opus has been found to be effective for single-and multi-channel ASR applications [11], it has been optimized for subjective quality, not for ASR accuracy. Moreover, its channel coupling has been optimized for human spatial perception, where the inter-aural phase difference (IPD) [20] is only noticeable at lower frequencies.…”

“…We here focus on Opus compression [9] due to its widespread use in voice over IP (VoIP) and ASR applications [10]. While Opus has been found to be effective for singleand multi-channel ASR applications [11], it is in some sense mismatched to the task at hand. First, like all lossy speech and audio compression methods, Opus has been optimized for subjective quality, not for ASR or subsequent spatial filtering.…”

“…With focus on ASR applications, [14] analyzes the impact of different compression algorithms on a single-channel acoustic model (AM) and concludes that using a compression algorithms as a data augmentation step in the training phase of an AM can lead to a more robust model. Closely related to the document at hand, Khare et al analyzed the impact of Opus compression on multi-channel AMs [11] with the main findings that multi-channel data is preferable over single-channel transmission with a higher bitrate. They further report gains when retraining the AM, which we do not repeat here.…”