The CAPIO 2017 Conversational Speech Recognition System

Abstract

In this paper we show how we have achieved the state-of-the-art performance on the industry-standard NIST 2000 Hub5 English evaluation set. We propose densely connected LSTMs, (namely, dense LSTMs), inspired by the densely connected convolutional networks recently introduced for image classification tasks. It is shown that the proposed dense LSTMs would provide more reliable performances as compared to the conventional, residual LSTMs as more LSTM layers are stacked in neural networks. We also propose an acoustic model adaptation scheme that simply averages the parameters of a seed neural network acoustic model and its adapted version. This method was applied with the CallHome training corpus and improved individual system performances by on average 6.1% (relative) against the Call-Home portion of the evaluation set with no performance loss on the Switchboard portion. With RNN-LM rescoring and lattice combination on the 5 systems trained across three different phone sets, our 2017 speech recognition system has obtained 5.0% and 9.1% on Switchboard and CallHome, respectively, both of which are the best word error rates reported thus far. According to IBM in their latest work to compare human and machine transcriptions, our reported Switchboard word error rate can be considered to surpass the human parity (5.1%) of transcribing conversational telephone speech. We also share the performance numbers of our system on non-telephony environments for readers' benefits.

Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions

“…Table 1 compares the TDS model with three other systems. The CAPIO system is a hybrid HMM-DNN with speaker adaptation [33]. The other two are end-to-end models, one using the CRF-style ASG loss [31] and the other a sequence-to-sequence model with an RNN encoder [23].…”

Relative Positional Encoding for Speech Recognition and Direct Translation

“…The error rates on those SWB and CH decrease from 6.5 and 11.9 to 6.2 and 11.4 (Table 2). Our best model is significantly better than previously published CTC [29] and LSTM-based [3] models, and approaches the heavily tuned hybrid system [28] with dense TDNN-LSTM. It is likely possible to reach better error rates, with the help of ensembled models, further data augmentation, and language models.…”

Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech

“…be found in telephony speech or readings of audio books. On standard tasks for this scenario, as switchboard and librispeech [1,2], typical WERs are below 10 % . Nevertheless, ASR on noisy data remains challenging.…”