Online Hybrid CTC/Attention Architecture for End-to-End Speech Recognition

Miao, Haoran; Cheng, Gaofeng; Zhang, Pengyuan; Ta, Li; Yan, Yonghong

doi:10.21437/interspeech.2019-2018

Cited by 50 publications

(50 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The hyperparameters λ and γ are tunable. For online decoding, we proposed DWJD algorithm [10] to 1) coordinate the forward propagation in the encoder and the beam search in the decoder; 2) address the unsynchronized predictions of the sMoChA-based decoder and CTC outputs. MTA [11], which performs attention on top of the truncated historical encoder outputs, outperforms the sMoChA by exploiting longer history.…”

Section: Online Ctc/attention E2e Architecturementioning

confidence: 99%

Transformer-Based Online CTC/Attention End-To-End Speech Recognition Architecture

Miao

Cheng

Gao

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Recently, Transformer has gained success in automatic speech recognition (ASR) field. However, it is challenging to deploy a Transformer-based end-to-end (E2E) model for online speech recognition. In this paper, we propose the Transformer-based online CTC/attention E2E ASR architecture, which contains the chunk self-attention encoder (chunk-SAE) and the monotonic truncated attention (MTA) based self-attention decoder (SAD). Firstly, the chunk-SAE splits the speech into isolated chunks. To reduce the computational cost and improve the performance, we propose the state reuse chunk-SAE. Sencondly, the MTA based SAD truncates the speech features monotonically and performs attention on the truncated features. To support the online recognition, we integrate the state reuse chunk-SAE and the MTA based SAD into online CTC/attention architecture. We evaluate the proposed online models on the HKUST Mandarin ASR benchmark and achieve a 23.66% character error rate (CER) with a 320 ms latency. Our online model yields as little as 0.19% absolute CER degradation compared with the offline baseline, and achieves significant improvement over our prior work on Long Short-Term Memory (LSTM) based online E2E models.

show abstract

Section: Online Ctc/attention E2e Architecturementioning

confidence: 99%

Transformer-Based Online CTC/Attention End-To-End Speech Recognition Architecture

Miao

Cheng

Gao

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…tend to zero for a certain output step, the weak attentions would inevitably pass down to all the following steps [24], which results in vanished gradients for backpropagation. To remedy this, sMoChA [22] is proposed to simplify eq. 7to:…”

Section: Transformer For Online Asrmentioning

confidence: 99%

“…MTA [22] further simplifies the workflow of attention computation, in which the production of attention weights and context vector has been unified for both training and inference, using eq. 12and (8) respectively.…”

Section: Transformer For Online Asrmentioning

confidence: 99%

End-to-end Speech Recognition with Adaptive Computation Steps

Li¹,

Liu²,

Hattori³

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we present Adaptive Computation Steps (ACS) algorithm, which enables end-to-end speech recognition models to dynamically decide how many frames should be processed to predict a linguistic output. The model that applies ACS algorithm follows the encoder-decoder framework, while unlike the attention-based models, it produces alignments independently at the encoder side using the correlation between adjacent frames. Thus, predictions can be made as soon as sufficient acoustic information is received, which makes the model applicable in online cases. Besides, a small change is made to the decoding stage of the encoder-decoder framework, which allows the prediction to exploit bidirectional contexts. We verify the ACS algorithm on a Mandarin speech corpus AIShell-1, and it achieves a 31.2% CER in the online occasion, compared to the 32.4% CER of the attention-based model. To fully demonstrate the advantage of ACS algorithm, offline experiments are conducted, in which our ACS model achieves an 18.7% CER, outperforming the attention-based counterpart with the CER of 22.0%.

show abstract

“…To overcome this limitation, online attention mechanisms have been proposed, and can be broadly classed into the following categories: i) Bernoulli-based attention methods formulate the triggering of outputs as a stochas-tic process, and the decision is sampled from the attending probabilities. These include Hard Monotonic Attention (HMA) [11], Monotonic Chunkwise Attention (MoChA) [12,13] and Monotonic Truncated Attention (MTA) [14]; ii) Triggered attention methods are conditioned on the forced alignment produced by CTC [15,16]; iii) Accumulation-based attention methods accumulate the attention weights along encoding timesteps and the computation is halted once the sum reaches a certain threshold. These include Adaptive Computation Steps (ACS) [17,18] and Decoder-end Adaptive Computation Steps (DACS) [19].…”

Section: Introductionmentioning

confidence: 99%

Head-Synchronous Decoding for Transformer-Based Streaming ASR

Zorilă

Doddipatla

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Online Transformer-based automatic speech recognition (ASR) systems have been extensively studied due to the increasing demand for streaming applications. Recently proposed Decoder-end Adaptive Computation Steps (DACS) algorithm for online Transformer ASR was shown to achieve state-of-the-art performance and outperform other existing methods. However, like any other online approach, the DACS-based attention heads in each of the Transformer decoder layers operate independently (or asynchronously) and lead to diverged attending positions. Since DACS employs a truncation threshold to determine the halting position, some of the attention weights are cut off untimely and might impact the stability and precision of decoding. To overcome these issues, here we propose a head-synchronous (HS) version of the DACS algorithm, where the boundary of attention is jointly detected by all the DACS heads in each decoder layer. ASR experiments on Wall Street Journal (WSJ), AIShell-1 and Librispeech show that the proposed method consistently outperforms vanilla DACS and achieves state-of-the-art performance. We will also demonstrate that HS-DACS has reduced decoding cost when compared to vanilla DACS.

show abstract

Online Hybrid CTC/Attention Architecture for End-to-End Speech Recognition

Cited by 50 publications

References 20 publications

Transformer-Based Online CTC/Attention End-To-End Speech Recognition Architecture

Transformer-Based Online CTC/Attention End-To-End Speech Recognition Architecture

End-to-end Speech Recognition with Adaptive Computation Steps

Head-Synchronous Decoding for Transformer-Based Streaming ASR

Contact Info

Product

Resources

About