Large-Context Pointer-Generator Networks for Spoken-to-Written Style Conversion

Ihori, Mana; Takashima, Akihiko; Masumura, Ryo

doi:10.1109/icassp40776.2020.9053930

“…While ITN as a problem is not new [19], work addressing the task using end to end neural sequence generation has been very limited. Most recently, [20] used pointer generator networks for ITN in Japanese. They limit their evaluation to a single dataset and note that the copy mechanism from the architecture [13] is key to performance improvements.…”

Section: Related Workmentioning

confidence: 99%

Neural Inverse Text Normalization

Sunkara

¹

,

Shivade

²

,

Bodapati

³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

While there have been several contributions exploring state of the art techniques for text normalization, the problem of inverse text normalization (ITN) remains relatively unexplored. The best known approaches leverage finite state transducer (FST) based models which rely on manually curated rules and are hence not scalable. We propose an efficient and robust neural solution for ITN leveraging transformer based seq2seq models and FST-based text normalization techniques for data preparation. We show that this can be easily extended to other languages without the need for a linguistic expert to manually curate them. We then present a hybrid framework for integrating Neural ITN with an FST to overcome common recoverable errors in production environments. Our empirical evaluations show that the proposed solution minimizes incorrect perturbations (insertions, deletions and substitutions) to ASR output and maintains high quality even on out of domain data. A transformer based model infused with pretraining consistently achieves a lower WER across several datasets and is able to outperform baselines on English, Spanish, German and Italian datasets.

show abstract

“…Large-context encoder-decoder models: Large-context encoderdecoder models that can capture long-range linguistic contexts beyond sentence boundaries or utterance boundaries have received significant attention in E2E-ASR [7,8], machine translation [14,15], and some natural language generation tasks [16,17]. In recent studies, transformer-based large-context encoder-decoder models have been introduced in machine translation [18,19].…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Transformer-Based Large-Context End-To-End ASR with Large-Context Knowledge Distillation

Masumura

¹

,

Makishima

²

,

Ihori

³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We present a novel large-context end-to-end automatic speech recognition (E2E-ASR) model and its effective training method based on knowledge distillation. Common E2E-ASR models have mainly focused on utterance-level processing in which each utterance is independently transcribed. On the other hand, large-context E2E-ASR models, which take into account long-range sequential contexts beyond utterance boundaries, well handle a sequence of utterances such as discourses and conversations. However, the transformer architecture, which has recently achieved state-of-the-art ASR performance among utterance-level ASR systems, has not yet been introduced into the large-context ASR systems. We can expect that the transformer architecture can be leveraged for effectively capturing not only input speech contexts but also long-range sequential contexts beyond utterance boundaries. Therefore, this paper proposes a hierarchical transformer-based large-context E2E-ASR model that combines the transformer architecture with hierarchical encoder-decoder based large-context modeling. In addition, in order to enable the proposed model to use long-range sequential contexts, we also propose a large-context knowledge distillation that distills the knowledge from a pre-trained large-context language model in the training phase. We evaluate the effectiveness of the proposed model and proposed training method on Japanese discourse ASR tasks.

show abstract

“…While ITN as a problem is not new [19], work addressing the task using end to end neural sequence generation has been very limited. Most recently, [20] used pointer generator networks for ITN in Japanese. They limit their evaluation to a single dataset and note that the copy mechanism from the architecture [13] is key to performance improvements.…”

Section: Related Workmentioning

confidence: 99%

Neural Inverse Text Normalization

Sunkara¹,

Shivade²,

Bodapati³

et al. 2021

Preprint

View full text Add to dashboard Cite

While there have been several contributions exploring state of the art techniques for text normalization, the problem of inverse text normalization (ITN) remains relatively unexplored. The best known approaches leverage finite state transducer (FST) based models which rely on manually curated rules and are hence not scalable. We propose an efficient and robust neural solution for ITN leveraging transformer based seq2seq models and FST-based text normalization techniques for data preparation. We show that this can be easily extended to other languages without the need for a linguistic expert to manually curate them. We then present a hybrid framework for integrating Neural ITN with an FST to overcome common recoverable errors in production environments. Our empirical evaluations show that the proposed solution minimizes incorrect perturbations (insertions, deletions and substitutions) to ASR output and maintains high quality even on out of domain data. A transformer based model infused with pretraining consistently achieves a lower WER across several datasets and is able to outperform baselines on English, Spanish, German and Italian datasets.

show abstract

Large-Context Pointer-Generator Networks for Spoken-to-Written Style Conversion

Cited by 16 publications

References 15 publications

Neural Inverse Text Normalization

Neural Inverse Text Normalization

Hierarchical Transformer-Based Large-Context End-To-End ASR with Large-Context Knowledge Distillation

Neural Inverse Text Normalization

Contact Info

Product

Resources

About