Low Latency MaxEnt- and RNN-Based Word Sequence Models for Punctuation Restoration of Closed Caption Data

Tündik, Máté Ákos; Tarján, Balázs; Szaszák, György

doi:10.1007/978-3-319-68456-7_13

Cited by 5 publications

(10 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, we have addressed punctuation of Hungarian and English using text (both on word and character basis) [2,16] and prosodic-acoustic approaches [1]. We observed, that punctuation models for English [2,9,10,13] consistently work better for periods (full stops) than commas. In contrast, for Hungarian, text-based punctuation works better for commas, whereas prosody yields better period recovery [1].…”

Section: Introductionmentioning

confidence: 86%

“…1 accepts a windowed non-overlapping word sequence (chunk) at its input and uses an embedding layer to map words to vectors. We use pre-trained embeddings (GloVe [17] for English and vectors from [18] for Hungarian); the word-level component module is adopted from [2], by using transfer learning. The vocabulary contains the k most common words in the training set (we Copyright © 2019 ISCA also add an unknown entry to map rare words, and an EOS endof-sequence symbol).…”

Section: Word-level Modelmentioning

confidence: 99%

“…and exclamation mark (!). The model is described in more detail in [2], the hyperparameters of the model are summarized in Table 1.…”

Section: Word-level Modelmentioning

confidence: 99%

“…Punctuating the produced transcripts is an important add-on in Automatic Speech Recognition (ASR), which is not inherently handled by most state-of-the-art ASR systems. The problem is well-known, therefore we overview here only the aspects relevant for our contribution, and refer the reader to [1,2,3] for further exploration of the topic.…”

Section: Introductionmentioning

confidence: 99%

“…Both text and acoustics can be exploited for punctuation; text-based models [2,6,8,10,12] are more complex, but yield better results than models based on prosodic-acoustic features [1,7,14], which are more resistant to ASR errors. The two types of features are used in parallel in some studies [7,13,11] and most of the time, pause duration is used by text-based systems.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Leveraging a Character, Word and Prosody Triplet for an ASR Error Robust and Agglutination Friendly Punctuation Approach

Szaszák

Tündik

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

Punctuating ASR transcript has received increasing attention recently, and well-performing approaches were presented based on sequence-to-sequence modelling, exploiting textual (word and character) and/or acoustic-prosodic features. In this work we propose to consider character, word and prosody based features all at once to provide a robust and highly language independent platform for punctuation recovery, which can deal also well with highly agglutinating languages with less constrained word order. We demonstrate that using such a feature triplet improves ASR error robustness of punctuation in two quite differently organized languages, English and Hungarian. Moreover, in the highly agglutinating Hungarian, where word-based approaches suffer from the exploding vocabulary (poorer semantic representation through embeddings) and less constrained word order, we show that prosodic cues and the character-based model can powerfully counteract this loss of information. We also perform a deep analysis of punctuation w.r.t. both ASR errors and agglutination to explain the improvements we observed on a solid basis.

show abstract

Section: Introductionmentioning

confidence: 86%

Section: Word-level Modelmentioning

confidence: 99%

“…and exclamation mark (!). The model is described in more detail in [2], the hyperparameters of the model are summarized in Table 1.…”

Section: Word-level Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Leveraging a Character, Word and Prosody Triplet for an ASR Error Robust and Agglutination Friendly Punctuation Approach

Szaszák

Tündik

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

Making Punctuation Restoration Robust and Fast with Multi-Task Learning and Knowledge Distillation

Hentschel

Tsunoo

Okuda

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In punctuation restoration, we try to recover the missing punctuation from automatic speech recognition output to improve understandability. Currently, large pre-trained transformers such as BERT set the benchmark on this task but there are two main drawbacks to these models. First, the pre-training data does not match the output data from speech recognition that contains errors. Second, the large number of model parameters increases inference time. To address the former, we use a multi-task learning framework with ELEC-TRA, a recently proposed improvement on BERT, that has a generator-discriminator structure. The generator allows us to inject errors into the training data and, as our experiments show, this improves robustness against speech recognition errors during inference. To address the latter, we investigate knowledge distillation and parameter pruning of ELECTRA.In our experiments on the IWSLT 2012 benchmark data, a model with less than 11% the size of BERT achieved better performance while having an 82% faster inference time.

show abstract

Proceedings of the 17th International Conference on Spoken Language Translation

Federico¹,

Waibel²,

Knight³

et al. 2020

View full text Add to dashboard Cite

The conference chairs and organizers would like to express their gratitude to everyone who contributed and supported IWSLT. Our IWSLT-20 program exceeds all our expectations in quality and breath, particularly when considering the challenges during a pandemic under lock-downs and health and travel restrictions. We thank the challenge track chairs, organizers, and participants, the program chairs and committee members, as well as all the authors that went the extra mile to submit system and research papers to IWSLT, and make this year's conference our most vibrant than ever. We also wish to express our sincere gratitude to ACL for hosting our conference and for arranging the logistics and infrastructure that allow us to hold IWSLT 2020 as a virtual online conference.

show abstract

Low Latency MaxEnt- and RNN-Based Word Sequence Models for Punctuation Restoration of Closed Caption Data

Cited by 5 publications

References 15 publications

Leveraging a Character, Word and Prosody Triplet for an ASR Error Robust and Agglutination Friendly Punctuation Approach

Leveraging a Character, Word and Prosody Triplet for an ASR Error Robust and Agglutination Friendly Punctuation Approach

Making Punctuation Restoration Robust and Fast with Multi-Task Learning and Knowledge Distillation

Proceedings of the 17th International Conference on Spoken Language Translation

Contact Info

Product

Resources

About