Towards better subtitles: A multilingual approach for punctuation restoration of speech transcripts

Guerreiro, Nuno M.; Rei, Ricardo; Batista, Fernando

doi:10.1016/j.eswa.2021.115740

Cited by 7 publications

(3 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work on multilingual punctuation prediction is described in Li and Lin (2020) and Guerreiro, Rei, and Batista (2021). Vandeghinste, Verwimp, Pelemans, and Wambacq (2018) models punctuation prediction in the context of speech translation, but also investigates a monolingual approach for Dutch, modelling punctuation prediction as a machine translation problem, in which the source language is the text without punctuation, and the target language is the text with punctuation.…”

Section: The Belgian Federal Covid-19 Sign Language (Becos) Corpusmentioning

confidence: 99%

FullStop: Punctuation and Segmentation Prediction for Dutch with Transformers

Vandeghinste

Guhr

2023

Preprint

View full text Add to dashboard Cite

When applying automated speech recognition (ASR) for Belgian Dutch (Van Dyck et al. 2021), the output consists of an unsegmented stream of words, without any punctuation. A next step is to perform segmentation and insert punctuation, making the ASR output more readable and easy to manually correct. As far as we know there is no publicly available punctuation insertion system for Dutch that functions at a usable level. The model we present here is an extension of the models of Guhr et al. (2021) for Dutch and is made publicly available. We trained a sequence classification model, based on the Dutch language model RobBERT (Delobelle et al. 2020). For every word in the input sequence, the models predicts a punctuation marker that follows the word. We have also extended a multilingual model, for cases where the language is unknown or where code switching applies. When performing the task of segmentation, the application of the best models onto out of domain test data, a sliding window of 200 words of the ASR output stream is sent to the classifier, and segmentation is applied when the system predicts a segmenting punctuation sign with a ratio above threshold. Results show to be much better than a machine translation baseline approach.

show abstract

Section: The Belgian Federal Covid-19 Sign Language (Becos) Corpusmentioning

confidence: 99%

FullStop: Punctuation and Segmentation Prediction for Dutch with Transformers

Vandeghinste

Guhr

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Previous work on multilingual punctuation prediction is described in Li and Lin (2020) and Guerreiro et al (2021). Vandeghinste et al (2018) models punctuation prediction in the context of speech translation, but also investigates a monolingual approach for Dutch, modelling punctuation prediction as a machine translation problem, in which the source language is the text without punctuation, and the target language is the text with punctuation.…”

Section: Related Workmentioning

confidence: 99%

FullStop: punctuation and segmentation prediction for Dutch with transformers

Vandeghinste

Guhr

2023

Lang Resources & Evaluation

View full text Add to dashboard Cite

When applying automated speech recognition (ASR) for Belgian Dutch (Van Dyck et al. 2021), the output consists of an unsegmented stream of words, without any punctuation. A next step is to perform segmentation and insert punctuation, making the ASR output more readable and easy to manually correct. As far as we know there is no publicly available punctuation insertion system for Dutch that functions at a usable level.The model we present here is an extension of the models of Guhr et al. ( 2021) for Dutch and is made publicly available. 1 We trained a sequence classification model, based on the Dutch language model RobBERT (Delobelle et al. 2020). For every word in the input sequence, the models predicts a punctuation marker that follows the word. We have also extended a multilingual model, for cases where the language is unknown or where code switching applies.When performing the task of segmentation, the application of the best models onto out of domain test data, a sliding window of 200 words of the ASR output stream is sent to the classifier, and segmentation is applied when the system predicts a segmenting punctuation sign with a ratio above threshold. Results show to be much better than a machine translation baseline approach.1. The Spoken Academic Belgian Dutch corpus, and 2. https://fortunelords.com/youtube-statistics/ 3. https://www.arts.kuleuven.be/ling/language-education-society/projects/sabed 4. https://vlo.clarin.eu 7.

show abstract

“…Text post-processing after ASR, e.g. punctuation and capitalization (Guerreiro et al, 2021) and inverse text normalization (Sunkara et al, 2021), is another important problem for multilingual and CS speech systems. Such post-processing is harder than the monolingual scenario as it requires accurate language identification in addition to transcript generation.…”

Section: Introductionmentioning

confidence: 99%

Proceedings of the 6th Workshop on Computational Approaches to Linguistic Code-Switching

2023

View full text Add to dashboard Cite

Language modeling is the technique to estimate the probability of a sequence of words. A bilingual language model is expected to model the sequential dependency for words across languages, which is difficult due to the inherent lack of suitable training data as well as diverse syntactic structure across languages. We propose a bilingual attention language model (BALM) that simultaneously performs language modeling objective with a quasi-translation objective to model both the monolingual as well as the cross-lingual sequential dependency. The attention mechanism learns the bilingual context from a parallel corpus. We will discuss the study of multilingualism in South East Asia and how code-switch language models can be useful for language processing.

show abstract

Towards better subtitles: A multilingual approach for punctuation restoration of speech transcripts

Cited by 7 publications

References 48 publications

FullStop: Punctuation and Segmentation Prediction for Dutch with Transformers

FullStop: Punctuation and Segmentation Prediction for Dutch with Transformers

FullStop: punctuation and segmentation prediction for Dutch with transformers

Proceedings of the 6th Workshop on Computational Approaches to Linguistic Code-Switching

Contact Info

Product

Resources

About