Analyzing ASR pretraining for low-resource speech-to-text translation

Mihaela, C Stoian; Bansal, Sameer; Sharon, Goldwater

doi:10.48550/arxiv.1910.10762

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system is trained only on transcribed SLT data, with two auxiliary tasks: pretraining the encoder and decoder with ASR and textual MT respectively. Stoian et al (2019) compare the effects of pretraining on auxiliary ASR datasets of different languages and sizes, concluding that the WER of the ASR system is more predictive of the final translation quality than language relatedness. Anastasopoulos and Chiang (2018) make the line between pipeline and end-to-end approaches more blurred by using a multi-task learning setup with two-step decoding.…”

Section: End-to-end Spoken Language Translationmentioning

confidence: 99%

Multimodal Machine Translation through Visuals and Speech

Sulubacak¹,

Çağlayan²,

Grönroos³

et al. 2019

Preprint

View full text Add to dashboard Cite

Multimodal machine translation involves drawing information from more than one modality, based on the assumption that the additional modalities will contain useful alternative views of the input data. The most prominent tasks in this area are spoken language translation, image-guided translation, and video-guided translation, which exploit audio and visual modalities, respectively. These tasks are distinguished from their monolingual counterparts of speech recognition, image captioning, and video captioning by the requirement of models to generate outputs in a different language. This survey reviews the major data resources for these tasks, the evaluation campaigns concentrated around them, the state of the art in end-to-end and pipeline approaches, and also the challenges in performance evaluation. The paper concludes with a discussion of directions for future research in these areas: the need for more expansive and challenging datasets, for targeted evaluations of model performance, and for multimodality in both the input and output space.

show abstract

Section: End-to-end Spoken Language Translationmentioning

confidence: 99%

Multimodal Machine Translation through Visuals and Speech

Sulubacak¹,

Çağlayan²,

Grönroos³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…We use speech features pre-trained on English, and first examine a high-resource within-language English-to-X ST setting (X denotes a non-English language), then we transfer the representations to 11 lower-resource X-to-English ST tasks. Transferring the parameters learned on a higher-resource ASR task has been shown to be an effective way to improve the performance and ameliorate the training of low-resource ST [16,17,18], thus we also study the interactions with selfsupervised representations and whether we can effectively combine both methods.…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Representations Improve End-to-End Speech Translation

Wang

Pino

et al. 2020

Preprint

View full text Add to dashboard Cite

End-to-end speech-to-text translation can provide a simpler and smaller system but is facing the challenge of data scarcity. Pre-training methods can leverage unlabeled data and have been shown to be effective on data-scarce settings. In this work, we explore whether self-supervised pre-trained speech representations can benefit the speech translation task in both highand low-resource settings, whether they can transfer well to other languages, and whether they can be effectively combined with other common methods that help improve low-resource end-to-end speech translation such as using a pre-trained highresource speech recognition system. We demonstrate that selfsupervised pre-trained features can consistently improve the translation performance, and cross-lingual transfer allows to extend to a variety of languages without or with little tuning.

show abstract

Analyzing ASR pretraining for low-resource speech-to-text translation

Cited by 2 publications

References 0 publications

Multimodal Machine Translation through Visuals and Speech

Multimodal Machine Translation through Visuals and Speech

Self-Supervised Representations Improve End-to-End Speech Translation

Contact Info

Product

Resources

About