Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translation

Lam, Tsz Kin; Schamoni, Shigehiko; Riezler, Stefan

doi:10.1109/icassp49357.2023.10094564

Cited by 3 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous research has established the benefits of generating synthetic examples by cropping or merging the original ones, with sub-sequence sampling for ASR (Nguyen et al, 2020), and concatenation for MT (Nguyen et al, 2021;Wu et al, 2021;Kondo et al, 2021), as well as for ASR and ST 1 github.com/mt-upc/SegAugment (Lam et al, 2022a). Our approach, however, segments documents at arbitrary points, thus providing access to a greater number of synthetic examples.…”

Section: Relevant Researchmentioning

confidence: 99%

SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations

Tsiamas,

Fonollosa,

Costa-jussà

2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

End-to-end Speech Translation is hindered by a lack of available data resources. While most of them are based on documents, a sentencelevel version is available, which is however single and static, potentially impeding the usefulness of the data. We propose a new data augmentation strategy, SEGAUGMENT, to address this issue by generating multiple alternative sentence-level versions of a dataset. Our method utilizes an Audio Segmentation system, which re-segments the speech of each document with different length constraints, after which we obtain the target text via alignment methods. Experiments demonstrate consistent gains across eight language pairs in MuST-C, with an average increase of 2.5 BLEU points, and up to 5 BLEU for low-resource scenarios in mTEDx. Furthermore, when combined with a strong system, SEGAUGMENT obtains stateof-the-art results in MuST-C. Finally, we show that the proposed method can also successfully augment sentence-level datasets, and that it enables Speech Translation models to close the gap between the manual and automatic segmentation at inference time.

show abstract