MLLP-VRAIN UPV systems for the IWSLT 2022 Simultaneous Speech Translation and Speech-to-Speech Translation tasks

Iranzo-Sánchez, Javier; Cano, Javier Jorge; Pérez-González-de-Martos, Alejandro; Pastor, Adrián Giménez; Díaz-Munío, Gonçal V. Garcés; Baquero-Arnal, Pau; Silvestre-Cerdà, Joan Albert; Saiz, Jorge Civera; Sanchís, Alberto; Juan, Alfons

doi:10.18653/v1/2022.iwslt-1.22

Search citation statements

Order By: Relevance

Paper Sections

Select...

Candidate Systems1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2022

2024

Publication Types

Select...

Other1

Article1

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automatic SST systems There are 5 evaluated SST systems: FBK (Gaido et al, 2022), NAIST (Fukuda et al, 2022), UPV (Iranzo-Sánchez et al, 2022), HW-TSC (Wang et al, 2022), and CUNI-KIT (Polák et al, 2022). More details are in system description papers.…”

Section: Candidate Systemsmentioning

confidence: 99%

MT Metrics Correlate with Human Ratings of Simultaneous Speech Translation

Macháček¹,

Bojar²,

Dabre³

2022

Preprint

View full text Add to dashboard Cite

There have been several studies on the correlation between human ratings and metrics such as BLEU, chrF2 and COMET in machine translation. Most, if not all consider full-sentence translation. It is unclear whether human ratings of simultaneous speech translation Continuous Rating (CR) correlate with these metrics or not. Therefore, we conduct an extensive correlation analysis of CR and the aforementioned automatic metrics on evaluations of candidate systems at English-German simultaneous speech translation task at IWSLT 2022. Our studies reveal that the offline MT metrics correlate with CR and can be reliably used for evaluating machine translation in the simultaneous mode, with some limitations on the test set size. This implies that automatic metrics can be used as proxies for CR, thereby alleviating the need for human evaluation.

show abstract

Section: Candidate Systemsmentioning

confidence: 99%