A Speech Test Set of Practice Business Presentations with Additional Relevant Texts

Macháček, Dominik; Kratochvíl, Jonáš; Vojtěchová, Tereza; Bojar, Ondřej

doi:10.1007/978-3-030-31372-2_13

Cited by 3 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Antrecorp 33 (Macháček et al, 2019), a test set of up to 90-second mock business presentations given by high school students in very noisy conditions. None of the speakers is a native speaker of English (see the paper for the composition of nationalities) and their English contains many lexical, grammatical and pronunciation errors as well as disfluencies due to the spontaneous nature of the speech.…”

Section: Acknowledgementsmentioning

confidence: 99%

Findings of the Iwslt 2020 Evaluation Campaign

Ansari¹,

Axelrod²,

Bach³

et al. 2020

Proceedings of the 17th International Conference on Spoken Language Translation

Self Cite

View full text Add to dashboard Cite

The evaluation campaign of the International Conference on Spoken Language Translation (IWSLT 2020) featured this year six challenge tracks: (i) Simultaneous speech translation, (ii) Video speech translation, (iii) Offline speech translation, (iv) Conversational speech translation, (v) Open domain translation, and (vi) Non-native speech translation. A total of 30 teams participated in at least one of the tracks. This paper introduces each track's goal, data and evaluation metrics, and reports the results of the received submissions.

show abstract

Section: Acknowledgementsmentioning

confidence: 99%

Findings of the Iwslt 2020 Evaluation Campaign

Ansari¹,

Axelrod²,

Bach³

et al. 2020

Proceedings of the 17th International Conference on Spoken Language Translation

Self Cite

View full text Add to dashboard Cite

show abstract

“…The non-native test set was already used in IWSLT Non-Native Translation Task in 2020 and it is described in Ansari et al ( 2020) Appendix A.6. Specifically, we used the Antrecorp (Macháček et al, 2019; mock business presentations by high-school students) and the auditing presentations (SAO) parts.…”

Section: Time Shift For Better Simultaneitymentioning

confidence: 99%

Findings of the IWSLT 2022 Evaluation Campaign

Anastasopoulos¹,

Barrault²,

Bentivogli³

et al. 2022

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022)

Self Cite

View full text Add to dashboard Cite

The evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved.

show abstract

“…The speakers in TED talks are native. "Non-Native" subset consists of mock business presentations of European high school students (Macháček et al, 2019), and of presentations by representatives of European supreme audit institutions. This subset is described in Findings on page 39 (numbered page 136).…”

Section: Evaluation Datamentioning

confidence: 99%

MT Metrics Correlate with Human Ratings of Simultaneous Speech Translation

Macháček¹,

Bojar²,

Dabre³

2022

Preprint

View full text Add to dashboard Cite

There have been several studies on the correlation between human ratings and metrics such as BLEU, chrF2 and COMET in machine translation. Most, if not all consider full-sentence translation. It is unclear whether human ratings of simultaneous speech translation Continuous Rating (CR) correlate with these metrics or not. Therefore, we conduct an extensive correlation analysis of CR and the aforementioned automatic metrics on evaluations of candidate systems at English-German simultaneous speech translation task at IWSLT 2022. Our studies reveal that the offline MT metrics correlate with CR and can be reliably used for evaluating machine translation in the simultaneous mode, with some limitations on the test set size. This implies that automatic metrics can be used as proxies for CR, thereby alleviating the need for human evaluation.

show abstract

A Speech Test Set of Practice Business Presentations with Additional Relevant Texts

Cited by 3 publications

References 13 publications

Findings of the Iwslt 2020 Evaluation Campaign

Findings of the Iwslt 2020 Evaluation Campaign

Findings of the IWSLT 2022 Evaluation Campaign

MT Metrics Correlate with Human Ratings of Simultaneous Speech Translation

Contact Info

Product

Resources

About