Quality of Post-Edited Interlingual Subtitling: FAR Model, Translator’s Assessment and Audience Reception

Abstract

This paper analyzes the quality of machine translated interlingual subtitles, which were post-edited in the language pair EN/PT-BR. Our analyses applied the FAR model, a Translation Quality Assessment Model, to the PT-BR subtitles of The Red Sea Diving Resort movie trailer, correlating it to empirical data collected with translators (quality assessment) and audience (reception). Reception data was collected with undergraduate students, which were divided into two groups: the control group that watched the subtitled trailer available on Netflix; and the experimental group that watched the trailer with post-edited subtitles. Quality data was collected with translators and they watched the trailer with post-edited subtitles. We used a 5-point Likert-type scale, a questionnaire and a guided think-aloud protocol to collect our data. Data collected with both translators and students were correlated with the FAR model error scores (Functional Equivalence, Acceptability and Readability). Our results indicate that the post-edited subtitles had good quality in terms of meaning and target language norms; however, the technical parameters had lower quality, which affected the trailer appreciation as reported by most of the audience. Due to the small sample size, further empirical studies are required to obtain solid standards for assessing the quality of post-edited subtitles.

Applying machine translation to Chinese–English subtitling: Constraints and challenges

“…In recent years, there has been a growing interest in studies that centre on the performance and evaluation of MT in subtitling (Bogucki & Díaz Cintas, 2020;Brendel & Vela, 2022;Gambier, 2023). Koglin et al (2022) applied Pedersen's FAR model to assess the quality of subtitles that were machine-translated and then post-edited by human beings using a triangulation research approach. Specifically, they correlated the results of their analysis with empirical data collected from think-aloud protocols and open-ended questionnaires.…”

“…Given such an overwhelming demand for audiovisual translation (AVT), bringing machine translation (MT) to the industry has been considered to have increased the potential to meet surging market needs (Chan, 2017;Díaz Cintas & Massidda, 2020). In fact, the topic of applying MT to AVT has aroused growing interest across academia, with focuses on the development and application of specific tools (Bogucki & Díaz Cintas, 2020;Georgakopoulou, 2019a), pre-processing and post-editing practices (Bouillon et al, 2018), the acceptability of MT output (Koglin et al, 2022), and many other related topics (Bogucki, 2016;Deng & Gambier, 2019;Petukhova et al, 2012;Turcato et al, 2000).…”

Quality assessment of machine-translated post-edited subtitles: an analysis of Brazilian translators’ perceptions

“…Considering the relationship between MTPE subtitling, quality assessment, and audience reception, the study carried out by Koglin et al (2022) combined the human evaluation of MTPE subtitles and the error scores of the FAR model. The study reports on two pilot experiments: one about the quality and the other about the reception of MTPE interlingual subtitles.…”

“…Despite that, the general subtitling quality was unaffected, as shown by the correlation of human evaluation (i.e., translators and undergraduate students) and the error categorizations of the FAR model. Following this, the pilot experiment on quality reported in Koglin et al (2022) was then conducted with a larger number of participants, as detailed in the following section.…”

“…In the academic field, the number of investigations into subtitling and MT has increased (Karakanta, 2022;Volk, 2008), mainly to assess the quality of subtitle translation (Brendel & Vela, 2022;Doherty & Kruger, 2018;Koglin et al, 2022) and to evaluate whether MT is feasible and able to help the translators to work more efficiently and with increased productivity (Castilho, 2011;Etchegoyhen et al, 2014;Karakanta et al, 2022;Koponen et al, 2020;Matusov et al, 2019).…”