Indices of cognitive effort in machine translation post-editing

Vieira, Lucas Nunes

doi:10.1007/s10590-014-9156-x

Cited by 31 publications

(16 citation statements)

References 37 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Here, the rationale is that increasing task load produces a sensation of effort that individuals can report in numerical terms (O'Donnell and Eggemeier 1986, p. 7). In post-editing, subjective ratings have been used as a measure of cognitive effort by, for example, Koponen (2012) and Vieira (2014). Moorkens et al (2015) contrasted subjective ratings with more objective measures, such as eye movements and number of changes in the raw MT output.…”

Section: Review Of Literaturementioning

confidence: 99%

“…Sentences at different Meteor levels, produced by different systems, were combined into a single machine-translated text to be presented for editing. This was done by randomly selecting sentences at each available decile of the Meteor spectrum (for further details see Vieira 2014). Sentences in the study sample had Meteor scores ranging between 0.14 and 1.…”

Section: Source Text and Machine Translation Outputmentioning

confidence: 99%

See 1 more Smart Citation

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

Vieira

2016

Machine Translation

Self Cite

View full text Add to dashboard Cite

There has been growing interest of late in the cognitive effort required by post-editing of machine translation. Compared to number of editing operations, cognitive (or mental) effort is frequently considered a more decisive indicator of the overall effort expended by post-editors. Estimating cognitive effort is not straightforward, however. Previous studies often triangulate different measures to obtain a consensus, but little post-editing research to date has attempted to show how measures of cognitive effort relate to each other in a multivariate analysis. This paper addresses this by presenting an exploratory comparison of cognitive measures based on eye tracking, pauses, editing time, and subjective ratings collected in a post-editing task carried out by professional and non-professional participants. All measures correlated with each other, but a principal components analysis showed that the measures cluster together in different ways. In particular, measures that increase with task time alone behaved differently from the others, with higher mutual associations and higher reliability. Regarding differences between professional and non-professional participants, it was observed that subjective ratings were overall more strongly associated with objective measures in the case of professionals. Surprising findings from previous research based on pause ratio are discussed. The paper argues that a pause typology will benefit the study of pause lengths and cognitive effort in post-editing.

show abstract

Section: Review Of Literaturementioning

confidence: 99%

Section: Source Text and Machine Translation Outputmentioning

confidence: 99%

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

Vieira

2016

Machine Translation

Self Cite

View full text Add to dashboard Cite

show abstract

“…While HTER is normalised by number of tokens, this could be reflecting a penalisation of extremely short sentences, where any modifications in the MT output will inevitably amount to a large part of the sentence in relative terms -an effect consistent with a pattern observed by O'Brien (2011) and Vieira (2014) for a similar score. However, excluding all occurrences of this sentence from the data did not change results, and the effect still holds for longer sentences, as seen in example (2).…”

Section: Post-editing Effort and Post-edited Qualitymentioning

confidence: 86%

Translation in Transition

Vieira¹

2017

Benjamins Translation Library

View full text Add to dashboard Cite

General rightsThis document is made available in accordance with publisher policies. Please cite only the published version using the reference above. Full terms of use are available: http://www.bristol.ac.uk/pure/about/ebr-terms This chapter is under copyright. the publisher should be contacted for permission to re-use or reprint the material in any form CHAPTER 6 From process to product: links between post-editing effort and post-edited quality Lucas Nunes Vieira University of Bristol Post-editing of machine translation (MT) is now increasingly implemented in the human translation workflow after studies in both industry and academia have demonstrated the efficacy of this practice. Post-editing still involves open questions, however, such as how best to train post-editors and how to estimate the effort required by post-editing tasks. In attempting to address some of these questions, many previous studies investigate the post-editing process, but less research has focused on the postedited product. This chapter examines the link between the process and product of post-editing by checking to see how post-editing effort data related to the quality of post-edited texts, assessed in terms of fluency (linguistic quality) and adequacy (translation accuracy). A statistical analysis indicated that the association between editing operations and the fluency of post-edited texts is dependent on the quality of the raw MT output. Interestingly, a negative association was observed between the number of eye fixations on the text and the quality of the post-edited translations. The chapter shows empirical evidence supporting the distinction between the concepts of translation fluency and adequacy, and postulates that automatic processes play a central role in post-editing performance.

show abstract

“…Aziz, Koponen and Specia 2014;Vieira 2014;O'Brien 2011). It should be noted that these studies explore the connection of effort with quantitative textual features (e.g.…”

Section: Related Workmentioning

confidence: 99%

“…2 For more information on the study's methodology, see Vieira (2016). Details of the eye-tracking task can also be found in Vieira (2014), where textual features and participants' working memory capacity are contrasted with post-editing effort.…”

Section: Notesmentioning

confidence: 99%

Cognitive effort and different task foci in post-editing of machine translation: A think-aloud study

Vieira¹

2017

Across Languages and Cultures

Self Cite

View full text Add to dashboard Cite

Post-editing of machine translation is gaining popularity as a solution to the ever-increasing demands placed on human translators. There has been a great deal of research in this area aimed at determining the feasibility of post-editing, and at predicting post-editing effort based on source text features and machine translation errors. However, considerably less is known about the mental workings of post-editing and post-editors' decision-making or, in particular, the relationship between post-editing effort and different mental processes. This paper investigates these issues by analysing data from a think-aloud study through the lens of eye movements and subjective ratings obtained in a separate task. The results show that mental processes associated with grammar and lexis are significantly associated with cognitive effort in post-editing. This association was not observed for other aspects of the task concerning, for example, discourse or the real-world use of the text. In addition, it was noted that lexical issues are linked to long sequences of thought processes. The paper shows that lexis plays a central role in post-editing, and argues that more emphasis should be placed on this issue in future research and in post-editor training.

show abstract

Indices of cognitive effort in machine translation post-editing

Cited by 31 publications

References 37 publications

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data

Translation in Transition

Cognitive effort and different task foci in post-editing of machine translation: A think-aloud study

Contact Info

Product

Resources

About