The effects of word order and segmentation on translation retrieval performance

Abstract

This research looks at the effects of word order and segmentation on translation retrieval performance for an experimental Japanese-English translation memory system. We implement a number of both bag-of-words and word order-sensitive similarity metrics, and test each over characterbased and word-based indexing. The translation retrieval performance of each system configuration is evaluated empirically through the notion of word edit distance between translation candidate outputs and the model translation. Our results indicate that character-based indexing is consistently superior to word-based indexing, suggesting that segmentation is an unnecessary luxury in the given domain. Word order-sensitive approaches are demonstrated to generally outperform bag-of-words methods, with source language segment-level edit distance proving the most effective similarity metric.

The hare and the tortoise: speed and accuracy in translation retrieval

“…Looking to the related literature, there is limited support for our findings, particularly in relation to IR where Fujii and Croft (1993) established that character-based indexing can perform at comparable levels to word-based indexing in Japanese IR. In research laying the foundations for this paper, Baldwin and Tanaka (2000) and Baldwin (2001a) report similar findings for a Japanese-English translation retrieval task. To the author's knowledge, there is no empirical evidence to suggest that segmentation enhances accuracy in an analogous retrieval context.…”

“…Our primary motivation in this was to make the final results for each method directly comparable. This differs from the methodology of Baldwin and Tanaka (2000) who consider the system to be "correct" if an optimal translation candidate is contained in the potentially multiple set of top-ranking outputs.…”

“…3-operation edit similarity computed in this fashion is identical to the "sequential correspondence" method of Baldwin and Tanaka (2000). To return to our example strings from Sect.…”

“…Weighted sequential correspondence (WSC) (originally proposed in Baldwin and Tanaka 2000) -the last of the segment-order-sensitive methods -goes one step further than edit distance in analysing not only segment sequentiality, but also the contiguity of matching segments. Given the input α 1 α 2 α 3 α 4 , the edit-distance and similarity methods would suggest α 1 β 1 α 2 β 2 α 3 β 3 α 4 and α 1 α 2 α 3 α 4 β 1 β 2 β 3 as equally close matches, for example, despite the second of these being more likely to produce a translation at least partially resembling that of the input string.…”

Balancing up Efficiency and Accuracy in Translation Retrieval.

“…Note that 3-operation edit similarity computed in this fashion is identical to the"sequential correspondence"method of Baldwin and Tanaka(2000),which determines the maximum sequential substring match between two strings.…”