Automatic identification of word translations from unrelated English and German corpora

Abstract

Algorithms for the alignment of words in translated texts are well established. However, only recently new approaches have been proposed to identify word translations from non-parallel or even unrelated texts. This task is more difficult, because most statistical clues useful in the processing of parallel texts cannot be applied to non-parallel texts. Whereas for parallel texts in some studies up to 99% of the word alignments have been shown to be correct, the accuracy for non-parallel texts has been around 30% up to now. The current study, which is based on the assumption that there is a correlation between the patterns of word co-occurrences in corpora of different languages, makes a significant improvement to about 72% of word translations identified correctly.

Using Term Position Similarity and Language Modeling for Bilingual Document Alignment

“…If two documents are mutual translations, the sequence of positions of those terms should be correlated. Much past research (Ma and Liberman, 1999;Rapp, 1999) has exploited these features, using a fixed-size window and counting the co-occurrences in this range. This method, however, requires considerable tuning of parameters, and if two shared terms are located outside of the window, no credit will be added.…”

“…the similarity of document URLs or language tags within URLs), some emphasize more the actual content of the documents. Previous work (Rapp, 1999;Ma and Liberman, 1999) focused on document alignment by counting word co-occurrences between source and target documents in a fixed-size window. More recently, methods from cross-lingual information retrieval (CLIR) have been used (Snover et al, 2008;Abdul Rauf and Schwenk, 2011), ranking lists of target documents given a source document by a probabilistic model.…”

A novel method for dictionary translation

“…Our approach for building bilingual dictionaries has been influenced by Rapp (1999) and Koehn and Knight (2002). Also in our approach we build a seed dictionary within the first phase of building bilingual dictionary, but opposite to Koehn and Knight (2002), we show that knowledge-poor data mining methods can be used successfully even for the languages belonging to different families (English and Polish).…”

Active learning for constructing transliteration lexicons from the Web

“…These include exploring query logs (Brill et al, 2001), unrelated corpus (Rapp, 1999), and comparable corpus (Fung & Yee, 1998;Huang, Zhang, & Vogel, 2005;Nie et al, 1999). To establish correspondence, these algorithms usually rely on one or more statistical clues, such as the correlation between word frequencies, cognates of similar spelling, or pronunciations.…”