Recent advances in machine translation using comparable corpora

Abstract

This paper highlights some of the recent developments in the field of machine translation using comparable corpora. We start by updating previous definitions of comparable corpora and then look at bilingual versions of continuous vector space models. Recently, neural networks have been used to obtain latent context representations with only few dimensions which are often called word embeddings. These promising new techniques cannot only be applied to parallel but also to comparable corpora. Subsequent sections of the paper discuss work specifically targeting at machine translation using comparable corpora, as well as work dealing with the extraction of parallel segments from comparable corpora. Finally, we give an overview on the design and the results of a recent shared task on measuring document comparability across languages.

Automated annotation of parallel bible corpora with cross-lingual semantic concordance

“…Biblical texts are usually well studied, so both Strong's numbers and morphological information are available for Hebrew and Greek texts. Automated glossing is also a widely studied area for other texts and languages (see Rapp, Sharoff, and Zweigenbaum 2016;McMillan-Major 2020;Zhao et al 2020). Much work in this area has been devoted to methods based on neural networks and word embeddings.…”

QS Subject Focus Summit 2020 on Modern Languages and Linguistics: Languages and migration in a globalized world

“…The use of information technologies and artificial intelligence in theoretical and applied linguistics is one of the most relevant and promising tracks of interdisciplinary research. Linguistic projects involving the use of computer technologies are proliferating (Alemi & Haeri 2020, Fuertes-Olivera et al 2016, Hirschberg & Manning 2015, Paris et al 2013, Rapp et al 2016. The creation of national corpora, participation of linguists in the development of artificial intelligence systems, the use of artificial intelligence in compiling dictionaries, the application of computers and robotics in language education were in the focus of the Summit.…”

“…Использование современных компьютерных технологий и искусственного интеллекта в теоретической и прикладной лингвистике -одно из актуальных и перспективных областей междисциплинарных исследований (Alemi & Haeri 2020, Fuertes-Olivera et al 2016, Hirschberg & Manning 2015, Paris et al 2013, Rapp et al 2016. Вопросы проведения лингвистических исследований с применением компьютерных технологий, создание национальных корпусов, участие лингвистов в создании искусственного интеллекта, применение искусственного интеллекта в создании словарей, использование компьютеров и робототехники в образовании и обучении иностранным языкам -это лишь немногие вопросы, которые обсуждались на саммите.…”

Cadlaws – An English–French Parallel Corpus of Legally Equivalent Documents

“…A review of the large body of research on mining parallel sentences in collections of monolingual texts from comparable corpora can be found at Schwenk (2019). The methodology used to extract a comparable corpus is different from that of a parallel one (see, for example, Hewavitharana & Vogel, 2016;Rapp et al, 2016).…”