This paper describes the Yandex School of Data Analysis Russian-English system submitted to the ACL 2014 Ninth Workshop on Statistical Machine Translation shared translation task. We start with the system that we developed last year and investigate a few methods that were successful at the previous translation task including unpruned language model, operation sequence model and the new reparameterization of IBM Model 2. Next we propose a {simple yet practical} algorithm to transform Russian sentence into a more easily translatable form before decoding. The algorithm is based on the linguistic intuition of native Russian speakers, also fluent in English.
Query speller is an indispensable part of any modern search engine. In this paper we define the problem of speller performance prediction and apply it to the task of query spelling autocorrection. As candidates for query autocorrection we used the suggestions generated by a query speller. To determine their reliability we used a binary classifier trained on manually labeled data. In addition to the basic standard lexical and statistical features we utilized a number of new click-based features, what allowed to significantly outperform the algorithm trained on the basic set of features.
Noisy channel models, widely used in modern spellers, cope with typical misspellings, but do not work well with infrequent and difficult spelling errors. In this paper, we have improved the noisy channel approach by iterative stochastic search for the best correction. The proposed algorithm allowed us to avoid local minima problem and improve the F 1 measure by 6.6% on distant spelling errors.
Our participation in Bilingual Document Alignment shared task at WMT16 focuses on building a language-independent, scalable system for aligning documents based on content as opposed to using webpage meta information. The resulting system is capable of producing scored n-best lists of candidate pages and can therefore be adapted to tasks where either precision or recall is maximized. We conduct a series of experiments that show the effectiveness of the system without any specific tuning.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.