Free/Open-Source Resources in the Apertium Platform for Machine Translation Research and Development

Tyers, Francis M.; Sánchez-Martínez, Felipe; Ortiz-Rojas, Sergio; Forcada, Mikel L.

doi:10.2478/v10108-010-0015-5

Cited by 15 publications

(8 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Several databases were used to find the word pairs. The 'apertium' translation database (Tyers, Sánchez-Martínez, Ortiz-Rojas, & Forcada, 2010) was used to define an initial set of Basque-Spanish noun translations. Next, using the citation-form phonological transcription in lexical databases for Spanish ('B-Pal', Davis & Perea, 2005) and Basque ('E-Hitz', Perea et al, 2006), pairs of nouns were chosen such that: the members of each pair had the same number of syllables but distinct initial phonemes, the Levenshtein distance between the CV transcriptions of each pair was less than three, the Levenshtein distance between the phonological transcriptions was greater than three (to avoid cognates), the absolute difference in log 10 frequency was less than 2, both the Spanish and Basque frequencies-per-million were greater than 5, both the Spanish and the Basque words had a noun part-of-speech tag, and that the absolute difference in the number of phonemes was not greater than 1.…”

Section: Materials and Designmentioning

confidence: 99%

Modeling accuracy as a function of response time with the generalized linear mixed effects model

Davidson¹,

Martin²

2013

Acta Psychologica

View full text Add to dashboard Cite

a b s t r a c t a r t i c l e i n f oIn psycholinguistic studies using error rates as a response measure, response times (RT) are most often analyzed independently of the error rate, although it is widely recognized that they are related. In this paper we present a mixed effects logistic regression model for the error rate that uses RT as a trial-level fixed-and random-effect regression input. Production data from a translation-recall experiment are analyzed as an example. Several model comparisons reveal that RT improves the fit of the regression model for the error rate. Two simulation studies then show how the mixed effects regression model can identify individual participants for whom (a) faster responses are more accurate, (b) faster responses are less accurate, or (c) there is no relation between speed and accuracy. These results show that this type of model can serve as a useful adjunct to traditional techniques, allowing psycholinguistic researchers to examine more closely the relationship between RT and accuracy in individual subjects and better account for the variability which may be present, as well as a preliminary step to more advanced RT-accuracy modeling.

show abstract

Section: Materials and Designmentioning

confidence: 99%

Modeling accuracy as a function of response time with the generalized linear mixed effects model

Davidson¹,

Martin²

2013

Acta Psychologica

View full text Add to dashboard Cite

show abstract

“…For example the UniMorph project (Kirov et al, 2016) extracts and normalizes morphological paradigms from the Wiktionary free online dictionary site. Further, finite state transducers for morphological analysis and generation for a multitude of languages are available in the Apertium framework (Tyers et al, 2010). Both of these resources can be used to collect inflected words and for each word a set of possible lemmas together with the corresponding morphological features.…”

Section: Morphological Transducersmentioning

confidence: 99%

Universal Lemmatizer: A sequence-to-sequence model for lemmatizing Universal Dependencies treebanks

2020

View full text Add to dashboard Cite

In this paper, we present a novel lemmatization method based on a sequence-to-sequence neural network architecture and morphosyntactic context representation. In the proposed method, our context-sensitive lemmatizer generates the lemma one character at a time based on the surface form characters and its morphosyntactic features obtained from a morphological tagger. We argue that a sliding window context representation suffers from sparseness, while in majority of cases the morphosyntactic features of a word bring enough information to resolve lemma ambiguities while keeping the context representation dense and more practical for machine learning systems. Additionally, we study two different data augmentation methods utilizing autoencoder training and morphological transducers especially beneficial for low-resource languages. We evaluate our lemmatizer on 52 different languages and 76 different treebanks, showing that our system outperforms all latest baseline systems. Compared to the best overall baseline, UDPipe Future, our system outperforms it on 62 out of 76 treebanks reducing errors on average by 19% relative. The lemmatizer together with all trained models is made available as a part of the Turku-neural-parsing-pipeline under the Apache 2.0 license.

show abstract

“…For the latter five treebanks with tiny training sample, we trained the tagger and parser in the standard manner, despite the tiny training set size. However, for four of these five languages (Armenian, Buryat, Kazakh and Kurmanji) we used Apertium morphological transducers (Tyers et al, 2010) to artificially extend the lemmatizer training data by including new words from the transducer not present in the original training data (methods are similar to those used with Breton and Faroese, for details see Section 4.1). Naija is parsed using the English-EWT models without any extra processing as it strongly resembles English language and at the same time lacks all resources.…”

Section: Near-zero Resource Languagesmentioning

confidence: 99%

Untitled

Kanerva¹,

Ginter²,

Miekka³

et al. 2018

Proceedings of The

View full text Add to dashboard Cite

In this paper we describe the TurkuNLP entry at the CoNLL 2018 Shared Task on Multilingual Parsing from Raw Text to Universal Dependencies. Compared to the last year, this year the shared task includes two new main metrics to measure the morphological tagging and lemmatization accuracies in addition to syntactic trees. Basing our motivation into these new metrics, we developed an end-to-end parsing pipeline especially focusing on developing a novel and state-of-the-art component for lemmatization. Our system reached the highest aggregate ranking on three main metrics out of 26 teams by achieving 1st place on metric involving lemmatization, and 2nd on both morphological tagging and parsing.

show abstract

Free/Open-Source Resources in the Apertium Platform for Machine Translation Research and Development

Cited by 15 publications

References 11 publications

Modeling accuracy as a function of response time with the generalized linear mixed effects model

Modeling accuracy as a function of response time with the generalized linear mixed effects model

Universal Lemmatizer: A sequence-to-sequence model for lemmatizing Universal Dependencies treebanks

Untitled

Contact Info

Product

Resources

About