Inflection Generation as Discriminative String Transduction

Nicolai, Garrett; Cherry, Colin; Kondrak, Grzegorz

doi:10.3115/v1/n15-1093

Cited by 63 publications

(95 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SIGMORPHON hosted two shared tasks on paradigm completion (Cotterell et al, 2016(Cotterell et al, , 2017, in order to encourage the development of systems for the task. One approach is to treat it as a string transduction problem by applying an alignment model with a semi-Markov model (Durrett and DeNero, 2013;Nicolai et al, 2015). Recently, neural sequenceto-sequence models are also widely used (Faruqui et al, 2016;Kann and Schütze, 2016;Aharoni and Goldberg, 2017;Zhou and Neubig, 2017).…”

Section: Related Workmentioning

confidence: 99%

Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models

Jin

Kann

2017

Proceedings of the First Workshop on Subword and Character Level Models in NLP

View full text Add to dashboard Cite

Multi-task training is an effective method to mitigate the data sparsity problem. It has recently been applied for crosslingual transfer learning for paradigm completion-the task of producing inflected forms of lemmata-with sequenceto-sequence networks. However, it is still vague how the model transfers knowledge across languages, as well as if and which information is shared. To investigate this, we propose a set of data-dependent experiments using an existing encoder-decoder recurrent neural network for the task. Our results show that indeed the performance gains surpass a pure regularization effect and that knowledge about language and morphology can be transferred.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models

Jin

Kann

2017

Proceedings of the First Workshop on Subword and Character Level Models in NLP

View full text Add to dashboard Cite

show abstract

“…SIGMORPHON hosted two shared tasks on paradigm completion (Cotterell et al, , 2017, in order to encourage the development of systems for the task. One approach is to treat it as a string transduction problem by applying an alignment model with a semi-Markov model Nicolai et al, 2015). Recently, neural sequenceto-sequence models are also widely used Kann and Schütze, 2016;Aharoni and Goldberg, 2017; Zhou and Neubig, 2017).…”

Section: Related Workmentioning

confidence: 99%

Proceedings of the First Workshop on Subword and Character Level Models in NLP

2017

View full text Add to dashboard Cite

IntroductionTraditional NLP starts with a hand-engineered layer of representation, the level of tokens or words. A tokenization component first breaks up the text into units using manually designed rules. Tokens are then processed by components such as word segmentation, morphological analysis and multiword recognition. The heterogeneity of these components makes it hard to create integrated models of both structure within tokens (e.g., morphology) and structure across multiple tokens (e.g., multi-word expressions). This approach can perform poorly (i) for morphologically rich languages, (ii) for noisy text, (iii) for languages in which the recognition of words is difficult and (iv) for adaptation to new domains; and (v) it can impede the optimization of preprocessing in end-to-end learning.The workshop provides a forum for discussing recent advances as well as future directions on sub-word and character-level natural language processing and representation learning that address these problems.We received 37 submissions, out of which we accepted 24 as papers and 4 as extended abstracts. AbstractMost of neural language models use different kinds of embeddings for word prediction. While word embeddings can be associated to each word in the vocabulary or derived from characters as well as factored morphological decomposition, these word representations are mainly used to parametrize the input, i.e. the context of prediction. This work investigates the effect of using subword units (character and factored morphological decomposition) to build output representations for neural language modeling. We present a case study on Czech, a morphologically-rich language, experimenting with different input and output representations. When working with the full training vocabulary, despite unstable training, our experiments show that augmenting the output word representations with character-based embeddings can significantly improve the performance of the model. Moreover, reducing the size of the output look-up table, to let the character-based embeddings represent rare words, brings further improvement. IntroductionMost of neural language models, such as n-gram models (Bengio et al., 2003) are word based and rely on the definition of a finite vocabulary V. Therefore, a look-up table maps each wordw ∈ V to a vector of real features, and is stored in a matrix. While this approach yields significant improvement for a variety of tasks and languages, see for instance (Schwenk, 2007) in speech recognition and (Le et al., 2012; Devlin et al., 2014; in machine translation, it induces several limitations.For morphologically-rich languages, like Czech or German, the lexical coverage is still an important issue, since there is a combinatorial explosion of word forms, most of which are hardly observed on training data. On the one hand, growing the look-up table is not a solution, since it would increase the number of parameters without having enough training examples for a proper estimation. On the other hand, rare words can be replaced...

show abstract

“…Existing methods to automatically learn/predict the inflection of the verbs for morphologically rich languages have used supervised or semi-supervised learning (Durrett and DeNero, 2013;Ahlberg et al, 2014;Nicolai et al, 2015;Faruqui et al, 2016) to learn morphological rules on word forms in order to inflect the desired words. Other approaches have relied on linguistic information, such as morphemes and phonology (Cotterell et al, 2016); morphosyntactic disambiguation rules (Suárez et al, 2005); and graphical models (Dreyer and Eisner, 2009).…”

Section: Related Workmentioning

confidence: 99%

Improving the Naturalness and Expressivity of Language Generation for Spanish

Barros¹,

Gkatzia²,

Lloret³

2017

Proceedings of the 10th International Conference on Natural Language Generation

View full text Add to dashboard Cite

We present a flexible Natural Language Generation approach for Spanish, focused on the surface realisation stage, which integrates an inflection module in order to improve the naturalness and expressivity of the generated language. This inflection module inflects the verbs using an ensemble of trainable algorithms whereas the other types of words (e.g. nouns, determiners, etc) are inflected using hand-crafted rules. We show that our approach achieves 2% higher accuracy than two stateof-art inflection generation approaches. Furthermore, our proposed approach also predicts an extra feature: the inflection of the imperative mood, which was not taken into account by previous work. We also present a user evaluation, where we demonstrate that the proposed method significantly improves the perceived naturalness of the generated language.

show abstract

Inflection Generation as Discriminative String Transduction

Cited by 63 publications

References 7 publications

Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models

Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models

Proceedings of the First Workshop on Subword and Character Level Models in NLP

Improving the Naturalness and Expressivity of Language Generation for Spanish

Contact Info

Product

Resources

About