IMS at the CoNLL 2017 UD Shared Task: CRFs and Perceptrons Meet Neural Networks

Björkelund, Anders; Faleńska, Agnieszka; Xiang, Yu; Kuhn, Jonas

doi:10.18653/v1/k17-3004

Cited by 24 publications

(23 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These representations are used to score arcs, which are greedily added to the tree. Björkelund et al (2017) perform best on Arabic, using an ensemble of many different types of bottom-up discriminative parsers. They have each of twelve parsers score potential arcs, learn a weighting function to combine them, and use the Chu-Liu-Edmonds algorithm (Chu, 1965;Edmonds, 1967) to output final parses.…”

Section: Related Workmentioning

confidence: 99%

Comparing Top-Down and Bottom-Up Neural Generative Dependency Models

Matthews

Neubig

Dyer

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

Recurrent neural network grammars (RNNGs) generate sentences using phrase-structure syntax and perform very well in terms of both language modeling and parsing performance. However, since dependency annotations are much more readily available than phrase structure annotations, we propose two new generative models of projective dependency syntax, so as to explore whether generative dependency models are similarly effective. Both models use RNNs to represent the derivation history with making any explicit independence assumptions, but they differ in how they construct the trees: one builds the tree bottom up and the other top down, which profoundly changes the estimation problem faced by the learner. We evaluate the two models on three typologically different languages: English, Arabic, and Japanese. We find that both generative models improve parsing performance over a discriminative baseline, but, in contrast to RNNGs, they are significantly less effective than non-syntactic LSTM language models. Little difference between the tree construction orders is observed for either parsing or language modeling. 1 We release code for these two models, which can be found at https://github.com/armatthews/ dependency-lm.2 In this work, we limit ourselves to models that are capable only of generating projective dependency trees.

show abstract

Section: Related Workmentioning

confidence: 99%

Comparing Top-Down and Bottom-Up Neural Generative Dependency Models

Matthews

Neubig

Dyer

2019

Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)

View full text Add to dashboard Cite

show abstract

“…Baselines for UD v2.0. We compare to the top performing models for EN, JA, VI, ZH from the CoNLL 2017 shared task: UDPipe 1.2 (Straka and Straková, 2017), Stanford (Dozat et al, 2017), FBAML (Qian and Liu, 2017), TRL (Kanayama et al, 2017), and IMS (Björkelund et al, 2017).…”

Section: Multilingual Experiments On Clean Datamentioning

confidence: 99%

Neural Semi-

Kemos¹,

Adel²,

Schütze³

2019

Proceedings of the 2019 Conference of the North

View full text Add to dashboard Cite

Character-level models of tokens have been shown to be effective at dealing with withintoken noise and out-of-vocabulary words. However, they often still rely on correct token boundaries. In this paper, we propose to eliminate the need for tokenizers with an end-toend character-level semi-Markov conditional random field. It uses neural networks for its character and segment representations. We demonstrate its effectiveness in multilingual settings and when token boundaries are noisy: It matches state-of-the-art part-of-speech taggers for various languages and significantly outperforms them on a noisy English version of a benchmark dataset. Our code and the noisy dataset are publicly available at http: //cistern.cis.lmu.de/semiCRF.

show abstract

“…Outside of the East Asian context, word segmentation-related research is focused mainly on languages with complex morphology and/or extensive compounding-such as Finnish, Turkish, German, Arabic and Hebrew-where splitting coarse-grained surface forms into smaller units leads to a significant reduction in the vocabulary size and thus lower proportion of out-of-vocabulary words [31][32][33][34][35]. Apart from that, even in languages normally using explicit word delimiters, there exist special types of text specific to the web domain, such as Uniform Resource Locators (URL) and hashtags, whose analysis requires the application of a word segmentation procedure [35,36].…”

Section: Related Workmentioning

confidence: 99%

MiNgMatch—A Fast N-gram Model for Word Segmentation of the Ainu Language

2019

View full text Add to dashboard Cite

Word segmentation is an essential task in automatic language processing for languages where there are no explicit word boundary markers, or where space-delimited orthographic words are too coarse-grained. In this paper we introduce the MiNgMatch Segmenter-a fast word segmentation algorithm, which reduces the problem of identifying word boundaries to finding the shortest sequence of lexical n-grams matching the input text. In order to validate our method in a low-resource scenario involving extremely sparse data, we tested it with a small corpus of text in the critically endangered language of the Ainu people living in northern parts of Japan. Furthermore, we performed a series of experiments comparing our algorithm with systems utilizing state-of-the-art lexical n-gram-based language modelling techniques (namely, Stupid Backoff model and a model with modified Kneser-Ney smoothing), as well as a neural model performing word segmentation as character sequence labelling. The experimental results we obtained demonstrate the high performance of our algorithm, comparable with the other best-performing models. Given its low computational cost and competitive results, we believe that the proposed approach could be extended to other languages, and possibly also to other Natural Language Processing tasks, such as speech recognition.

show abstract

IMS at the CoNLL 2017 UD Shared Task: CRFs and Perceptrons Meet Neural Networks

Cited by 24 publications

References 16 publications

Comparing Top-Down and Bottom-Up Neural Generative Dependency Models

Comparing Top-Down and Bottom-Up Neural Generative Dependency Models

Neural Semi-

MiNgMatch—A Fast N-gram Model for Word Segmentation of the Ainu Language

Contact Info

Product

Resources

About