Chinese Morphological Analysis with Character-level POS Tagging

Shen, Mo; Liu, Hongxiao; Kawahara, Daisuke; Kurohashi, Sadao

doi:10.3115/v1/p14-2042

Cited by 13 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to space-delimited languages like English, word segmentation is the first and most crucial step for natural language processing (NLP) in unsegmented languages like Japanese, Chinese, and Thai (Kudo et al, 2004;Kaji and Kitsuregawa, 2014;Shen et al, 2014;Kruengkrai et al, 2006). Word segmentation is usually performed jointly with related analysis: POS tagging for Chinese, and POS tagging and lemmatization (analysis of inflected words) for Japanese.…”

Section: Introductionmentioning

confidence: 99%

Morphological Analysis for Unsegmented Languages using Recurrent Neural Network Language Model

Morita

Kawahara²,

Kurohashi

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

We present a new morphological analysis model that considers semantic plausibility of word sequences by using a recurrent neural network language model (RNNLM). In unsegmented languages, since language models are learned from automatically segmented texts and inevitably contain errors, it is not apparent that conventional language models contribute to morphological analysis. To solve this problem, we do not use language models based on raw word sequences but use a semantically generalized language model, RNNLM, in morphological analysis. In our experiments on two Japanese corpora, our proposed model significantly outperformed baseline models. This result indicates the effectiveness of RNNLM in morphological analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

Morphological Analysis for Unsegmented Languages using Recurrent Neural Network Language Model

Morita

Kawahara²,

Kurohashi

2015

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…English sentences are first parsed by nlparser (Charniak and Johnson, 2005) and then converted into word dependency trees using Collins' head percolation table (Collins, 1999). We used Chinese word segmenter KKN (Shen et al, 2014) and dependency parser SKP (Shen et al, 2012) for Chinese sentences. The supervised word alignment Nile (Riesa et al, 2011) was used.…”

Section: Methodsmentioning

confidence: 99%

Insertion Position Selection Model for Flexible Non-Terminals in Dependency Tree-to-Tree Machine Translation

Nakazawa

Richardson

Kurohashi

2016

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

Self Cite

View full text Add to dashboard Cite

Dependency tree-to-tree translation models are powerful because they can naturally handle long range reorderings which is important for distant language pairs. The translation process is easy if it can be accomplished only by replacing non-terminals in translation rules with other rules. However it is sometimes necessary to adjoin translation rules. Flexible non-terminals have been proposed as a promising solution for this problem. A flexible non-terminal provides several insertion position candidates for the rules to be adjoined, but it increases the computational cost of decoding. In this paper we propose a neural network based insertion position selection model to reduce the computational cost by selecting the appropriate insertion positions. The experimental results show the proposed model can select the appropriate insertion position with a high accuracy. It reduces the decoding time and improves the translation quality owing to reduced search space.

show abstract

“…These models learn features automatically which alleviate the efforts in feature engineering. However, joint S&T is a more difficult task than Chinese word segmentation and POS tagging since it has a larger decoding space and need more contextual information and long distance dependency (Zhang and Clark, 2008;Jiang et al, 2008;Kruengkrai et al, 2009;Zhang and Clark, 2010;Sun, 2011;Qian and Liu, 2012;Zheng et al, 2013;Qiu et al, 2013;Shen et al, 2014). Therefore, we need a customized architecture to alleviate these problems.…”

Section: Related Workmentioning

confidence: 99%

A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

Chen

Qiu

Huang

2017

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Recently, neural network models for natural language processing tasks have been increasingly focused on for their ability of alleviating the burden of manual feature engineering. However, the previous neural models cannot extract the complicated feature compositions as the traditional methods with discrete features. In this work, we propose a featureenriched neural model for joint Chinese word segmentation and part-of-speech tagging task. Specifically, to simulate the feature templates of traditional discrete feature based models, we use different filters to model the complex compositional features with convolutional and pooling layer, and then utilize long distance dependency information with recurrent layer. Experimental results on five different datasets show the effectiveness of our proposed model.

show abstract

Chinese Morphological Analysis with Character-level POS Tagging

Cited by 13 publications

References 10 publications

Morphological Analysis for Unsegmented Languages using Recurrent Neural Network Language Model

Morphological Analysis for Unsegmented Languages using Recurrent Neural Network Language Model

Insertion Position Selection Model for Flexible Non-Terminals in Dependency Tree-to-Tree Machine Translation

A Feature-Enriched Neural Model for Joint Chinese Word Segmentation and Part-of-Speech Tagging

Contact Info

Product

Resources

About