Gated Recursive Neural Network for Chinese Word Segmentation

Abstract

Recently, neural network models for natural language processing tasks have been increasingly focused on for their ability of alleviating the burden of manual feature engineering. However, the previous neural models cannot extract the complicated feature compositions as the traditional methods with discrete features. In this paper, we propose a gated recursive neural network (GRNN) for Chinese word segmentation, which contains reset and update gates to incorporate the complicated combinations of the context characters. Since GRNN is relative deep, we also use a supervised layer-wise training method to avoid the problem of gradient diffusion. Experiments on the benchmark datasets show that our model outperforms the previous neural network models as well as the state-of-the-art methods.

Word-Context Character Embeddings for Chinese Word Segmentation

“…As shown in Table 4, pre-training with conventional skip-gram embeddings gives only small improvements, which is consistent as findings of previous work (Chen et al, 2015a;Ma and 2015; Cai and Zhao, 2016). Segmentation with self-training even shows accuracy drops on PKU and MSR.…”

Neural Word Segmentation Learning for Chinese

“…Pei et al (2014) improved upon (Zheng et al, 2013) by explicitly modeling the interactions between local context and previous tag. Chen et al (2015a) proposed a gated recursive neural network to model the feature combinations of context characters. Chen et al (2015b) used an LSTM architecture to capture potential long-distance dependencies, which alleviates the limitation of the size of context window but introduced another window for hidden states.…”

Corpus Expansion for Neural CWS on Microblog-Oriented Data with <i>λ</i>-Active Learning Approach

“…Pei et al [4] proposed a Max-Margin Tensor Neural Network (MMTNN) for CWS tasks, which can model complicated interactions between tags and context characters and speed up the model and avoid over-fitting. Chen et al [19] proposed a Gated Recursive Neural Network (GRNN) segmentation model, incorporating the complicated combinations of context characters by reset and update gates. In order to gain long-distance information, various long shortterm memory (LSTM) neural networks were proposed to get local and long-distance dependency information of current observed tokens, and the experimental results showed that the LSTM neural networks outperform other DNNs [20]- [22].…”