Inducing Word and Part-of-Speech with Pitman-Yor Hidden Semi-Markov Models

Abstract

We propose a nonparametric Bayesian model for joint unsupervised word segmentation and part-of-speech tagging from raw strings. Extending a previous model for word segmentation, our model is called a Pitman-Yor Hidden Semi-Markov Model (PYHSMM) and considered as a method to build a class n-gram language model directly from strings, while integrating character and word level information. Experimental results on standard datasets on Japanese, Chinese and Thai revealed it outperforms previous results to yield the state-of-the-art accuracies. This model will also serve to analyze a structure of a language whose words are not identified a priori.

Towards easier and faster sequence labeling for natural language processing: A search-based probabilistic online learning framework (SAPO)

“…The natural language processing tasks include (1) Part-of-Speech Tagging (POS-Tag): Part-of-Speech (POS) tagging is an important and highly competitive task in natural language processing. We use the standard benchmark dataset in prior work [5,40], which is derived from raw features in total. The evaluation metric is balanced F-score.…”

A Hybrid Generative/Discriminative Model for Rapid Prototyping of Domain-Specific Named Entity Recognition

“…The model proposed in this paper has a close connection to unsupervised word segmentation and part-of-speech (POS) induction [6]. A key difference is that, while they use characters as the unit for the input sequence, we utilize word sequences.…”

“…Uchiumi et al [6] can be seen as an extension to Mochihashi et al [35], who focused on unsupervised word segmentation. They proposed a nonparametric Bayesian n-gram language model based on Pitman-Yor processes.…”

“…Given an unsegmented corpus, the model infers word segmentation using Gibbs sampling. Uchiumi et al [6] worked on the joint task of unsupervised word segmentation and POS induction. We employ their model, PYHSMM, for our task.…”

“…Regarding the technical aspects, the proposed tagger is a hybrid of a generative model and a discriminative model. The generative model called the Pitman-Yor hidden semi-Markov model (PYHSMM) [6] recognizes high-frequency word sequences as NE chunks and identifies their classes. The discriminative model, semi-Markov CRF (semiCRF) [7], initializes the learning process using the seed terms and generalizes to other NEs of the same classes.…”