Effective self-training for parsing

Abstract

We present a simple, but surprisingly effective, method of self-training a twophase parser-reranker system using readily available unlabeled data. We show that this type of bootstrapping is possible for parsing when the bootstrapped parses are processed by a discriminative reranker. Our improved model achieves an f-score of 92.1%, an absolute 1.1% improvement (12% error reduction) over the previous best result for Wall Street Journal parsing. Finally, we provide some analysis to better understand the phenomenon.

Semi-supervised Structured Prediction with Neural CRF Autoencoder

“…The difficulty of providing sufficient supervision has motivated work on semi-supervised and unsupervised learning for many of these tasks (McClosky et al, 2006;Spitkovsky et al, 2010;Subramanya et al, 2010;Stratos and Collins, 2015;Marinho et al, 2016;Tran et al, 2016), including several that also used autoencoders (Ammar et al, 2014;Lin et al, 2015;Miao and Blunsom, 2016;Kociský et al, 2016;Cheng et al, 2017). In this paper we expand on these works, and suggest a neural CRF autoencoder, that can leverage both labeled and unlabeled data.…”

Self-Trained Bilexical Preferences to Improve Disambiguation Accuracy

“…Although naively adding self-labeled material to extend training data is normally not successful, there have been successful variants of self-learning for parsing as well. For instance, in [16] self-learning is used to improve a twophase parser reranker, with very good results for the classical Wall Street Journal parsing task.…”

Semi-supervised model adaptation for statistical machine translation

“…McClosky et al (2006a) introduces self-training techniques for two-step parsers. In McClosky et al (2006b), these methods are then used to adapt a parser trained on Wall Street Journal data, without using labeled data from the latter domain.…”