Discriminative unsupervised learning of structured predictors

Abstract

We present a new unsupervised algorithm for training structured predictors that is discriminative, convex, and avoids the use of EM. The idea is to formulate an unsupervised version of structured learning methods, such as maximum margin Markov networks, that can be trained via semidefinite programming. The result is a discriminative training criterion for structured predictors (like hidden Markov models) that remains unsupervised and does not create local minima. To reduce training cost, we reformulate the training procedure to mitigate the dependence on semidefinite programming, and finally propose a heuristic procedure that avoids semidefinite programming entirely. Experimental results show that the convex discriminative procedure can produce better conditional models than conventional Baum-Welch (EM) training.

Collective Classification in Network Data

“…More recently, there have been some attempts to extend collective classification techniques to the semi-supervised learning scenario [62,37].…”

Tractable Semi-supervised Learning of Complex Structured Prediction Models

“…Several semi-supervised learning algorithms [2,23,36,40,1,34,7] have been proposed in the literature. However, with a few exceptions, they all focus on the cases that exact inference and learning are tractable.…”

Discovering Latent Clusters from Geotagged Beach Images

“…Using the same reasoning of unsupervised SVM [18,19], we can try to solve the following optimization problem:…”

“…The optimization problem in Eq. 3 tries to find {z i } so that the resultant SVM has the maximum margin (please refer to [18,19] for details). Unfortunately, without additional constraints or regularization, Eq.…”

“…Basically we can assign all training data to the same cluster and learn w to achieve arbitrarily large margin. In [18,19], this problem is addressed by adding a constraint that tries to make sure that the clusters are balanced.…”