Structured learning with constrained conditional models

Chang, Ming‐Wei; Ratinov, Lev; Roth, Dan

doi:10.1007/s10994-012-5296-5

Cited by 93 publications

(91 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One particular model, initially proposed by Joachims [12] for binary support vector machines, has given impressive results on problems involving large feature spaces, such as those encountered in text classification and natural language processing. This model has been nicely extended to multi-class classification, ordinal regression and structured output problems [31,5,2]. The key ideas behind these methods are: (a) using the labels of the unlabeled data (Y U ) as extra variables and the associated loss function in training; (b) optimizing the model weight vector (θ) and Y U via alternating optimization steps; (c) using constraints on Y U that come from domain knowledge to effectively guide the training towards good solutions; and (d) employing annealing to avoid getting caught in local minima.…”

Section: Introductionmentioning

confidence: 99%

Tractable Semi-supervised Learning of Complex Structured Prediction Models

Chang

Sellamanickam

Keerthi

2013

Advanced Information Systems Engineering

View full text Add to dashboard Cite

Abstract. Semi-supervised learning has been widely studied in the literature. However, most previous works assume that the output structure is simple enough to allow the direct use of tractable inference/learning algorithms (e.g., binary label or linear chain). Therefore, these methods cannot be applied to problems with complex structure. In this paper, we propose an approximate semi-supervised learning method that uses piecewise training for estimating the model weights and a dual decomposition approach for solving the inference problem of finding the labels of unlabeled data subject to domain specific constraints. This allows us to extend semi-supervised learning to general structured prediction problems. As an example, we apply this approach to the problem of multi-label classification (a fully connected pairwise Markov random field). Experimental results on benchmark data show that, in spite of using approximations, the approach is effective and yields good improvements in generalization performance over the plain supervised method. In addition, we demonstrate that our inference engine can be applied to other semi-supervised learning frameworks, and extends them to solve problems with complex structure.

show abstract

Section: Introductionmentioning

confidence: 99%

Tractable Semi-supervised Learning of Complex Structured Prediction Models

Chang

Sellamanickam

Keerthi

2013

Advanced Information Systems Engineering

View full text Add to dashboard Cite

show abstract

“…We approach this challenge using undirected probabilistic graphical models which integrate coherence constraints over pairs of slots within a schema. Similar techniques have been proposed for the more shallow problems of HMM-based sequence labeling by Chang et al [5] and relation extraction by Lopez de Lacalle & Lapata [12]. In line with the latter approach, we aim at inducing constraint knowledge automatically from training data.…”

Section: Related Workmentioning

confidence: 98%

Assessing the Impact of Single and Pairwise Slot Constraints in a Factor Graph Model for Template-Based Information Extraction

Horst

Hartung

Klinger

et al. 2018

Natural Language Processing and Information Systems

View full text Add to dashboard Cite

Abstract. Template-based information extraction generalizes over standard token-level binary relation extraction in the sense that it attempts to fill a complex template comprising multiple slots on the basis of information given in a text. In the approach presented in this paper, templates and possible fillers are defined by a given ontology. The information extraction task consists in filling these slots within a template with previously recognized entities or literal values. We cast the task as a structure prediction problem and propose a joint probabilistic model based on factor graphs to account for the interdependence in slot assignments. Inference is implemented as a heuristic building on Markov chain Monte Carlo sampling. As our main contribution, we investigate the impact of soft constraints modeled as single slot factors which measure preferences of individual slots for ranges of fillers, as well as pairwise slot factors modeling the compatibility between fillers of two slots. Instead of relying on expert knowledge to acquire such soft constraints, in our approach they are directly captured in the model and learned from training data. We show that both types of factors are effective in improving information extraction on a real-world data set of full-text papers from the biomedical domain. Pairwise factors are shown to particularly improve the performance of our extraction model by up to +0.43 points in precision, leading to an F1 score of 0.90 for individual templates.

show abstract

“…We followed the settings in [3], where the HMM is assumed to be used for identifying the role, such as 'author' and 'title', of each word in a given citation text snippet. The sequential labeling task correspond to finding the Viterbi path of the HMM.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…We trained the HMM with 10 training samples and used 99 samples as test data. Both sets were obtained from [3]. There are 12 types of constraints.…”

Section: Experimental Settingsmentioning

confidence: 99%

“…In the same way, we can solve constrained version of these combinatorial optimization problems via CDSP. These include problems of practical importance such as the 0-1 knapsack problem with disjunctive constraints [1], sequence alignment with user-defined anchor points [2], and incorporating knowledge based constraints into statistical machine learning models [3].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

BDD-Constrained A<sup>*</sup> Search: A Fast Method for Solving Constrained Shortest-Path Problems

Fumito¹,

Nishino

Yasuda

et al. 2017

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper deals with the constrained DAG shortest path problem (CDSP), which finds the shortest path on a given directed acyclic graph (DAG) under any logical constraints posed on taken edges. There exists a previous work that uses binary decision diagrams (BDDs) to represent the logical constraints, and traverses the input DAG and the BDD simultaneously. The time and space complexity of this BDD-based method is derived from BDD size, and tends to be fast only when BDDs are small. However, since it does not prioritize the search order, there is considerable room for improvement, particularly for large BDDs. We combine the well-known A * search with the BDD-based method synergistically, and implement several novel heuristic functions. The key insight here is that the 'shortest path' in the BDD is a solution of a relaxed problem, just as the shortest path in the DAG is. Experiments, particularly practical machine learning applications, show that the proposed method decreases search time by up to 2 orders of magnitude, with the specific result that it is 2,000 times faster than a commercial solver. Moreover, the proposed method can reduce the peak memory usage up to 40 times less than the conventional method.

show abstract

Structured learning with constrained conditional models

Cited by 93 publications

References 30 publications

Tractable Semi-supervised Learning of Complex Structured Prediction Models

Tractable Semi-supervised Learning of Complex Structured Prediction Models

Assessing the Impact of Single and Pairwise Slot Constraints in a Factor Graph Model for Template-Based Information Extraction

BDD-Constrained A<sup>*</sup> Search: A Fast Method for Solving Constrained Shortest-Path Problems

Contact Info

Product

Resources

About