Enhanced answer type inference from questions using sequential models

Abstract

Question classification is an important step in factual question answering (QA) and other dialog systems. Several attempts have been made to apply statistical machine learning approaches, including Support Vector Machines (SVMs) with sophisticated features and kernels. Curiously, the payoff beyond a simple bag-ofwords representation has been small. We show that most questions reveal their class through a short contiguous token subsequence, which we call its informer span. Perfect knowledge of informer spans can enhance accuracy from 79.4% to 88% using linear SVMs on standard benchmarks. In contrast, standard heuristics based on shallow pattern-matching give only a 3% improvement, showing that the notion of an informer is non-trivial. Using a novel multi-resolution encoding of the question's parse tree, we induce a Conditional Random Field (CRF) to identify informer spans with about 85% accuracy. Then we build a meta-classifier using a linear SVM on the CRF output, enhancing accuracy to 86.2%, which is better than all published numbers.

Integrating Genetic Algorithms with Conditional Random Fields to Enhance Question Informer Prediction

“…Krishnan et al [3] also showed that the effect of the features chosen by a CRF model varies significantly depending on the accuracy of the CRF model. In a machine learning approach, feature selection is an optimization problem that involves choosing an appropriate feature subset.…”

“…Recent works [1,4,8] have shown that CRFs have a consistent advantage over traditional Hidden Markov Models (HMMs) and Maximum Entropy Markov Models (MEMMs) [6] in the face of many redundant features. Krishnan et al [3] reported that they achieved 85%-87% accuracy of question informer prediction by using CRF model with a set of features.…”

“…Question informers play an important role in enhancing question classification for factual question answering [3]. Krishnan et al [3] introduced the notion of the answer type informer span of a question for question classification and showed that human-annotated informer spans lead to large improvements in the accuracy of question classification.…”

“…Krishnan et al [3] introduced the notion of the answer type informer span of a question for question classification and showed that human-annotated informer spans lead to large improvements in the accuracy of question classification. They defined that choosing a minimal, appropriate contiguous span of a question token, or tokens, as the informer span of a question, which is adequate for question classification.…”

“…Previous works have used Conditional Random Fields (CRFs) to identify question informer spans. By using a parse of the question, Krishnan et al [3] derived a set of multiresolution features to train a CRF model and achieved 85%-87% accuracy for question informer prediction.…”