The Effect of Entity Recognition on Answer Validation

Abstract

The Answer Validation Exercise (AVE) 2006 is aimed at developing systems able to decide whether the answer of a Question Answering (QA) system is correct or not using textual entailment. The most answers to be validated are given from questions that need entities as responses. The paper presents a system that has only used entities to participate in the AVE 2006. The results of the propose system are better than the ones of a baseline system that always accepts all answers, therefore the use of entities can improve the results of an answer validation system. KeywordsQuestion Answering, Answer Validation, Textual Entailment, Entity Recognition MotivationThe Answer Validation Exercise (AVE) [6] of the Cross Language Evaluation Forum (CLEF) 2006 is aimed at developing systems able to decide whether the answer of a Question Answering (QA) system is correct or not using textual entailment [2]. The test corpus has hypothesis-text pairs where the hypothesis contains the response from a QA system and the text is the snippet given by the system to support its answer. Participant systems must return YES or NO for each hypothesis-text pair to indicate if the text entails the hypothesis or not (i.e. the answer is correct according to the text).The paper shows a participant system that only uses information about entities (numeric expressions, temporal expressions and named entities) in order to study the importance of entities in answer validation.The motivation for this experiment comes from the study of the QA task in the CLEF 2005, where 75% of the questions in Spanish were factoids [7]. The responses to factoid questions contain entities (e.g. person names, locations, numbers, dates...). In this way and following the methodology to build the test corpus in the AVE, the most of the pairs will contain entities. For this reason, we think it is important the study of entities in this task.In this experiment, we have used the hypothesis that in the recognition of textual entailment all the elements in the hypothesis must be entailed by elements of the text. We consider this hypothesis mainly must happen with the entities due to entities contain the main information about the hypothesis-text pairs.

UNED at Answer Validation Exercise 2007

“…In order to avoid errors in the process of named entities entailment, as it is explained in [7], all named entities receive the same tag NE ignoring the named entity categorization given by the tool.…”

“…In [6] and [7], we detected the entailment relation between named entities in the text and in the hypothesis. In AVE 2007 [4], this is no possible due to the fact that none hypothesis is given.…”

“…The system we have presented is based on the one presented to AVE 2006 [7], which gave good results in Spanish, and the one used in our participation at RTE-3 [6], which obtained also good results in textual entailment over pairs from QA. These two systems were based on named entities (NE).…”

A Supervised Learning Approach to Spanish Answer Validation

“…Representative methods of this approach determine that H (the hypothesis) is entailed from T (the support text) only considering characteristics such as named entity overlaps [2], n-gram overlaps and the size of the longest common subsequence (LCS) [3].…”

“…The proposed system adopts several ideas from recent systems (in particular from [2,3]): it is based on a supervised learning approach that considers a combination of some previously-used features. However, in addition, it also includes some new characteristics that allow tackling the discussed problem.…”