Elhuyar-IXA: Semantic Relatedness and Cross-Lingual Passage Retrieval

Agirre, Eneko; Ansa, Olatz; Arregi, Xabier; Lacalle, Maddalen Lopez de; Otegi, Arantxa; Saralegi, Xabier; Zaragoza, Hugo

doi:10.1007/978-3-642-15754-7_31

Cited by 1 publication

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ResPubliQA: Given a pool of 500 independent questions in natural language, systems must return the passage -not the exact answer -that answers each question. The document collection is JRC-Acquis about EU documentation 1 . Both questions and documents are translated into and aligned for a subset of official European languages, i.e.…”

Section: Introductionmentioning

confidence: 99%

Overview of ResPubliQA 2009: Question Answering Evaluation over European Legislation

Peñas

Forner²,

Sutcliffe

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper describes the first round of ResPubliQA, a Question Answering (QA) evaluation task over European legislation, proposed at the Cross Language Evaluation Forum (CLEF) 2009. The exercise consists of extracting a relevant paragraph of text that satisfies completely the information need expressed by a natural language question. The general goals of this exercise are (i) to study if the current QA technologies tuned for newswire collections and Wikipedia can be adapted to a new domain (law in this case); (ii) to move to a more realistic scenario, considering people close to law as users, and paragraphs as system output; (iii) to compare current QA technologies with pure Information Retrieval (IR) approaches; and (iv) to introduce in QA systems the Answer Validation technologies developed in the past three years. The paper describes the task in more detail, presenting the different types of questions, the methodology for the creation of the test sets and the new evaluation measure, and analyzing the results obtained by systems and the more successful approaches. Eleven groups participated with 28 runs. In addition, we evaluated 16 baseline runs (2 per language) based only in pure IR approach, for comparison purposes. Considering accuracy, scores were generally higher than in previous QA campaigns.

show abstract

Section: Introductionmentioning

confidence: 99%