Cross Lingual Question Answering Using QRISTAL for CLEF 2006

Abstract

Abstract. QRISTAL [9] is a question answering system making intensive use of natural language processing both for indexing documents and extracting answers. It ranked first in the EQueR evaluation campaign (Evalda, Technolangue [3]) and in CLEF 2005 for monolingual task (French-French) and multilingual task (English-French and Portuguese-French). This article describes the improvements of the system since last year. Then, it presents our benchmarked results for the CLEF 2006 campaign and a critical description of the system. Since Synapse Développement is participating to Quaero project, QRISTAL is most likely to be integrated in a mass market search engine in the forthcoming years.

FIDJI: using syntax for validating answers in multiple documents

“…Table 1 gives the results that our system obtained on CLEF 2005 (7 participants), CLEF 2006 (7 participants) and Quaero 2008 (3 participants) question sets and document collections. Table 2 provides results by question types and compares them to those obtained by QRISTAL at CLEF 2005and CLEF 2006(Laurent et al 2005, the best QA system for French for these campaigns. Quaero results are given in Table 3, as well as a comparison with overall results obtained by the best system during the official campaign (Quintard 2008).…”

“…Each answer must be associated with the passage from which it has been extracted (supporting passage). State-of-the-art best QA systems obtain close to 70% correct answers for English (Harabagiu and Bejan 2005), between 55 and 68% for French (Laurent et al 2005(Laurent et al , 2008. The main difficulty is to evaluate what a correct answer is and whether this answer is supported (or validated) by its supporting passage.…”

Automatic Semantic Analysis for NLP Applications

“…QA systems using an integrated approach include QUETAL (Frank et al 2005), JAVELIN (Nyberg et al 2003), COGEX (Moldovan et al 2003), PowerAnswer (Bowden et al 2006), START (Katz 1997), QRISTAL (Laurent et al 2005), and TACITUS (Hobbs et al 1993). Each of them has its special merits, but all of them use a shallow semantic representation such as, for instance, Minimal Recursion Semantics (Copestake et al 2006) with QUETAL.…”

Priberam’s Question Answering System for Portuguese

“…It is thus possible to operate in a cross-language environment, allowing, for instance, to obtain answers in French for questions formulated in Portuguese, or vice-versa. Synapse Développement carried out such an experiment and submitted a Portuguese-French run to this year's bilingual task of QA@CLEF [7], making use of Priberam's TRUST Portuguese module to analyse the test set of questions.…”