The MIRACLE Team at the CLEF 2008 Multilingual Question Answering Track

Abstract

The MIRACLE team is a consortium formed by three universities from Madrid, (Universidad Politécnica de Madrid, Universidad Autónoma de Madrid and Universidad Carlos III de Madrid) and DAEADALUS, a small and medium size enterprise (SME). The MIRACLE team participated in the monolingual Spanish and crosslanguage French to Spanish subtasks at QA@CLEF 2008. For the Spanish subtask, we used an almost completely rebuilt version of our system, designed with the aim of flexibly combining information sources and linguistic annotators for different languages. To allow easy development for new languages, most of the modules don't make any language dependent assumptions. This language dependent knowledge is encapsulated in a rule language developed within the MIRACLE team. By the time of submitting the runs, work on the new version was still ongoing, so we consider the results as a partial test of the possibilities of the new architecture. Subsystems for other languages were not yet available, so we tried a very simple approach for the French to Spanish subtask: questions were translated to Spanish with Babylon, and the output of this translation was fed into our system. The results were an accuracy of 16% for the monolingual Spanish task and 5% for the crosslanguage task.

Passage retrieval in legal texts

“…The comparison of the PR systems used in the TREC 8 competition [12] shows that the best PR systems are those that are based on density, with the methods of IBM [13] and SiteQ [14] being the most effective ones. The PR systems that participated in the CLEF competition can also be split into these two categories, even if most of them were based on term overlap [15,16,17,18] rather than density [19,20].…”

Are Passages Enough? The MIRACLE Team Participation in QA@CLEF2009

“…The system architecture is based on the approach taken by the MIRACLE QA system participating in CLEF 2008 [2] and consists in a pipeline which analyzes questions, retrieves documents and performs answer extraction based on linguistic and semantic information. A rule engine has been used in the Question Classification, Answer Filter, Timex Analyzer and Topic Detection modules.…”

“…In this campaign, temporal management preserves the approach taken by our previous system [2]. This decision is based on later complementary work that was made in order to evaluate the QA system performance versus a baseline system without temporal management capabilities [8].…”