Quantitative evaluation of passage retrieval algorithms for question answering

Tellex, Stefanie; Katz, Boris; Lin, Jimmy; Fernandes, Aaron; Marton, Gregory

doi:10.1145/860435.860445

Cited by 176 publications

(87 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this study, we employed five high-performance yet portable ranking models, namely, Okapi BM-25 [27], [42], [46], INQUERY [33], [34], language model (LM) [28], [29], and SiteQ's [35] approaches that were implemented in the Lemur toolkit [51] (with the exception of SiteQ's method for comparison.) Our previous method (suffix-tree-based [31]) was also included.…”

Section: B Experimental Resultsmentioning

confidence: 99%

“…Tellex et al [42] had compared seven passage retrieval algorithms such as BM-25 [27], and SiteQ [35] for the TREC-Q/A task; however they excluded two ad-hoc methods which needed human-generated patterns and ontology. They reported that the term weighting methods such as BM-25 [27] were slightly worse than the SiteQ's approach [35] for integration with external resources such as named entity recognizers, WordNet, and thesauruses.…”

Section: Finding Optimal String Patterns For Weightingmentioning

confidence: 99%

See 1 more Smart Citation

A Robust Passage Retrieval Algorithm for Video Question Answering

Yang

2008

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

In this paper, we present a robust passage retrieval algorithm to extend the conventional text question answering (Q/A) to videos. Users interact with our videoQ/A system through natural language queries, while the top-ranked passage fragments with associated video clips are returned as answers. We compare our method with five of the high-performance ranking algorithms that are portable to different languages and domains. The experiments were evaluated with 75.3 h of Chinese videos and 253 questions. The experimental results showed that our method outperformed the second best retrieval model (language models) in relatively 1.43% in mean reciprocal rank (MRR) score and 11.36% when employing a Chinese word segmentation tool. By adopting the initial retrieval results from the retrieval models, our method yields an improvement of at least 5.94% improvement in MRR score. This makes it very attractive for the Asia-like languages since the use of a well-developed word tokenizer is unnecessary. Index Terms-Multimedia retrieval, question answering (Q/A), video question answering (videoQ/A).

show abstract

Section: B Experimental Resultsmentioning

confidence: 99%

Section: Finding Optimal String Patterns For Weightingmentioning

confidence: 99%

A Robust Passage Retrieval Algorithm for Video Question Answering

Yang

2008

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…For example, instead of returning documents in response to a query one can return passages that supposedly contain pertaining information (Denoyer et al 2001;Allan 2003;Jiang and Zhai 2004;Murdock and Croft 2005;Wade and Allan 2005). Retrieving passages is also common in work on question answering (Corrada-Emmanuel et al 2003;Lin et al 2003;Tellex et al 2003;Hussain 2004;Zhang and Lee 2004;Otterbacher et al 2005) wherein answers are being extracted (or compiled) from passages that are deemed relevant to the question at hand. While we demonstrate the merits in using our proposed passage language model for ad hoc document retrieval, it can also potentially be used in any task that calls for passage retrieval.…”

Section: Related Workmentioning

confidence: 99%

Utilizing passage-based language models for ad hoc document retrieval

Bendersky

Kurland

2009

Inf Retrieval

View full text Add to dashboard Cite

To cope with the fact that, in the ad hoc retrieval setting, documents relevant to a query could contain very few (short) parts (passages) with query-related information, researchers proposed passage-based document ranking approaches. We show that several of these retrieval methods can be understood, and new ones can be derived, using the same probabilistic model. We use language-model estimates to instantiate specific retrieval algorithms, and in doing so present a novel passage language model that integrates information from the containing document to an extent controlled by the estimated document homogeneity. Several document-homogeneity measures that we present yield passage language models that are more effective than the standard passage model for basic document retrieval and for constructing and utilizing passage-based relevance models; these relevance models also outperform a document-based relevance model. Finally, we demonstrate the merits in using the document-homogeneity measures for integrating document-query and passage-query similarity information for document retrieval.Keywords Ad hoc document retrieval Á Passage-based language models Á Document homogeneity Á Relevance models Á Passage-based relevance models Portions of this work appeared in Bendersky and Kurland (2008b) and Bendersky (2007).

show abstract

“…Tellex et al [4] have made a thorough quantitative evaluation of passage retrieval algorithms. Multitext [5] system takes a density-based algorithm that retrieve short passages which contain query terms with high idf values.…”

Section: Related Workmentioning

confidence: 99%

Automatic question answering from Web documents

et al. 2007

Wuhan Univ. J. of Nat. Sci.

View full text Add to dashboard Cite

A passage retrieval strategy for web-based question answering (QA) systems is proposed in our QA system. It firstly analyzes the question based on semantic patterns to obtain its syntactic and semantic information and then form initial queries. The queries are used to retrieve documents from the World Wide Web (WWW) using the Google search engine. The queries are then rewritten to form queries for passage retrieval in order to improve the precision. The relations between keywords in the question are employed in our query rewrite method. The experimental result on the question set of the TREC-2003 passage task shows that our system performs well for factoid questions.

show abstract

Quantitative evaluation of passage retrieval algorithms for question answering

Cited by 176 publications

References 11 publications

A Robust Passage Retrieval Algorithm for Video Question Answering

A Robust Passage Retrieval Algorithm for Video Question Answering

Utilizing passage-based language models for ad hoc document retrieval

Automatic question answering from Web documents

Contact Info

Product

Resources

About