A passage-based approach to learning to rank documents

Sheetrit, Eilon; Shtok, Anna; Kurland, Oren

doi:10.1007/s10791-020-09369-x

Cited by 13 publications

(10 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Planned future improvements to Fxt include the addition of a benchmark tool to measure the computational cost of feature extraction so that research exploring efficiency and effectiveness tradeoffs, such as in cascade ranking [17,40], can be more accessible to new researchers in the community. Another interesting line of work involves adding passage level feature extraction, as they are useful for passage-based retrieval and in document retrieval [8,15,37].…”

Section: Resultsmentioning

confidence: 99%

Feature Extraction for Large-Scale Text Collections

Gallagher

Mallia

Culpepper

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Feature engineering is a fundamental but poorly documented component in Learning-to-Rank (LTR) search engines. Such features are commonly used to construct learning models for web and product search engines, recommender systems, and questionanswering tasks. In each of these domains, there is a growing interest in the creation of open-access test collections that promote reproducible research. However, there are still few open-source software packages capable of extracting high-quality machine learning features from large text collections. Instead, most featurebased LTR research relies on "canned" test collections, which often do not expose critical details about the underlying collection or implementation details of the extracted features. Both of these are crucial to collection creation and deployment of a search engine into production. So in this regard, the experiments are rarely reproducible with new features or collections, or helpful for companies wishing to deploy LTR systems. In this paper, we introduce Fxt, an open-source framework to perform efficient and scalable feature extraction. Fxt can easily be integrated into complex, high-performance software applications to help solve a wide variety of text-based machine learning problems. To demonstrate the software's utility, we build and document a reproducible feature extraction pipeline and show how to recreate several common LTR experiments using the ClueWeb09B collection. Researchers and practitioners can benefit from Fxt to extend their machine learning pipelines for various text-based retrieval tasks, and learn how some static document features and query-specific features are implemented.

show abstract

Section: Resultsmentioning

confidence: 99%

Feature Extraction for Large-Scale Text Collections

Gallagher

Mallia

Culpepper

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

show abstract

“…The visually relevant features of the input images are first derived by the exploitation of image descriptors, and different weights are then allocated to each feature to retrieve the relevant images-to-image query. Sheetrit et al [28] explored the passage-based information to improve document retrieval effectiveness. They investigated the use of learning-to-rankbased document retrieval methods that utilize a ranking of passages produced in response to the query.…”

Section: Earlier Ir Methodsmentioning

confidence: 99%

“…In this experiment, CRPM with PDRM [18] and JPD-LDR [28] were compared using the two big collections: Wikilinks for information retrieval and Football for hashtag retrieval. PDRM integrates swarm intelligence power and clustering techniques to solve the information retrieval problem, whereas JPD-LDR integrates the deep learning and decomposition techniques to satisfy user queries.…”

Section: Comparisons On Big Datamentioning

confidence: 99%

Cluster-based information retrieval using pattern mining

et al. 2020

View full text Add to dashboard Cite

This paper addresses the problem of responding to user queries by fetching the most relevant object from a clustered set of objects. It addresses the common drawbacks of cluster-based approaches and targets fast, high-quality information retrieval. For this purpose, a novel cluster-based information retrieval approach is proposed, named Cluster-based Retrieval using Pattern Mining (CRPM). This approach integrates various clustering and pattern mining algorithms. First, it generates clusters of objects that contain similar objects. Three clustering algorithms based on k-means, DBSCAN (Density-based spatial clustering of applications with noise), and Spectral are suggested to minimize the number of shared terms among the clusters of objects. Second, frequent and high-utility pattern mining algorithms are performed on each cluster to extract the pattern bases. Third, the clusters of objects are ranked for every query. In this context, two ranking strategies are proposed: i) Score Pattern Computing (SPC), which calculates a score representing the similarity between a user query and a cluster; and ii) Weighted Terms in Clusters (WTC), which calculates a weight for every term and uses the relevant terms to compute the score between a user query and each cluster. Irrelevant information derived from the pattern bases is also used to deal with unexpected user queries. To evaluate the proposed approach, extensive experiments were carried out on two use cases: the documents and tweets corpus. The results showed that the designed approach outperformed traditional and cluster-based information retrieval approaches in terms of the quality of the returned objects while being very competitive in terms of runtime.

show abstract

“…Since W2V1 is asymmetric, we also use W2V2, which is computed by simply switching between д and д ′ in W2V1. The next semantic similarity measure that we consider is adopted from work on passage and document retrieval [42]. The ESA (Explicit Semantic Analysis [14]) similarity is used by representing each n-gram using a vector that is defined over Wikipedia concepts and computing the cosine similarity between the vectors.…”

Section: Learning a Similaritymentioning

confidence: 99%

Recommending Search Queries in Documents Using Inter N-Gram Similarities

Sheetrit¹,

Fyodorov²,

Raiber³

et al. 2021

Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval

Self Cite

View full text Add to dashboard Cite

Reading a document can often trigger a need for additional information. For example, a reader of a news article might be interested in information about the persons and events mentioned in the article. Accordingly, there is a line of work on recommending search-engine queries given a document read by a user. Often, the recommended queries are selected from a query log independently of each other, and are presented to the user without any context. We address a novel query recommendation task where the recommended queries must be n-grams (sequences of consecutive terms) in the document. Furthermore, inspired by work on using inter-document similarities for document retrieval, we explore the merits of using inter n-gram similarities for query recommendation. Specifically, we use a supervised approach to learn an inter n-gram similarity measure where the goal is that n-grams that are likely to serve as queries will be deemed more similar to each other than to other n-grams. We use the similarity measure in a wide variety of query recommendation approaches which we devise as adaptations of ad hoc document retrieval techniques. Empirical evaluation performed using data gathered from Yahoo!'s search engine logs attests to the effectiveness of the resultant recommendation methods.

show abstract

A passage-based approach to learning to rank documents

Cited by 13 publications

References 64 publications

Feature Extraction for Large-Scale Text Collections

Feature Extraction for Large-Scale Text Collections

Cluster-based information retrieval using pattern mining

Recommending Search Queries in Documents Using Inter N-Gram Similarities

Contact Info

Product

Resources

About