Toward creating a fairer ranking in search engine results

Gao, Ruoyuan; Shah, Chirag

doi:10.1016/j.ipm.2019.102138

Cited by 86 publications

(72 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Gao and Shah [15] recently proposed a randomized policy for diversifying search results very similar to our work, albeit in the context of group fairness. While studying connection between fairness and diversity empirically, we attempt to more formally elucidate the relationship and study broader connections beyond group fairness.…”

Section: Fairnessmentioning

confidence: 83%

Evaluating Stochastic Rankings with Expected Exposure

Dı́az

Mitra

Ekstrand

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

127

140

View full text Add to dashboard Cite

We introduce the concept of expected exposure as the average attention ranked items receive from users over repeated samples of the same query. Furthermore, we advocate for the adoption of the principle of equal expected exposure: given a fixed information need, no item should receive more or less expected exposure than any other item of the same relevance grade. We argue that this principle is desirable for many retrieval objectives and scenarios, including topical diversity and fair ranking. Leveraging user models from existing retrieval metrics, we propose a general evaluation methodology based on expected exposure and draw connections to related metrics in information retrieval evaluation. Importantly, this methodology relaxes classic information retrieval assumptions, allowing a system, in response to a query, to produce a distribution over rankings instead of a single fixed ranking. We study the behavior of the expected exposure metric and stochastic rankers across a variety of information access conditions, including ad hoc retrieval and recommendation. We believe that measuring and optimizing expected exposure metrics using randomization opens a new area for retrieval algorithm development and progress. CCS CONCEPTS • Information systems → Evaluation of retrieval results; Learning to rank.

show abstract

Section: Fairnessmentioning

confidence: 83%

Evaluating Stochastic Rankings with Expected Exposure

Dı́az

Mitra

Ekstrand

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

127

140

View full text Add to dashboard Cite

show abstract

“…The presentation of web search results should help the users to obtain the desired information as efficiently as possible [8][9][10]. Therefore, the search interface should have a minimalist design and avoid unnecessary elements that can distract users [11].…”

Section: Introductionmentioning

confidence: 99%

Can search result summaries enhance the web search efficiency and experiences of the visually impaired users?

Aqle

Al-Thani

Jaoua

2020

Univ Access Inf Soc

View full text Add to dashboard Cite

There are limited studies that are addressing the challenges of visually impaired (VI) users when viewing search results on a search engine interface by using a screen reader. This study investigates the effect of providing an overview of search results to VI users. We present a novel interactive search engine interface called InteractSE to support VI users during the results exploration stage in order to improve their interactive experience and web search efficiency. An overview of the search results is generated using an unsupervised machine learning approach to present the discovered concepts via a formal concept analysis that is domain-independent. These concepts are arranged in a multi-level tree following a hierarchical order and covering all retrieved documents that share maximal features. The InteractSE interface was evaluated by 16 legally blind users and compared with the Google search engine interface for complex search tasks. The evaluation results were obtained based on both quantitative (as task completion time) and qualitative (as participants’ feedback) measures. These results are promising and indicate that InteractSE enhances the search efficiency and consequently advances user experience. Our observations and analysis of the user interactions and feedback yielded design suggestions to support VI users when exploring and interacting with search results.

show abstract

“…The success of the proposed frameworks in this paper revealed that explicit diversification performance can be considerably further improved using supervised learning approaches that do not require very large training sets and/or excessive computing resources (contrary to the popular deep learning paradigm of our day), and hence, they are applicable in real life scenarios that require diversity (and even fairness, as in (Gao & Shah, 2020;McDonald et al, 2019)) among the results of a search system. In our future work, we plan to investigate ways of employing alternative diversification metrics as the objective function in the LmDiv framework.…”

Section: Discussionmentioning

confidence: 91%

Supervised approaches for explicit search result diversification

Yigit-Sert

Altıngövde

Macdonald

et al. 2020

Information Processing & Management

View full text Add to dashboard Cite

Diversification of web search results aims to promote documents with diverse content (i.e., covering di↵erent aspects of a query) to the top-ranked positions, to satisfy more users, enhance fairness and reduce bias. In this work, we focus on the explicit diversification methods, which assume that the query aspects are known at the diversification time, and leverage supervised learning methods to improve their performance in three di↵erent frameworks with di↵erent features and goals. First, in the LTRDiv framework, we focus on applying typical learning to rank (LTR) algorithms to obtain a ranking where each top-ranked document covers as many aspects as possible. We argue that such rankings optimize various diversification metrics (under certain assumptions), and hence, are likely to achieve diversity in practice. Second, in the AspectRanker framework, we apply LTR for ranking the aspects of a query with the goal of more accurately setting the aspect importance values for diversification. As features, we exploit several pre-and postretrieval query performance predictors (QPPs) to estimate how well a given aspect is covered among the candidate documents. Finally, in the LmDiv framework, we cast the diversification problem into an alternative fusion task, namely, the supervised merging of rankings per query aspect. We again use QPPs computed over the candidate set for each aspect, and optimize an objective function that is tailored for the diversification goal. We conduct thorough comparative experiments using both the basic systems (based on the well-known BM25 matching function) and the best-performing systems (with more sophisticated retrieval methods) from previous TREC campaigns. Our findings reveal that the proposed frameworks, especially AspectRanker and LmDiv, outperform both non-diversified rankings and two strong diversification baselines (i.e., xQuAD and its variant) in terms of various e↵ectiveness metrics.

show abstract

Toward creating a fairer ranking in search engine results

Cited by 86 publications

References 22 publications

Evaluating Stochastic Rankings with Expected Exposure

Evaluating Stochastic Rankings with Expected Exposure

Can search result summaries enhance the web search efficiency and experiences of the visually impaired users?

Supervised approaches for explicit search result diversification

Contact Info

Product

Resources

About