Can

Bian, Weijie; Wu, Kailun; Ren, Lejian; Pi, Qi; Zhang, Yujing; Xiao, Can; Sheng, Xiang-Rong; Zhu, Yong-Nan; Chan, Zhangming; Mou, Na; Luo, Xinchen; Xiang, Shiming; Zhou, Guorui; Zhu, Xiaoqiang; Deng, Hongbo

doi:10.1145/3488560.3498435

Cited by 41 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Different from the retrieval and the pre-ranking stage which usually takes billions or thousands of items into consideration, the ranking stage only needs to process hundreds of items. Therefore, the ranking models usually contain a large set of features describing the user, items, and context and use complicated architectures [1,11,18]. The ranking stage gives the predictions of different objectives by different ranking models.…”

Section: Multi-objective Ranking System With Cascade Architecturementioning

confidence: 99%

On Ranking Consistency of Pre-ranking Stage

Gu¹,

Sheng²,

Jiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Industrial ranking systems, such as advertising systems, rank items by aggregating multiple objectives into one final objective to satisfy user demand and commercial intent. Cascade architecture, composed of retrieval, pre-ranking, and ranking stages, is usually adopted to reduce the computational cost. Each stage may employ various models for different objectives and calculate the final objective by aggregating these models' outputs. The multi-stage ranking strategy causes a new problem -the ranked lists of the ranking stage and previous stages may be inconsistent. For example, items that should be ranked at the top of the ranking stage may be ranked at the bottom of previous stages. In this paper, we focus on the ranking consistency between the pre-ranking and ranking stages. Specifically, we formally define the problem of ranking consistency and propose the Ranking Consistency Score (RCS) metric for evaluation. We demonstrate that ranking consistency has a direct impact on online performance. Compared with the traditional evaluation manner that mainly focuses on the individual ranking quality of every objective, RCS considers the ranking consistency of the fused final objective, which is more proper for evaluation. CCS CONCEPTS• Information systems → Information retrieval.

show abstract

Section: Multi-objective Ranking System With Cascade Architecturementioning

confidence: 99%