A review on ranking problems in statistical learning

Abstract

Ranking problems define a widely spread class of statistical learning problems with many applications, including fraud detection, document ranking, medicine, credit risk screening, image ranking or media memorability. In this article, we systematically describe different types of non-probabilistic supervised ranking problems, i.e., ranking problems that require the prediction of an order of the response variables, and the corresponding loss functions resp. goodness criteria. We discuss the difficulties when trying to optimize those criteria. As for a detailed and comprehensive overview of existing machine learning techniques to solve such ranking problems, we group the suitable techniques into SVM-, tree-, Boosting and Neural Network-type approaches and recapitulate the corresponding optimization problems in a unified notation. We also discuss to which of the ranking problems the respective algorithms are tailored and identify their strengths and limitations. Computational aspects and open research problems are also considered.

Nonlinear classifiers for ranking problems based on kernelized SVM

“…The nonlinear Gaussian kernel significantly outperforms the linear kernel. This will be confirmed later in 21) and the right column corresponds to the Gaussian kernel (22).…”

“…In Figure 1 we present the PR curves for all methods with two different kernels evaluated on the FashionMNIST dataset. The left column corresponds to the linear kernel (21) while the right one to the Gaussian kernel (22) with σ = 0.01. The nonlinear Gaussian kernel significantly outperforms the linear kernel.…”

“…High scores predict positive labels while low scores predict negative labels. The rows of Figure 2 depict the linear (21) and the Gaussian kernels (22) with σ = 0.01 while each column corresponds to one method. The black vertical lines depict the top 5%-quantile of all scores (on the testing set).…”

“…This is a modification of the coordinate-wise dual ascent from [14]. For a review of other approaches see [3,22].…”

Quantitative robustness of instance ranking problems

“…where n − indicates the number of negatives in the data set. Since Clémençon and Vayatis [2007] introduced this loss function for binary-valued responses, Werner [2019b] suggested to define n − := (n − K) for continuously-valued responses since the top K instances can be identified with class 1 objects in this case. Denoting the index set of the true best K ≤ n instances by Best K and its empirical counterpart, i.e., the indices of the instances that have been predicted to be the best K ones, by Best K , one can rewrite the ranking loss by…”