Solving large-scale support vector ordinal regression with asynchronous parallel coordinate descent algorithms

Gu, Bin; Geng, Xin; Shi, Wanli; Shan, Yingying; Huang, Yufang; Wang, Zhijie; Zheng, Guansheng

doi:10.1016/j.patcog.2020.107592

Cited by 11 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A plenty of research works are performed in ordinal regression [4,7,12,14,16,17,38]. Among these, Wenzhi et al [22] uses a neural network architecture with a non-traditional loss function which is particularly suited for the ordinal regression task.…”

Section: Related Workmentioning

confidence: 99%

ORDSIM: Ordinal Regression for E-Commerce Query Similarity Prediction

Kabir¹,

Hasan²,

Mandal³

et al. 2022

Preprint

View full text Add to dashboard Cite

Query similarity prediction task is generally solved by regression based models with square loss. Such a model is agnostic of absolute similarity values and it penalizes the regression error at all ranges of similarity values at the same scale. However, to boost e-commerce platform's monetization, it is important to predict high-level similarity more accurately than low-level similarity, as highly similar queries retrieves items according to user-intents, whereas moderately similar item retrieves related items, which may not lead to a purchase. Regression models fail to customize its loss function to concentrate around the high-similarity band, resulting poor performance in query similarity prediction task. We address the above challenge by considering the query prediction as an ordinal regression problem, and thereby propose a model, ORDSIM (ORDinal Regression for SIMilarity Prediction). ORDSIM exploits variable-width buckets to model ordinal loss, which penalizes errors in high-level similarity harshly, and thus enable the regression model to obtain better prediction results for high similarity values. We evaluate ORDSIM on a dataset of over 10 millions e-commerce queries from eBay platform and show that ORDSIM achieves substantially smaller prediction error compared to the competing regression methods on this dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

ORDSIM: Ordinal Regression for E-Commerce Query Similarity Prediction

Kabir¹,

Hasan²,

Mandal³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Therefore, both the algorithms and the evaluation metrics have to be adapted to face this problem and evaluate their solutions adequately. For this purpose, traditional methods such as support vector machines have been adapted to this problem by imposing constraints in order to respect the ordering of the classes (Chu & Keerthi, 2007;Gu et al, 2020), or by reducing to binary problems in an appropriate way (Lin & Li, 2012). Other proposals try to adapt conventional loss or gain functions in order to handle ordinal data in the best possible way.…”

Section: Ordinal Regression and Similarity Approachesmentioning

confidence: 99%

Ordinal regression with explainable distance metric learning based on ordered sequences

2021

View full text Add to dashboard Cite

The purpose of this paper is to introduce a new distance metric learning algorithm for ordinal regression. Ordinal regression addresses the problem of predicting classes for which there is a natural ordering, but the real distances between classes are unknown. Since ordinal regression walks a fine line between standard regression and classification, it is a common pitfall to either apply a regression-like numerical treatment of variables or underrate the ordinal information applying nominal classification techniques. On a different note, distance metric learning is a discipline that has proven to be very useful when improving distance-based algorithms such as the nearest neighbors classifier. In addition, an appropriate distance can enhance the explainability of this model. In our study we propose an ordinal approach to learning a distance, called chain maximizing ordinal metric learning. It is based on the maximization of ordered sequences in local neighborhoods of the data. This approach takes into account all the ordinal information in the data without making use of any of the two extremes of classification or regression, and it is able to adapt to data for which the class separations are not clear. We also show how to extend the algorithm to learn in a non-linear setup using kernel functions. We have tested our algorithm on several ordinal regression problems, showing a high performance under the usual evaluation metrics in this domain. Results are verified through Bayesian non-parametric testing. Finally, we explore the capabilities of our algorithm in terms of explainability using the case-based reasoning approach. We show these capabilities empirically on two different datasets, experiencing significant improvements over the case-based reasoning with the traditional Euclidean nearest neighbors.

show abstract

“…Ordinary regression has been successfully applied to a great number of real-world problems. Typical ordinary regression trains models in a batch manner, i.e., feeding all data into training model at once [6][7][8]. e batch training manner demands huge computation and memory for large-scale problems, and it is also not adaptable to streaming data.…”

Section: Learning To Rank In Online Ordinary Regressionmentioning

confidence: 99%

“…e lower interval endpoint can be defined similarly. Let L IMC I (f(x), θ, y l , y r ) denote the implicit constraints for ordering of thresholds in θ i , and I stands for the interval [8].…”

Section: Learning To Rank In Online Ordinary Regressionmentioning

confidence: 99%

“…Both approaches outperformed existing benchmarks when they were applied to ranking and multiclass classification. Furthermore, to accelerate the training, Gu et al [8] proposed asynchronous parallel coordinate descent algorithms. Besides, Berg et al [9] showed that training a classifier can improve neural network accuracy and proposed that using several discrete data representations simultaneously can improve neural network learning compared to a single representation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Perceptron Ranking Using Interval Labels with Ramp Loss for Online Ordinal Regression

Zhang

Liang

et al. 2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Due to its wide applications and learning efficiency, online ordinal regression using perceptron algorithms with interval labels (PRIL) has been increasingly applied to solve ordinal ranking problems. However, it is still a challenge for the PRIL method to handle noise labels, in which case the ranking results may change dramatically. To tackle this problem, in this paper, we propose noise-resilient online learning algorithms using ramp loss function, called PRIL-RAMP, and its nonlinear variant K-PRIL-RAMP, to improve the performance of PRIL method for noisy data streams. The proposed algorithms iteratively optimize the decision function under the framework of online gradient descent (OGD), and we justify the algorithms by showing the order preservation of thresholds. It is validated in the experiments that both approaches are more robust and efficient to noise labels than state-of-the-art online ordinal regression algorithms on real-world datasets.

show abstract

Solving large-scale support vector ordinal regression with asynchronous parallel coordinate descent algorithms

Cited by 11 publications

References 26 publications

ORDSIM: Ordinal Regression for E-Commerce Query Similarity Prediction

ORDSIM: Ordinal Regression for E-Commerce Query Similarity Prediction

Ordinal regression with explainable distance metric learning based on ordered sequences

Perceptron Ranking Using Interval Labels with Ramp Loss for Online Ordinal Regression

Contact Info

Product

Resources

About