A convergent and fully distributable SVMs training algorithm

Manno, Andrea; Sagratella, Simone; Livi, Lorenzo

doi:10.1109/ijcnn.2016.7727590

Cited by 10 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the overall Algorithm 1, executing iteratively a distributed minimization over the variables x, can be viewed itself as a distributed algorithm (see e.g. [31,37]). Nevertheless, solving an (inexact) optimization problem at each iteration may be numerically inefficient.…”

Section: Algorithm 1: Basic Augmented Lagrangian Methodsmentioning

confidence: 99%

“…The latter is the case of support vector machines training, because in its dual quadratic formulation a column of the matrix Q requires in general O(n 2 ) nonlinear calculations [28,30]. Nonetheless, we underline that the above scheme is not directly applicable to the support vector machines quadratic formulation due to the presence of a coupling linear constraint (m = 1) [31].…”

Section: Motivationmentioning

confidence: 99%

“…In particular, in the dual formulation of the training problem of support vector machines the presence of a linear equality constraint (i.e. a coupling constraint) over all the variables makes the distribution of the data uneasy [26,28,30,31,33].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Distributed algorithms for convex problems with linear coupling constraints

Colombo

Sagratella

2019

J Glob Optim

Self Cite

View full text Add to dashboard Cite

Distributed and parallel algorithms have been frequently investigated in the recent years, in particular in applications like machine learning. Nonetheless, only a small subclass of the optimization algorithms in the literature can be easily distributed, for the presence, e.g., of coupling constraints that make all the variables dependent from each other with respect to the feasible set. Augmented Lagrangian methods are among the most used techniques to get rid of the coupling constraints issue, namely by moving such constraints to the objective function in a structured, well-studied manner. Unfortunately, standard augmented Lagrangian methods need the solution of a nested problem by needing to (at least inexactly) solve a subproblem at each iteration, therefore leading to potential inefficiency of the algorithm. To fill this gap, we propose an augmented Lagrangian method to solve convex problems with linear coupling constraints that can be distributed and requires a single gradient projection step at every iteration. We give a formal convergence proof to at least ε-approximate solutions of the problem and a detailed analysis of how the parameters of the algorithm influence the value of the approximating parameter ε. Furthermore, we introduce a distributed version of the algorithm allowing to partition the data and perform the distribution of the computation in a parallel fashion.

show abstract

Section: Algorithm 1: Basic Augmented Lagrangian Methodsmentioning

confidence: 99%

Section: Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Distributed algorithms for convex problems with linear coupling constraints

Colombo

Sagratella

2019

J Glob Optim

Self Cite

View full text Add to dashboard Cite

show abstract

“…Generally, according to the type of ML architecture, the training phase is formulated as a specific optimization problem. Among all existing architectures, some of the most popular are the Support Vector Machines (SVMs) [18], [19] and the Artificial Neural Networks (ANNs) [20]. While SVMs are mainly used for classification tasks, ANNs are used for both classification and regression.…”

Section: An Overview On Supervised Learning and Neural Networkmentioning

confidence: 99%

Key-Cost Drivers Selection in Local Public Bus Transport Services Through Machine Learning

Avenali

Catalano

D’Alfonso

et al. 2017

WIT Transactions on the Built Environment

Self Cite

View full text Add to dashboard Cite

This paper is aimed at developing a workable model for the identification of key-cost drivers in the Italian Local Public Bus Transport (LPBT) sector. Disaggregated information about costs, technical characteristics and environmental characteristics have been collected by means of questionnaires sent to LPBT companies producing more than 500 million bus revenue kilometres in Italy in 2011. A supervised regression model is built by training a regularized Artificial Neural Network in order to determine the quantitative and qualitative characteristics that contribute to explaining the variability of the driving personnel and the unit cost of the fleet (which usually covers more than 50% of the total economic cost) and the remaining portion of the unit cost. The proposed models could be an effective and simple tool for local authorities to validate reserve prices in tender procedures. Keywords: standard costs, local public transport, fiscal federalism, cost drivers, machine learning. INTRODUCTIONIn Italy, the Local Public Transport (LPT) industry reform bill (1997) touched upon two dimensions: the allocation of public funds to Regions -and, in turns, to Local Authorities (LAs) -as well as the definition of an upper bound to public compensations to LPT firms. Although the reform bill stated that all subsidized LPT services should have been tendered off by January 2004, later legislative interventions left discretion to local governments about whether tendering out concessions or adopting in-house provision. Some competitive tendering took place after 1998 [1]. However, at present, the amount of LPT services that are tendered off is negligible with respect to in house provision that still prevails, especially in large cities [2]. In recent years, a number of policy interventions reaffirmed the political aim to improve market-oriented mechanisms in the allotment of LPT services; moreover, several parliamentary acts established that the maximum economic compensation to LPT firms should be based on standard costs (Law number 135/2012). The standard cost is defined as the cost of a LPT service provided by a reasonably efficient operator given a pre-specified level of service quality. However, at present, the Italian policy makers have not yet promulgated an appropriate methodology for the calculation of unit standard costs.Our paper contributes to the literature since we develop two Top-Down cost models, based on recent advancements of machine learning approaches [3], in order to identify key cost drivers with respect to different cost items. The proposed models could be an effective and simple tool for LAs to validate reserve prices in tender procedures. Detailed information about costs has been gathered in order to fairly compute the total economic costs of the LPBT services; the collected economic and transport data relates to LPBT companies producing more than 500 million of bus revenue kilometers (BRK) in Italy in 2011 (approximately 30% of the overall amount of BRK offered in Italy for bus LPT services).Thi...

show abstract

“…On the other hand, it is well-known that in certain cases, for example when the distribution of the data in the two classes is uneven (see [30]), the bias plays a crucial role concerning the generalization performance. Recently a new provably convergent method has been proposed in [26] that, iteratively adjusting the offset values, computes a solution of the bias problem (2) by solving a sequence of dual formulations that do not include the difficult equality constraint and, then, can be solved in parallel. Even if this method solves the more general bias version, it requires multiple parallel optimizations in order to solve any single SVM training.…”

mentioning

confidence: 99%

Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training

2018

Self Cite

View full text Add to dashboard Cite

We consider the convex quadratic linearly constrained problem with bounded variables and with huge and dense Hessian matrix that arises in many applications such as the training problem of bias support vector machines. We propose a decomposition algorithmic scheme suitable to parallel implementations and we prove global convergence under suitable conditions. Focusing on support vector machines training, we outline how these assumptions can be satisfied in practice and we suggest various specific implementations. Extensions of the theoretical results to general linearly constrained problem are provided. We included numerical results on support vector machines with the aim of showing the viability and the effectiveness of the proposed scheme.

show abstract

A convergent and fully distributable SVMs training algorithm

Cited by 10 publications

References 8 publications

Distributed algorithms for convex problems with linear coupling constraints

Distributed algorithms for convex problems with linear coupling constraints

Key-Cost Drivers Selection in Local Public Bus Transport Services Through Machine Learning

Parallel decomposition methods for linearly constrained problems subject to simple bound with application to the SVMs training

Contact Info

Product

Resources

About