Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC

Necoara, Ion; Clipici, Dragos

doi:10.48550/arxiv.1302.3092

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speedup depends on the sparsity of the data matrix that defines the loss functions. A similar synchronous parallel method was studied in [26] and [8]; the latter focuses on the case of g(x) = x 1 . Scherrer et al [36] make greedy choices of multiple blocks of variables to update in parallel.…”

mentioning

confidence: 99%

Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties

Liu¹,

Wright²

2015

SIAM J. Optim.

195

237

View full text Add to dashboard Cite

We describe an asynchronous parallel stochastic proximal coordinate descent algorithm for minimizing a composite objective function, which consists of a smooth convex function added to a separable convex function. In contrast to previous analyses, our model of asynchronous computation accounts for the fact that components of the unknown vector may be written by some cores simultaneously with being read by others. Despite the complications arising from this possibility, the method achieves a linear convergence rate on functions that satisfy an optimal strong convexity property and a sublinear rate (1/k) on general convex functions. Near-linear speedup on a multicore system can be expected if the number of processors is O(n 1/4 ). We describe results from implementation on ten cores of a multicore processor.

show abstract

mentioning

confidence: 99%

Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties

Liu¹,

Wright²

2015

SIAM J. Optim.

195

237

View full text Add to dashboard Cite

show abstract

“…Doing this accurately is a costly operation; on the other hand, inaccurate setting using cheaper computations (e.g., using the number of non-zero elements as suggested in their work) results in slower convergence (see Section 6). Necoara and Clipici (2013) suggest another variant of parallel coordinate descent in which all the variables are updated in each iteration. HYDRA and GROCK can be considered as two key, distinct methods that represent the set of methods discussed above.…”

Section: Related Workmentioning

confidence: 99%

A distributed block coordinate descent method for training $l_1$ regularized linear classifiers

Mahajan,

Keerthi,

Sundararajan

2014

Preprint

View full text Add to dashboard Cite

Distributed training of l 1 regularized classifiers has received great attention recently. Most existing methods approach this problem by taking steps obtained from approximating the objective by a quadratic approximation that is decoupled at the individual variable level. These methods are designed for multicore and MPI platforms where communication costs are low. They are inefficient on systems such as Hadoop running on a cluster of commodity machines where communication costs are substantial. In this paper we design a distributed algorithm for l 1 regularization that is much better suited for such systems than existing algorithms. A careful cost analysis is used to support these points and motivate our method. The main idea of our algorithm is to do block optimization of many variables on the actual objective function within each computing node; this increases the computational cost per step that is matched with the communication cost, and decreases the number of outer iterations, thus yielding a faster overall method. Distributed Gauss-Seidel and Gauss-Southwell greedy schemes are used for choosing variables to update in each step. We establish global convergence theory for our algorithm, including Q-linear rate of convergence. Experiments on two benchmark problems show our method to be much faster than existing methods.

show abstract

Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC

Cited by 2 publications

References 0 publications

Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties

Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties

A distributed block coordinate descent method for training $l_1$ regularized linear classifiers

Contact Info

Product

Resources

About