Flexible parallel algorithms for big data optimization

Abstract

We propose a decomposition framework for the parallel optimization of the sum of a differentiable function and a (block) separable nonsmooth, convex one. The latter term is typically used to enforce structure in the solution as, for example, in LASSO problems. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (Southwell-type) ones, as well as virtually all possibilities in between (e.g., gradient- or Newton-type methods) with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results show that the new method compares favorably to existing algorithms

Resolução do Problema das n-Rainhas com Programação Paralela

“…Outros estudos recentes na área envolvem o uso da Computação Paralela em Nuvem (Ekanayake, Qiu, Gunarathne, Beason & Fox, 2010), criação de linguagens de domínio específico para aplicações paralelas (Janjic et al, 2016), processamento digital de imagens (Olmedo, De La Calleja, Benitez, & Medina, 2012) e otimização de algoritmos para Big Data (Facchinei, Sagratella & Scutari, 2014).…”

An online parallel algorithm for spectrum sensing in cognitive radio networks

“…In particular, our contributions are as follows: 1) At each time instance, all elements are updated in parallel, and the convergence speed is thus greatly enhanced compared with [7]. As a nontrivial extension of [7] from sequential update to parallel update and [10], [11] from deterministic optimization problems to stochastic optimization problems, we rigorously show that the proposed algorithm almost surely converges.…”

“…It is tempting to use the parallel algorithm proposed in [10], [11], but it converges for deterministic optimization problems only. Besides, its convergence speed heavily depends on the decay rate of the diminishing stepsize: on the one hand, a slowly decaying stepsize is preferable to make notable progress in each iteration and to achieve satisfactory convergence speed; on the other hand, theoretical convergence is guaranteed only when the stepsize decays fast enough.…”

“…2) The proposed stepsize is based on the so-called minimization rule, so notable progress is achieved all the time and the trouble of parameter tuning in [10], [11] is saved. Besides, both the update of each element and the stepsize can be calculated in closed-form, so the algorithm is very easy to implement and fast to converge.…”

Optimization algorithms for big data with application in wireless networks

“…When the coordinates are updated according to the traditional G-S rule, a few recent works [15][16][17] have proven the 0(1 / t) rate forthe G-S BCD algorithm when applied to certain special convex problems. Some recent works [ 18,19] propose BCD-based algorithms with parallel block update rules. These algorithms are…”