Relative Interior Rule in Block-Coordinate Descent

Werner, Tomáš; Průša, Daniel; Dlask, Tomáš

doi:10.1109/cvpr42600.2020.00758

Cited by 14 publications

(18 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar to other dual block coordinate ascent schemes Algorithm 1 can get stuck in suboptimal points, see [58,59]. As seen in experiments these are usually not far away from the optimum, however.…”

Section: Parallel Deferred Min-marginal Averagingmentioning

confidence: 72%

See 1 more Smart Citation

FastDOG: Fast Discrete Optimization on GPU

Abbas¹,

Swoboda²

2021

Preprint

View full text Add to dashboard Cite

We present a massively parallel Lagrange decomposition method for solving 0-1 integer linear programs occurring in structured prediction. We propose a new iterative update scheme for solving the Lagrangean dual and a perturbation technique for decoding primal solutions. For representing subproblems we follow [37] and use binary decision diagrams (BDDs). Our primal and dual algorithms require little synchronization between subproblems and optimization over BDDs needs only elementary operations without complicated control flow. This allows us to exploit the parallelism offered by GPUs for all components of our method. We present experimental results on combinatorial problems from MAP inference for Markov Random Fields, quadratic assignment and cell tracking for developmental biology. Our highly parallel GPU implementation improves upon the running times of the algorithms from [37] by up to an order of magnitude. In particular, we come close to or outperform some state-of-the-art specialized heuristics while being problem agnostic.

show abstract

Section: Parallel Deferred Min-marginal Averagingmentioning

confidence: 72%

“…The current min-marginal difference is subtracted and the one from previous iteration is added (line 10) by distributing it equally among subproblems J i . Following [59] we use a damping factor ω ∈ (0, 1) (0.5 in our experiments) to obtain better final solutions. Proposition 1.…”

Section: Parallel Deferred Min-marginal Averagingmentioning

confidence: 99%

FastDOG: Fast Discrete Optimization on GPU

Abbas¹,

Swoboda²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the construction of the decomposition into combinatorial subproblems, the specific choice of updates and their efficient implementation are still left open for the algorithm designer to decide anew for each new problem class. The work [47] analyzes different update operations for DBCA problems and theoretically characterizes the resulting stationary points.…”

Section: Related Work and Contributionmentioning

confidence: 99%

“…It is well-known that, except in special cases [13], DBCA can fail to reach the optimum of the relaxation. Suboptimal stationary points of DBCA algorithms are analyzed in [47]. One way to attain optima of Lagrangean relaxations with DBCA algorithms is to replace the original non-smooth dual objective with a smooth approximation on which DBCA is guaranteed to find the optimum.…”

Section: Smoothingmentioning

confidence: 99%

Efficient Message Passing for 0-1 ILPs with Binary Decision Diagrams

Lange¹,

Swoboda²

2020

Preprint

View full text Add to dashboard Cite

We present an efficient message passing method for 0-1 integer linear programs. Our algorithm is based on a decomposition of the original problem into subproblems that are represented as binary decision diagrams. The resulting Lagrangean dual is solved iteratively by a series of efficient block coordinate ascent steps. Our method combines techniques from dedicated solvers developed for subclasses of combinatorial problems in structured prediction with the generality of 0-1 integer linear programming. In particular, our solver has linear iteration complexity w.r.t. the problem decomposition size and empirically needs a moderate number of iterations to converge. We present experimental results on combinatorial problems from MAP inference for Markov Random Fields, graph matching, discrete tomography and cell tracking for developmental biology and show competitive performance.

show abstract

“…This is a motivation to look for other classes of convex optimization problems for which (block-)coordinate descent would work well or, alternatively, to extend convergent message passing methods to a wider class of convex problems than the dual LP relaxation of MAP inference. A step in this direction is the work [30], where it was observed that if the minimizer of the problem over the current variable block is not unique, one should choose a minimizer that lies in the relative interior of the set of block-optimizers. It is shown that any update satisfying this rule is, in a precise sense, not worse than any other exact update.…”

Section: Introductionmentioning

confidence: 99%

A Class of Linear Programs Solvable by Coordinate-Wise Minimization

Dlask¹,

Werner²

2020

Preprint

View full text Add to dashboard Cite

Coordinate-wise minimization is a simple popular method for large-scale optimization. Unfortunately, for general (non-differentiable) convex problems it may not find global minima. We present a class of linear programs that coordinate-wise minimization solves exactly. We show that dual LP relaxations of several well-known combinatorial optimization problems are in this class and the method finds a global minimum with sufficient accuracy in reasonable runtimes. Moreover, for extensions of these problems that no longer are in this class the method yields reasonably good suboptima. Though the presented LP relaxations can be solved by more efficient methods (such as max-flow), our results are theoretically non-trivial and can lead to new large-scale optimization algorithms in the future.

show abstract

Relative Interior Rule in Block-Coordinate Descent

Cited by 14 publications

References 15 publications

FastDOG: Fast Discrete Optimization on GPU

FastDOG: Fast Discrete Optimization on GPU

Efficient Message Passing for 0-1 ILPs with Binary Decision Diagrams

A Class of Linear Programs Solvable by Coordinate-Wise Minimization

Contact Info

Product

Resources

About