Frank–Wolfe and friends: a journey into projection-free first-order optimization methods

Bomze, Immanuel M.; Rinaldi, Francesco; Zeffiro, Damiano

doi:10.1007/s10288-021-00493-y

Cited by 18 publications

(4 citation statements)

References 63 publications

(93 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FW is demonstrated as an effective method for optimization over simplex [23] or spactrahedron domains [41]. We refer to [48] for convergence analysis of FW and a detailed discussion on its applications, and to [13] for a review on recent advances in FW.…”

Section: Related Workmentioning

confidence: 99%

Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization

Yurtsever¹,

Birdal²,

Golyanik³

2022

Preprint

View full text Add to dashboard Cite

We present a hybrid classical-quantum framework based on the Frank-Wolfe algorithm, Q-FW, for solving quadratic, linearly-constrained, binary optimization problems on quantum annealers (QA). The computational premise of quantum computers has cultivated the re-design of various existing vision problems into quantum-friendly forms. Experimental QA realisations can solve a particular non-convex problem known as the quadratic unconstrained binary optimization (QUBO). Yet a naive-QUBO cannot take into account the restrictions on the parameters. To introduce additional structure in the parameter space, researchers have crafted ad-hoc solutions incorporating (linear) constraints in the form of regularizers. However, this comes at the expense of a hyper-parameter, balancing the impact of regularization. To date, a true constrained solver of quadratic binary optimization (QBO) problems has lacked. Q-FW first reformulates constrained-QBO as a copositive program (CP), then employs Frank-Wolfe iterations to solve CP while satisfying linear (in)equality constraints. This procedure unrolls the original constrained-QBO into a set of unconstrained QUBOs all of which are solved, in a sequel, on a QA. We use D-Wave Advantage QA to conduct synthetic and real experiments on two important computer vision problems, graph matching and permutation synchronization, which demonstrate that our approach is effective in alleviating the need for an explicit regularization coefficient.

show abstract

Section: Related Workmentioning

confidence: 99%

Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization

Yurtsever¹,

Birdal²,

Golyanik³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Such a strategy might however be costly even when the projection is performed over some structured sets like, e.g., the flow polytope, the nuclear-norm ball, the Birkhoff polytope, the permutahedron (see, e.g., [18]). This is the reason why, in recent years, projection-free methods (see, e.g., [13,21,25]) have been massively used when dealing with those structured constraints.…”

Section: Introductionmentioning

confidence: 99%

Projection free methods on product domains

Bomze¹,

Rinaldi²,

Zeffiro³

2023

Preprint

View full text Add to dashboard Cite

Projection-free block-coordinate methods avoid high computational cost per iteration and at the same time exploit the particular problem structure of product domains. Frank-Wolfelike approaches rank among the most popular ones of this type. However, as observed in the literature, there was a gap between the classical Frank-Wolfe theory and the block-coordinate case. Moreover, most of previous research concentrated on convex objectives. This study now deals also with the non-convex case and reduces above-mentioned theory gap, in combining a new, fully developed convergence theory with novel active set identification results which ensure that inherent sparsity of solutions can be exploited in an efficient way. Preliminary numerical experiments seem to justify our approach and also show promising results for obtaining global solutions in the non-convex case.

show abstract

“…Notably, these methods are well suited to handle complicated constraints and possess a low iteration complexity. This makes them very effective in the context of large-scale machine learning problems (see, e.g., Lacoste-Julien et al [54], Jaggi [48], Négiar et al [64], Dahik [26], Jing et al [49]), image processing (see, e.g., Joulin et al [50], Tang et al [75]), quantum physics (see, e.g., Gilbert [41], Designolle et al [30]), submodular function maximization (see, e.g., Feldman et al [33], Vondrák [79], Badanidiyuru and Vondrák [5], Mirzasoleiman et al [60], Hassani et al [45], Mokhtari et al [61], Anari et al [1], Anari et al [2], Mokhtari et al [62], Bach [4]), online learning (see, e.g., Hazan and Kale [46], Zhang et al [86], Chen et al [20], Garber and Kretzu [39], Kerdreux et al [51], Zhang et al [87]) and many more (see, e.g., Bolte et al [6], Clarkson [22], Pierucci et al [70], Harchaoui et al [44], Wang et al [81], Cheung and Li [21], Ravi et al [72], Hazan and Minasyan [47], Dvurechensky et al [32], Carderera and Pokutta [17], Macdonald et al [58], Carderera et al [18], Garber and Wolf [40], Bomze et al [7], Wäldchen et al [80], Chen and Sun…”

Section: Introductionmentioning

confidence: 99%

The Frank-Wolfe Algorithm: A Short Introduction

Pokutta

2023

Jahresber. Dtsch. Math. Ver.

View full text Add to dashboard Cite

In this paper we provide an introduction to the Frank-Wolfe algorithm, a method for smooth convex optimization in the presence of (relatively) complicated constraints. We will present the algorithm, introduce key concepts, and establish important baseline results, such as e.g., primal and dual convergence. We will also discuss some of its properties, present a new adaptive step-size strategy as well as applications.

show abstract

Frank–Wolfe and friends: a journey into projection-free first-order optimization methods

Cited by 18 publications

References 63 publications

Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization

Q-FW: A Hybrid Classical-Quantum Frank-Wolfe for Quadratic Binary Optimization

Projection free methods on product domains

The Frank-Wolfe Algorithm: A Short Introduction

Contact Info

Product

Resources

About