Nonconvex generalization of Alternating Direction Method of Multipliers for nonlinear equality constrained problems

Wang, Junxiang; Zhao, Liang

doi:10.1016/j.rico.2021.100009

Cited by 16 publications

(14 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the convergence of ADMM is well-known for convex problems [18], convergence can also be proven in some other scenarios that often necessitate special proofs, such as classes of weakly convex problems [126], certain non-convex ADMMs with linear constraints [124,116,47], non-convex and non-linear, but equality-constrained problems [114], and bilinear constraints [123]. For specific non-linear non-convex ADMMs, specialized proofs exist [36,121].…”

Section: Alternating Direction Methods Of Multipliersmentioning

confidence: 99%

A Splitting Scheme for Flip-Free Distortion Energies

Stein¹,

Li²,

Solomon³

2021

Preprint

View full text Add to dashboard Cite

We introduce a robust optimization method for flip-free distortion energies used, for example, in parametrization, deformation, and volume correspondence. This method can minimize a variety of distortion energies, such as the symmetric Dirichlet energy and our new symmetric gradient energy. We identify and exploit the special structure of distortion energies to employ an operator splitting technique, leading us to propose a novel Alternating Direction Method of Multipliers (ADMM) algorithm to deal with the non-convex, non-smooth nature of distortion energies. The scheme results in an efficient method where the global step involves a single matrix multiplication and the local steps are closed-form per-triangle/per-tetrahedron expressions that are highly parallelizable. The resulting general-purpose optimization algorithm exhibits robustness to flipped triangles and tetrahedra in initial data as well as during the optimization. We establish the convergence of our proposed algorithm under certain conditions and demonstrate applications to parametrization, deformation, and volume correspondence.

show abstract

Section: Alternating Direction Methods Of Multipliersmentioning

confidence: 99%

A Splitting Scheme for Flip-Free Distortion Energies

Stein¹,

Li²,

Solomon³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Equation ( 5) is convex, which can be solved by Fast Iterative Soft Thresholding Algorithm (FISTA) [1]. For Equation (4), nonsmooth activations usually lead to closedform solutions [23], [24]. For example, for Relu f l (z l ) = max(z l , 0), the solution to Equation ( 4) is shown as follows:…”

Section: Update W K+1mentioning

confidence: 99%

Toward Model Parallelism for Deep Neural Network Based on Gradient-Free ADMM Framework

Wang

Chai

Cheng

et al. 2020

2020 IEEE International Conference on Data Mining (ICDM)

Self Cite

View full text Add to dashboard Cite

The Graph Augmented Multi-Layer Perceptron (GA-MLP) model is an attractive alternative to Graph Neural Networks (GNNs). This is because it is resistant to the oversmoothing problem, and deeper GA-MLP models yield better performance. GA-MLP models are traditionally optimized by the Stochastic Gradient Descent (SGD). However, SGD suffers from the layer dependency problem, which prevents the gradients of different layers of GA-MLP models from being calculated in parallel. In this paper, we propose a parallel deep learning Alternating Direction Method of Multipliers (pdADMM) framework to achieve model parallelism: parameters in each layer of GA-MLP models can be updated in parallel. The extended pdADMM-Q algorithm reduces communication cost by utilizing the quantization technique. Theoretical convergence to a critical point of the pdADMM algorithm and the pdADMM-Q algorithm is provided with a sublinear convergence rate o(1/k). Extensive experiments in six benchmark datasets demonstrate that the pdADMM can lead to high speedup, and outperforms all the existing state-of-the-art comparison methods. 1

show abstract

“…One of the reasons is that convergence has only been guaranteed under certain assumptions when ADMM is directly applied to nonconvex problems [9]. The work in [10] designs a new generic two-block ADMM framework called neADMM for handling nonlinear constraints, which can converge to a global optimal solution of the problem and yield a sublinear convergence rate o(1/k), where k is the number of iterations. Whereas, the practical utility of two-block ADMM for nonconvex optimization problems is limited by the excessive computational effort due to its slow (sublinear) convergence rate and non-parallelizability [7], which usually results in real-time control being impossible.…”

Section: Introductionmentioning

confidence: 99%

Cloud-based computational model predictive control using a parallel multi-block ADMM approach

Ma¹,

Gao²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

Heavy computational load for solving nonconvex problems for large-scale systems or systems with real-time demands at each sample step has been recognized as one of the reasons for preventing a wider application of nonlinear model predictive control (NMPC). To improve the real-time feasibility of NMPC with input nonlinearity, we devise an innovative scheme called cloud-based computational model predictive control (MPC) by using an elaborately designed parallel multi-block alternating direction method of multipliers (ADMM) algorithm. This novel parallel multi-block ADMM algorithm is tailored to tackle the computational issue of solving a nonconvex problem with nonlinear constraints. It is ensured that the designed algorithm converges to a locally optimal solution of the optimization problem under reasonable assumptions by using the Kurdyka-Łojasiewicz property. With the help of this distributed optimization algorithm, a computational MPC scheme is developed, which can transform the NMPC optimization problem into a set of subproblems only associated with the decision variables at one prediction step. Through the parallel computing algorithm, the computational MPC can deal with large computational loads caused by high dimensional optimization problems, and improve computational efficiency. Furthermore, to allow for a more efficient implementation of the developed computational MPC and alleviate local calculation loads, a cloud-based computational MPC architecture is devised, which makes significantly better use of computational resources provided by a cloud server. An important advantage of this architecture with Docker 1 container to implement parallelization is that it does not lead to large increases in the solution time regardless of how long the prediction horizon is set. Finally, the developed cloud-based computational MPC architecture is trialed on a group of plug-in hybrid electric vehicles (PHEVs).

show abstract

Nonconvex generalization of Alternating Direction Method of Multipliers for nonlinear equality constrained problems

Cited by 16 publications

References 20 publications

A Splitting Scheme for Flip-Free Distortion Energies

A Splitting Scheme for Flip-Free Distortion Energies

Toward Model Parallelism for Deep Neural Network Based on Gradient-Free ADMM Framework

Cloud-based computational model predictive control using a parallel multi-block ADMM approach

Contact Info

Product

Resources

About