An Efficient Descent Method for Locally Lipschitz Multiobjective Optimization Problems

Gebken, Bennet; Peitz, Sebastian

doi:10.1007/s10957-020-01803-w

Cited by 18 publications

(21 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The continuation method we present in this paper allows researchers and practitioners with an interest in sparse optimization to systematically calculate the entire set of compromise solutions between sparsity and the main objective, thereby allowing for much more insight as well as informed model selection than weighted-sum approaches or descent methods providing single Pareto optima [21], [40]. The presented approach extends the well-known homotopy methods to the much more complex nonlinear problem setting, where weighting approaches fail due to non-convexity.…”

Section: Discussionmentioning

confidence: 99%

On the Treatment of Optimization Problems With L1 Penalty Terms via Multiobjective Continuation

Bieker

Gebken

Peitz

2022

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

We present a novel algorithm that allows us to gain detailed insight into the effects of sparsity in linear and nonlinear optimization. Sparsity is of great importance in many scientific areas such as image and signal processing, medical imaging, compressed sensing, and machine learning, as it ensures robustness against noisy data and yields models that are easier to interpret due to the small number of relevant terms. It is common practice to enforce sparsity by adding the 1 -norm as a penalty term. In order to gain a better understanding and to allow for an informed model selection, we directly solve the corresponding multiobjective optimization problem (MOP) that arises when minimizing the main objective and the 1 -norm simultaneously. As this MOP is in general non-convex for nonlinear objectives, the penalty method will fail to provide all optimal compromises. To avoid this issue, we present a continuation method specifically tailored to MOPs with two objective functions one of which is the 1 -norm. Our method can be seen as a generalization of homotopy methods for linear regression problems to the nonlinear case. Several numerical examples -including neural network training -demonstrate our theoretical findings and the additional insight gained by this multiobjective approach.

show abstract

Section: Discussionmentioning

confidence: 99%

On the Treatment of Optimization Problems With L1 Penalty Terms via Multiobjective Continuation

Bieker

Gebken

Peitz

2022

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…There are various different methods for solving nonsmooth MOPs, see, e.g., [26,15,10]. If both f and g are locally Lipschitz continuous, then the following theorem yields a necessary condition for Pareto optimality based on the Clarke subdifferentials (cf.…”

Section: Multiobjective Optimizationmentioning

confidence: 99%

“…For these remaining assumptions we consider the following example, where the feasible set is given by continuously differentiable but nonconvex inequality constraints. It is inspired by problem (15) in [41].…”

Section: Examplesmentioning

confidence: 99%

On the structure of regularization paths for piecewise differentiable regularization terms

Gebken¹,

Bieker²,

Peitz³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Regularization is used in many different areas of optimization when solutions are sought which not only minimize a given function, but also possess a certain degree of regularity. Popular applications are image denoising, sparse regression and machine learning. Since the choice of the regularization parameter is crucial but often difficult, path-following methods are used to approximate the entire regularization path, i.e., the set of all possible solutions for all regularization parameters. Due to their nature, the development of these methods requires structural results about the regularization path. The goal of this article is to derive these results for the case of a smooth objective function which is penalized by a piecewise differentiable regularization term. We do this by treating regularization as a multiobjective optimization problem. Our results suggest that even in this general case, the regularization path is piecewise smooth. Moreover, our theory allows for a classification of the nonsmooth features that occur in between smooth parts. This is demonstrated in two applications, namely support-vector machines and exact penalty methods.

show abstract

“…What is more, the structure of the Pareto Set can be exploited to find multiple solutions [12,13]. There are also methods for non-smooth problems [14,15] and multiobjective direct-search variants [16,17]. Both scalarization and descent techniques may be included in Evolutionary Algorithms (EA) [18][19][20][21][22].…”

Section: Introductionmentioning

confidence: 99%

Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models

Berkemeier

Peitz

2021

MCA

Self Cite

View full text Add to dashboard Cite

We present a local trust region descent algorithm for unconstrained and convexly constrained multiobjective optimization problems. It is targeted at heterogeneous and expensive problems, i.e., problems that have at least one objective function that is computationally expensive. Convergence to a Pareto critical point is proven. The method is derivative-free in the sense that derivative information need not be available for the expensive objectives. Instead, a multiobjective trust region approach is used that works similarly to its well-known scalar counterparts and complements multiobjective line-search algorithms. Local surrogate models constructed from evaluation data of the true objective functions are employed to compute possible descent directions. In contrast to existing multiobjective trust region algorithms, these surrogates are not polynomial but carefully constructed radial basis function networks. This has the important advantage that the number of data points needed per iteration scales linearly with the decision space dimension. The local models qualify as fully linear and the corresponding general scalar framework is adapted for problems with multiple objectives.

show abstract

An Efficient Descent Method for Locally Lipschitz Multiobjective Optimization Problems

Cited by 18 publications

References 34 publications

On the Treatment of Optimization Problems With L1 Penalty Terms via Multiobjective Continuation

On the Treatment of Optimization Problems With L1 Penalty Terms via Multiobjective Continuation

On the structure of regularization paths for piecewise differentiable regularization terms

Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models

Contact Info

Product

Resources

About