Essentials of numerical nonsmooth optimization

Gaudioso, Manlio; Giallombardo, Giovanni; Miglionico, Giovanna

doi:10.1007/s10479-021-04498-y

Cited by 2 publications

(1 citation statement)

References 126 publications

(136 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rooted in the field of non-smooth optimization [19], the (gradient-based) optimization of discontinuous programs has recently seen major interest across many domains, for example machine learning [20], computer graphics [8] and optimal control [9]. Besides gradient-free approaches such as genetic algorithms or the Nelder-Mead method [15], the state of the art in non-smooth optimization includes bundle methods, which augment the subgradient method through the exploitation of past subgradient information [21] and gradient sampling methods exploiting piecewise differentiability [22].…”

Section: Related Workmentioning

confidence: 99%

Smoothing Methods for Automatic Differentiation Across Conditional Branches

Kreikemeyer,

Andelfinger

2023

IEEE Access

View full text Add to dashboard Cite

Programs involving discontinuities introduced by control flow constructs such as conditional branches pose challenges to mathematical optimization methods that assume a degree of smoothness in the objective function's response surface. Smooth interpretation (SI) is a form of abstract interpretation that approximates the convolution of a program's output with a Gaussian kernel, thus smoothing its output in a principled manner. Here, we combine SI with automatic differentiation (AD) to efficiently compute gradients of smoothed programs. In contrast to AD across a regular program execution, these gradients also capture the effects of alternative control flow paths. The combination of SI with AD enables the direct gradient-based parameter synthesis for branching programs, allowing for instance the calibration of simulation models or their combination with neural network models in machine learning pipelines. We detail the effects of the approximations made for tractability in SI and propose a novel Monte Carlo estimator that avoids the underlying assumptions by estimating the smoothed programs' gradients through a combination of AD and sampling. Using DiscoGrad, our tool for automatically translating simple C++ programs to a smooth differentiable form, we perform an extensive evaluation. We compare the combination of SI with AD and our Monte Carlo estimator to existing gradient-free and stochastic methods on four non-trivial and originally discontinuous problems ranging from classical simulation-based optimization to neural networkdriven control. While the optimization progress with the SI-based estimator depends on the complexity of the programs' control flow, our Monte Carlo estimator is competitive in all problems, exhibiting the fastest convergence by a substantial margin in our highest-dimensional problem.INDEX TERMS automatic differentiation, optimization, imperative programs, discontinuous control flow, parameter synthesis, gradient estimation, probabilistic execution, monte carlo approximation. definition of the partial derivative, they do not provide sufficient gradient information if the control flow of the problem

show abstract

Section: Related Workmentioning

confidence: 99%

Smoothing Methods for Automatic Differentiation Across Conditional Branches

Kreikemeyer,

Andelfinger

2023

IEEE Access

View full text Add to dashboard Cite

show abstract

The Boussinesq System with Nonsmooth Boundary Conditions: Existence, Relaxation, and Topology Optimization

Vieira¹,

Cocquet

2022

SIAM J. Control Optim.

View full text Add to dashboard Cite

In this paper, we tackle a topology optimization problem which consists in finding the optimal shape of a solid located inside a fluid that minimizes a given cost function. The motion of the fluid is modeled thanks to the Boussinesq system which involves the unsteady Navier-Stokes equation coupled to a heat equation. In order to cover several models presented in the literature, we choose a non-smooth formulation for the outlet boundary conditions. This paper aims at proving existence of solutions to the resulting equations, along with the study of a relaxation scheme of the non-smooth conditions. A second part covers the topology optimization problem itself for which we proved the existence of optimal solutions and provides the definition of first order necessary optimality conditions.

show abstract

Essentials of numerical nonsmooth optimization

Cited by 2 publications

References 126 publications

Smoothing Methods for Automatic Differentiation Across Conditional Branches

Smoothing Methods for Automatic Differentiation Across Conditional Branches

The Boussinesq System with Nonsmooth Boundary Conditions: Existence, Relaxation, and Topology Optimization

Contact Info

Product

Resources

About