Hölder Gradient Descent and Adaptive Regularization Methods in Banach Spaces for First-Order Points

Abstract

This paper considers optimization of smooth nonconvex functionals in smooth infinite dimensional spaces. A Hölder gradient descent algorithm is first proposed for finding approximate first-order points of regularized polynomial functionals. This method is then applied to analyze the evaluation complexity of an adaptive regularization method which searches for approximate first-order points of functionals with β-Hölder continuous derivatives. It is shown that finding an ǫ-approximate first-order point requires at most O(ǫ − p+β p+β−1 ) evaluations of the functional and its first p derivatives.

Convergence properties of an Objective-Function-Free Optimization regularization algorithm, including an $\mathcal{O}(ε^{-3/2})$ complexity bound

“…A third possibility is to consider optimization in infinite-dimensional smooth Banach spaces, a development presented for the standard framework in [19]. This requires specific techniques for computing the step and a careful handling of the norms involved.…”