An Inexact Proximal Path-Following Algorithm for Constrained Convex Minimization

Abstract

Abstract. Many scientific and engineering applications feature nonsmooth convex minimization problems over convex sets. In this paper, we address an important instance of this broad class where we assume that the nonsmooth objective is equipped with a tractable proximity operator and that the convex constraint set affords a self-concordant barrier. We provide a new joint treatment of proximal and self-concordant barrier concepts and illustrate that such problems can be efficiently solved, without the need of lifting the problem dimensions, as in disciplined convex optimization approach. We propose an inexact path-following algorithmic framework and theoretically characterize the worst-case analytical complexity of this framework when the proximal subproblems are solved inexactly. To show the merits of our framework, we apply its instances to both synthetic and real-world applications, where it shows advantages over standard interior point methods. As a by-product, we describe how our framework can obtain points on the Pareto frontier of regularized problems with self-concordant objectives in a tuning free fashion.

Inexact proximal Newton methods for self-concordant functions

“…The analysis extends results from [28] by taking into account inexactness of the computation of the proximal Newton steps, and differs from [16,27] in the conditions used to describe inexactness of the Newton steps. The conclusions are similar to the results reached in [8,17] under different assumptions on the smooth component of the cost function.…”

“…Moreover, in constrast to many other nonsmooth optimization algorithms, the same line search strategies can be adopted as for the unconstrained Newton method. These convergence properties are discussed in [17] under the assumptions that g is strongly convex with Lipschitz continuous gradient, and in [16,27,28] for self-concordant functions g.…”

“…The papers [8,17,28] also analyze variable metric or quasi-Newton methods, in which approximate Hessians are used in the approximation (3). The effect of inexactness on the proximal Newton method with a self-concordant function g is discussed in [16,27]. In this analysis, inexactness is measured by the suboptimality (in function value) of the approximate solution of (2).…”

Inexact Successive quadratic approximation for regularized optimization

“…This together with (31) imply that {δ k } is a monotonically decreasing sequence. Dividing both sides of (60) by δ k+1 δ k , and from the fact that δ k is decreasing and nonnegative, we conclude…”

“…For the case in which f is convex, thrice continuously differentiable, and self-concordant, and ψ is the indicator function of a closed convex set, [31] analyzed global and local convergence rates of inexact damped proximal Newton with a fixed step size. The paper [19] extends this convergence analysis to general convex ψ.…”