Coordinate descent is a well-studied variant of the gradient descent algorithm which offers significant per-iteration computational savings for large-scale problems defined on Euclidean domains. On the other hand, optimization on a general manifold domain can capture a variety of constraints encountered in many problems, but are difficult to model with the usual Euclidean geometry. In this paper, we present an extension of coordinate descent to general manifold domains, and provide a convergence rate analysis for geodesically convex and non-convex smooth objective functions. Our key insight is to draw an analogy between coordinates in Euclidean space and tangent subspaces of a manifold, hence our method is called tangent subspace descent. We study the effect that non-trivial curvature of manifolds have on the convergence rate and its computational properties. We find that on a product manifold, we can achieve per-iteration computational savings analogous to the Euclidean setting. On such product manifolds, we further extend our method to handle non-smooth composite objectives, and provide the corresponding convergence rates.
The condition number of a differentiable convex function, namely the ratio of its smoothness to strong convexity constants, is closely tied to fundamental properties of the function. In particular, the condition number of a quadratic convex function is the square of the aspect ratio of a canonical ellipsoid associated to the function. Furthermore, the condition number of a function bounds the linear rate of convergence of the gradient descent algorithm for unconstrained convex minimization.We propose a condition number of a differentiable convex function relative to a reference set and distance function pair. This relative condition number is defined as the ratio of a relative smoothness to a relative strong convexity constants. We show that the relative condition number extends the main properties of the traditional condition number both in terms of its geometric insight and in terms of its role in characterizing the linear convergence of first-order methods for constrained convex minimization.When the reference set X is a cone or a polyhedron and the function f is of the form f = g • A, we provide characterizations of and bounds on the condition number of f relative to X in terms of the usual condition number of g and a suitable condition number of the pair (A, X).
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.