A non-monotone linear search algorithm with mixed direction on Stiefel manifold

Abstract

In this paper, we propose a non-monotone line search method for solving optimization problems on Stiefel manifold. Our method uses as a search direction a mixed gradient based on a descent direction, and a Barzilai-Borwein line search. Feasibility is guaranteed by projecting each iterate on the Stiefel manifold, through SVD factorizations. Some theoretical results for analyzing the algorithm are presented. Finally, we provide numerical experiments comparing our algorithm with other state-of-the-art procedures.

Computing the completely positive factorization via alternating minimization

“…In order to design an alternating minimization method with improved convergence guarantees, we propose a variant of Algorithm 1 inspired by the recently developed proximal point iteration on the Stiefel Manifold. 7 Specifically, we update the iterates using the following iterative process…”

“…We note that the equality constraints in (3) are known as orthogonality constraints and define the so called Stiefel manifold. [5][6][7] The method we propose, referred to as splitting alternating minimization (SAM), seeks a solution X, Y of (3) by generating sequences X k , Y k coming from the following alternating minimization scheme…”

Core Content Selection and Textbook Design for High School Mathematics

“…Non-monotone linear search method [27][28][29][30][31]: Starting from a point W on the Stiefel manifold (where W is initialized such that W T W = I k ), the direction within the tangent space of W is determined as the search direction F. An suitable step length dt was chosen, whereby iteration is performed within the tangent space, and the iterated points are remapped back onto the Stiefel manifold after completion of the step length. This process was repeated until the objective function converged.…”

Proximal Point Algorithm with Euclidean Distance on the Stiefel Manifold

“…The orthogonality-constrained minimization problem ( 1) is widely applicable in many fields, such as the nearest low-rank correlation matrix problem [2,3], the linear eigenvalue problem [4][5][6], sparse principal component analysis [5,7], Kohn-Sham total energy minimization [4,6,8,9], low-rank matrix completion [10], the orthogonal Procrustes problem [8,11], maximization of sums of heterogeneous quadratic functions from statistics [4,12,13], the joint diagonalization problem [13], dimension reduction techniques in pattern recognition [14], and deep neural networks [15,16], among others.…”