Gradient based block coordinate descent algorithms for joint approximate diagonalization of matrices

Abstract

In this paper, we propose a gradient based block coordinate descent (BCD-G) framework to solve the joint approximate diagonalization of matrices defined on the product of the complex Stiefel manifold and the special linear group. Instead of the cyclic fashion, we choose the block for optimization in a way based on the Riemannian gradient. To update the first block variable in the complex Stiefel manifold, we use the well-known line search descent method. To update the second block variable in the special linear group, based on four different kinds of elementary rotations, we construct two classes: Jacobi-GLU and Jacobi-GLQ, and then get two BCD-G algorithms: BCD-GLU and BCD-GLQ. We establish the weak convergence and global convergence of these two algorithms using the Lojasiewicz gradient inequality under the assumption that the iterates are bounded. In particular, the problem we focus on in this paper includes as special cases the well-known joint approximate diagonalization of Hermitian (or complex symmetric) matrices by invertible transformations in blind source separation, and our algorithms specialize as the gradient based Jacobi-type algorithms. All the algorithms and convergence results in this paper also apply to the real case.

Projectively and weakly simultaneously diagonalizable matrices and their applications

“…As generalizations of the eigenvalue decomposition of a single symmetric matrix, the SDO and SD properties of multiple symmetric matrices can be seen as finding a set of basis on which they all have simple representations. In independent component analysis(ICA) [5,7,9], since multiple symmetric matrices are often not SDO or SD, the approximate simultaneous diagonalization(ASD) [5,7,16,17,18,28] of them has become an important approach to solve ICA. Let the set C be as in (1).…”

“…When the feasible set of P is compact, e.g., the special orthogonal group SO m , the algorithms to solve problem ( 26) have been extensively studied; see for example [16,17,28]. When the feasible set of P is not compact, e.g., the special linear group SL m (R), several algorithms have been developed as well [1,18]. The new notions in Table 1 we studied in this paper can be regarded as the "weakly joint eigenvalue decomposition" or "projectively joint eigenvalue decomposition" of multiple symmetric matrices, and have much broader scopes than SD and SDO.…”

“…To this end, let us first introduce some definitions and notations. Let St(m, n) def = {P ∈ R n×m , P T P = I m } be the Stiefel manifold with n ≥ m. Define the rectangular special linear set [18] as RSL(m, n) def = {P ∈ R n×m , P T P ∈ SL m (R)}, which can be seen as a non-orthogonal analogue of St(m, n). It is easy to verify that 1 :…”