Some Mathematical Properties of the Matrix Decomposition Solution in Factor Analysis

Abstract

A new factor analysis (FA) procedure has recently been proposed which can be called matrix decomposition FA (MDFA). All FA model parameters (common and unique factors, loadings, and unique variances) are treated as fixed unknown matrices. Then, the MDFA model simply becomes a specific data matrix decomposition. The MDFA parameters are found by minimizing the discrepancy between the data and the MDFA model. Several algorithms have been developed and some properties have been discussed in the literature (notably by Stegeman in Comput Stat Data Anal 99:189-203, 2016), but, as a whole, MDFA has not been studied fully yet. A number of new properties are discovered in this paper, and some existing ones are derived more explicitly. The properties provided concern the uniqueness of results, covariances among common factors, unique factors, and residuals, and assessment of the degree of indeterminacy of common and unique factor scores. The properties are illustrated using a real data example.

Factor analysis: Latent variable, matrix decomposition, and constrained uniqueness formulations

“…One outstanding strength of MDFA is that the residual can be obtained not only per variable, but also per observation × variable: we can identify the residual matrix = X −′. This fact follows from the identifiability of the model part ′ as shown next:Proposition (Theorem 3.1 of Adachi and Trendafilov ()). If the solution is unique, then the resulting model part ′= ′+ is also unique, which is given by …”

“…As in the proposition, the identifiability requires the uniqueness of . Its sufficient condition is shown (Adachi & Trendafilov, ) to be given by the following proposition:Proposition (Theorem 5.1 of Anderson and Rubin ()). A sufficient condition for identification of B B = [ Λ , Ψ ] except the rotational indeterminacy ΛΛ ′ = ΛTT ′ Λ ′ is that if any row of Λ is deleted , there remain two disjoint submatrices of rank m .…”

“…The solution (34) for Z = [ F , U ] can be viewed as the higher rank approximation of XB in an LS sense (Adachi, ; Adachi & Trendafilov, ). It is because the MDFA loss function (30) can be rewritten using (31) as with only || Z − XB || 2 relevant to Z and rank( Z ) = m + p greater than (3), that is, rank( XB ) = p .…”

“…The norm of the undetermined part is constant (Adachi, ) with || n 1/2 K ⊥ L ⊥ ′|| = ( nm ) 1/2 and that part is orthogonal to the determined part with ( KL ′)′( K ⊥ L ⊥ ′) = O p × m . Those facts allow us to depict (34) as the cone in Figure , where matrices are depicted as arrows (Adachi & Trendafilov, ). This figure shows that the optimal common‐unique factor score matrix forms the cone whose central axis is determined part n 1/2 KL ′ = X LΔ −1 L ′.…”

“…In this section, we consider properties of covariance matrices = n −1 X ′, = n ‐1 ′, and = n ‐1 ′.Proposition (Theorem 4.1 of Adachi and Trendafilov ()). The covariance matrices between residuals and factors satisfy …”