Estimating common principal components in high dimensions

Abstract

We consider the problem of minimizing an objective function that depends on an orthonormal matrix. This situation is encountered when looking for common principal components, for example, and the Flury method is a popular approach. However, the Flury method is not effective for higher dimensional problems. We obtain several simple majorization-minizmation (MM) algorithms that provide solutions to this problem and are effective in higher dimensions. We then use simulated data to compare them with other approaches in terms of convergence and computational time.

Mixture model selection via hierarchical BIC

“…The first objective is to empirically examine Theorem 1 and the second is to compare the proposed HBIC with the two commonly used model selection criteria: (i) BIC (Fraley and Raftery, 2002) and (ii) ICL (Biernacki et al, 2000). The parameter estimation is performed by a variant of the EM implementation (Celeux and Govaert, 1995) as detailed in Section 4.1, except that the EVE and VVE models are based on the majorization minimization implementation recently proposed in Browne and McNicholas (2014). Since BIC, ICL, and HBIC differ only in the penalty term, for fair comparison, we compute the values of all criteria using the same estimate.…”

“…The well known R software mclust (Fraley and Raftery, 2002;Fraley et al, 2012) implements a subset of ten of these models. The Matlab software mixmod (Biernacki et al, 2006) and R software mixture (Browne and McNicholas, 2014) implement all of the fourteen models. Recent developments of EDGMM include the incorporation of t distributions to deal with outliers (t-EDGMM; Andrews and McNicholas, 2012), the incorporation of skew distributions to further tackle asymmetry (skew-EDGMM; Vrbik and McNicholas, 2014) and the extension of t-EDGMM in the presence of missing data (Lin, 2014).…”

teigen: An R Package for Model-Based Clustering and Classification via the Multivariate t Distribution

“…Following recent work by Browne and McNicholas (2014) that is implemented in the mixture package, an iterative majorize-minimize (MM) algorithm is implemented in the estimation of the common eigenvectors. As an illustration, we provide the specifics for the CCUC model where Σ g = λDA g D .…”

Model-Based Clustering

“…They develop alternative algorithms for these models, based on an accelerated line search on the orthogonal Stiefel manifold (see Browne and McNicholas 2014c, for details). Browne and McNicholas (2014a) develop another approach, using fast majorizationminimization algorithms, for the EVE and VVE models and it is this approach that is implemented in the mixture package for R. Details on this latter approach are given in Browne and McNicholas (2014a).…”