Generalized Principal Component Analysis: Projection of Saturated Model Parameters

Abstract

To arrive at (2.2), we first obtain the gradient of the deviance with respect to U from the steps below.

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

“…It was clearly articulated as a basis of future data mining in the Donoho "Millenium manifesto" [14]. After that, the effects of the blessing of dimensionality were discovered in many applications, for example in face recognition [15], in analysis and separation of mixed data that lie on a union of multiple subspaces from their corrupted observations [16], in multidimensional cluster analysis [17], in learning large Gaussian mixtures [18], in correction of errors of multidimensonal machine learning systems [19], in evaluation of statistical parameters [20], and in the development of generalized principal component analysis that provides low-rank estimates of the natural parameters by projecting the saturated model parameters [21].…”

“…It was clearly articulated as a basis of future data mining in the Donoho "Millenium manifesto" [14]. After that, the effects of the blessing of dimensionality were discovered in many applications, for example in face recognition [15], in analysis and separation of mixed data that lie on a union of multiple subspaces from their corrupted observations [16], in multidimensional cluster analysis [17], in learning large Gaussian mixtures [18], in correction of errors of multidimensonal machine learning systems [19], in evaluation of statistical parameters [20], and in the development of generalized principal component analysis that provides low-rank estimates of the natural parameters by projecting the saturated model parameters [21].Ideas of the blessing of dimensionality became popular in signal processing, for example in compressed sensing [22,23] or in recovering a vector of signals from corrupted measurements [24], and even in such specific problems as analysis and classification of EEG patterns for attention deficit hyperactivity disorder diagnosis [25].There exist exponentially large sets of pairwise almost orthogonal vectors ('quasiorthogonal' bases, [26]) in Euclidean space. It was noticed in the analysis of several n-dimensional random vectors drawn from the standard Gaussian distribution with zero mean and identity covariance matrix, that all the rays from the origin to the data points have approximately equal length, are nearly orthogonal and the distances between data points are all about √ 2 times larger [27].…”

Probabilistic count matrix factorization for single cell expression data analysis

“…A probabilistic version of the Gaussian PCA was proposed by Pierson and Yau (2015) in the context of single cell data analysis, with the modeling of zero inflation (the ZIFA method). ScRNA-seq data may be better analyzed by methods dedicated to count data such as the Non-negative Matrix Factorization (NMF) introduced in a Poisson-based framework by Lee and Seung (1999) or the Gamma-Poisson factor model (Cemgil, 2009;Févotte and Cemgil, 2009;Landgraf and Lee, 2015). None of the currently available dimension reduction meth-ods fully model single-cell expression data, characterized by over-dispersed zero inflated counts (Kharchenko et al, 2014;Zappia et al, 2017).…”

Probabilistic Count Matrix Factorization for Single Cell Expression Data Analysis

“…A probabilistic version of the Gaussian PCA was proposed by Pierson and Yau (2015) in the context of single cell data analysis, with the modeling of zero inflation (the ZIFA method). ScRNA-seq data may be better analyzed by methods dedicated to count data such as the Non-negative Matrix Factorization (NMF) introduced in a Poisson-based framework by Lee and Seung (1999) or the Gamma-Poisson factor model (Cemgil, 2009;Févotte and Cemgil, 2009;Landgraf and Lee, 2015). None of the currently available dimension reduction methods fully model single-cell expression data, characterized by over-dispersed zero inflated counts (Kharchenko et al, 2014;Zappia et al, 2017).…”

“…This projection onto a lower-dimensional space (of dimension K) allows one to catch gene co-expression patterns and clusters of individuals. PCA can be viewed either geometrically or through the light of a statistical model (Landgraf and Lee, 2015). Standard PCA is based on the 2 distance as a metric and is implicitly based on a Gaussian distribution (Eckart and Young, 1936).…”