A Short Note on Resolving Singularity Problems in Covariance Matrices

Abstract

In problems where a distribution is concentrated in a lower-dimensional subspace, the covariance matrix faces a singularity problem. In downstream statistical analyzes this can cause a problem as the inverse of the covariance matrix is often required in the likelihood. There are several methods to overcome this challenge. The most wellknown ones are the eigenvalue, singular value, and Cholesky decompositions. In this short note, we develop a new method to deal with the singularity problem while preserving the covariance structure of the original matrix. We compare our alternative with other methods. In a simulation study, we generate various covariance matrices that have different dimensions and dependency structures, and compare the CPU times of each approach.

K-th Nearest Neighbor (KNN) Entropy Estimates of Complexity and Integration from Ongoing and Stimulus-Evoked Electroencephalographic (EEG) Recordings of the Human Brain

“…A multivariate Gaussian distribution was achieved via this procedure, but any preexisting interelectrode dependencies were likely weakened and thus the estimates of I(X) and C I (X) were likely distorted to some degree. Although it is possible to use Cholesky decomposition to implement a Gaussian transformation of multivariate data that preserves data covariance, this technique may fail when the covariance matrix is nearly singular due to high interdependencies among variables [ 95 ], such as is often observed for EEG signals.)…”

SDCOR: Scalable density-based clustering for local outlier detection in massive-scale datasets

“…According to the fact that during scalable clustering, we use the Mahalanobis distance measure to assign each object to a minicluster, and besides, the size of temporary clusters is much smaller than that of original clusters, it would be worth mentioning an important matter here. Concerning [41,[70][71][72], in the case of high-dimensional data, classical approaches based on the Mahalanobis distance are usually not applicable. Because when the cardinality of a cluster is less than or equal to its dimensionality, the sample covariance matrix will become singular and not invertible [73]; hence, the corresponding Mahalanobis distance will no longer be reliable.…”

Dimensionality reduction of multielement glass evidence to calculate likelihood ratios

“…An issue with the implementation of these formulae lies in the estimation of within‐ and between‐sample variance–covariance matrices. It is well‐known, in statistics, that the (standard) maximum likelihood estimates of the variance–covariance matrices become singular with probability almost equal to 1 as the dimension of the data increases 9,10 . A singular or non‐invertible matrix can cause problems in a number of ways.…”