Error rates in quadratic discrimination with constraints on the covariance matrices

Abstract

Common principal components, Linear Discriminant Function, Monte Carlo Simulation, Proportional Covariance Matrices,

Discriminant analysis under the common principal components model

“…These results concur with those reported by Flury et al (1994) and Bianco et al (2008 For the CPC situation when the rank orders of the common eigenvectors in the two population covariance matrices were exactly the same, CPC performed the best, followed by CPC and QDA.…”

“…a mixture of data from two different normal distributions), and their results confirmed what was found previously by Flury et al (1994) concerning the performance of the different discriminant functions. The results of the Monte Carlo simulation experiments presented in Section 4 of this paper are compared to these previous studies.…”

“…This hypothesis have been investigated by Schmid (1987), Flury (1988), Flury and Schmid (1992), Flury et al (1994) and Bianco et al (2008). For CPC discrimination, the unbiased covariance matrix estimators in (5) and (6) are simply replaced by covariance matrix estimators under the CPC model.…”

“…Relatively large sample sizes are needed for ordinary QDA to outperform LDA, even when the population covariance matrices are not nearly equal. Flury et al (1994) made the more general observation that using a more parsimonious but theoretically incorrect model in the Flury hierarchy of covariance matrices often leads to better classification. This is particularly true in situations where the number of groups and/or number of variables are large relative to the sample sizes, as the stricter constraints imposed on the covariance matrices 6 lead to more stable estimators and often to improved discrimination between the groups.…”

“…For sparse data, theoretically incorrect but more parsimonious models can outperform theoretically correct models in discriminant analysis applications, because the discriminant func-2 tion coefficients for a more parsimonious model will generally be less variable (Flury and Schmid, 1992). Flury et al (1994) followed up these investigations with a simulation study to determine the misclassification error rates for the different covariance matrix estimators when plugged into the quadratic discriminant function. They showed that CPC discrimination often does not have a great advantage over ordinary quadratic discrimination (using the unbiased sample estimators of the population covariance matrices), and that in many situations the performance of quadratic discrimination under the proportional model compares well with the results achieved by using the linear discriminant rule.…”