Pathologies of Between-Groups Principal Components Analysis in Geometric Morphometrics

Abstract

Good empirical applications of geometric morphometrics (GMM) typically involve several times more variables than specimens, a situation the statistician refers to as "high p/n," where p is the count of variables and n the count of specimens. This note calls your attention to two predictable catastrophic failures of one particular multivariate statistical technique, between-groups principal components analysis (bgPCA), in this high-p/n setting. The more obvious pathology is this: when applied to the patternless (null) model of p identically distributed Gaussians over groups of the same size, both bgPCA and its algebraic equivalent, partial least squares (PLS) analysis against group, necessarily generate the appearance of huge equilateral group separations that are fictitious (absent from the statistical model). When specimen counts by group vary greatly or when any group includes fewer than about ten specimens, an even worse failure of the technique obtains: the smaller the group, the more likely a bgPCA is to fictitiously identify that group as the end-member of one of its derived axes. For these two reasons, when used in GMM and other high-p/n settings the bgPCA method very often leads to invalid or insecure biological inferences. This paper demonstrates and quantifies these and other pathological outcomes both for patternless models and for models with one or two valid factors, then offers suggestions for how GMM practitioners should protect themselves against the consequences for inference of these lamentably predictable misrepresentations. The bgPCA method should never be used unskeptically-it is always untrustworthy, never authoritative-and whenever it appears in partial support of any biological inference it must be accompanied by a wide range of diagnostic plots and other challenges, many of which are presented here for the first time.Keywords Between-group principal components analysis · Ratio of variables to cases · High-p/n data sets · Discriminant function analysis in high dimensions · Factor analysis and discrimination · The Marchenko-Pastur theorem · Geometric morphometrics · Predictable mistakes in biometric data analysis · Effects of varying group size · Alternatives to canonical variates analysis

A spectral theory for Wright’s inbreeding coefficients and related quantities

“…For those artificial groups, we computed the non-null singular value of the between-population matrix, Z sc ST = ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi n À 1 p , and the leading singular value of the within-population matrix, Z sc S = ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi n À 1 p . For the simulations, the separation condition was never verified, rejecting population structure in all cases (S2(A) Fig) . For smaller sample sizes (n = 10 and L � 1, 000), the separation condition was erroneously checked in 21% simulations, indicating that we had less power to discriminate among artificial groups with small sample sizes (S2(B) Fig) . Those results were also consistent with difficulties reported for between-group PCA [46].…”

Initial Investigations of the Cranial Size and Shape of Adult Eurasian Otters (Lutra lutra) in Great Britain

“…Therefore, specific influences or factors should be more effectively isolated at specific levels of the model, as this should, in principle, reduce the effects of the common problem in PCA that leads to mixing of different effects in components if they are not orthogonal "in reality". However, it has been remarked that between-groups PCA [35] (a form of two-level mPCA) can overestimate differences between groups when sample sizes are small, because between-group variation is represented well by differences between means, but within-group variation can be underestimated. Another limitation of mPCA is that the number of non-zero eigenvalues can be constrained by the number of groups at a given level.…”

Modern morphometrics and the study of population differences: Good data behind clever analyses and cool pictures?

“…Although errors due to measurement and sampling must always be assessed in relation to the specific questions and statistical model, when the questions concerns small differences and crucially depend on accurate estimates of means, variances and covariances, not only sophisticated methods but also high density measurements are no easy fix for very large sampling error. In fact, more variables, as increasingly common in analyses employing semilandmarks, could inflate differences and increase the distortion of between group shape relationships (Bookstein, 2019;Cardini, O'Higgins, & Rohlf, 2019). Again, the problem is neither new nor specific to morphometrics: "Having bucketloads of data only increases the challenges in producing robust and responsible conclusions.…”

“…Common solutions to deal with a large number of variables and relatively small samples are: excluding smallest samples; dimensionality reduction; the use of distance-based resampling statistics in the full shape data space. None of these remedies is perfect and it has been shown that unfavorable N/p ratios might produce spurious patterns in simulated data that only contain random isotropic noise (Bookstein, 2017(Bookstein, , 2019Cardini et al, 2019). Thus, for instance, a PCA might suggest dominant PCs and elliptical scatter on PC1-PC2 (e.g., fig.…”