Permutation tests for ASCA in multivariate longitudinal intervention studies

Abstract

Permutation tests are the standard technique for significance testing in Analysis of Variance Simultaneous Component Analysis. However, there is a vast number of alternative approaches for permutation testing, and the number of choices grows in relation to the complexity of the study design. In this paper, we focus on longitudinal intervention studies with multivariate outcomes, a relevant experimental design in clinical studies where the outcome is an omics profile (such as in genomics, metabolomics, and the like). We propose a new technique to derive power curves tailored to the size and (un)balanced nature of the data set in the study. This technique is useful to identify misleading permutation tests, with lack of power or overly optimistic outcomes. We found that choosing the best permutation approach is far from intuitive and that there is a significant risk of deriving incorrect conclusions in real-life analyses.

On the complementary nature of ANOVA simultaneous component analysis (ASCA+) and Tucker3 tensor decompositions on designed multi‐way datasets

“…In ASCA+, the original ASCA methodology is extended to unbalanced designs by using general linear models (GLMs) to estimate the effect matrices, instead of the classical ANOVA estimators based on differences in means. 8,9 Simultaneous component analysis (SCA) was then performed on the individual effect matrices to model and visualize the variability of each effect. In SCA, the different samples are modeled using PCA.…”

“…This provides an approximate test 13 with better properties that other alternatives or even exact tests. 8 Permutation tests were performed by using 10,000 randomizations, where the p-value of the test is defined as the fraction of the permutations for which the employed metric was better than the unpermuted one. An effect is considered significant if its p-value is smaller than an appropriate significance threshold.…”

“…Although the underlying experimental design was unbalanced because of outlier removal, the ParGLM function in the MEDA toolbox is able to accommodate unbalanced designs using GLMs to estimate the effect matrices. [8][9][10] The amount of sum of squares, degrees of freedom, F ratios, and p-values for the different factors the blue crab dataset is reported in Table 2. The total variance preserved was 81.85%.…”

Variable-Selection ANOVA Simultaneous Component Analysis (VASCA)

“…There are several choices for the ASCA test statistic-see Camacho et al (2022) for a recent review on the permutation approach and the relevance of the chosen statistic. The (Type-I) sum-of-squares of the factor matrix Z 2 F was proposed as the original one (Vis et al, 2007).…”

“…E * = X * − CΘ * . Equivalently, one can permute the rows or values in C instead of those in X(Camacho et al, 2022).Permutation tests are carried out by comparing a given statistic, computed from the ASCA-factorized data set, with the corresponding statistic computed from hundreds or more permutations. The p-value is obtained…”