Permutation-based true discovery proportions for fMRI cluster analysis

Andreella, Angela; Hemerik, Jesse; Weeda, Wouter D.; Finos, Livio; Goeman, Jelle J.

doi:10.48550/arxiv.2012.00368

Cited by 3 publications

(8 citation statements)

References 29 publications

(54 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An interesting perspective of this work is to compare the performance of other templates. Our experience in DE studies indicates that improving on the Simes family is challenging; similar conclusions have been reported in Andreella et al (2020) for the analysis of fMRI data.…”

Section: Discussionsupporting

confidence: 73%

See 1 more Smart Citation

Powerful and interpretable control of false discoveries in differential expression studies

Enjalbert-Courrech

Neuvial

2022

Preprint

View full text Add to dashboard Cite

Motivation: The standard approach for statistical inference in differential expression (DE) analyses is to control the False Discovery Rate (FDR). However, controlling the FDR does not in fact imply that the proportion of false discoveries is upper bounded. Moreover, no statistical guarantee can be given on subsets of genes selected by FDR thresholding. These known limitations are overcome by post hoc inference, which provides guarantees of the number of proportion of false discoveries among arbitrary gene selections. However, post hoc inference methods are not yet widely used for DE studies. Results: In this paper, we demonstrate the relevance and illustrate the performance of adaptive interpolation-based post hoc methods for DE studies. First, we formalize the use of permutation-based methods to obtain sharp confidence bounds that are adaptive to the dependence between genes. Then, we introduce a generic linear time algorithm for computing post hoc bounds, making these bounds applicable to large-scale DE studies. The use of the resulting Adaptive Simes bound is illustrated on a RNA sequencing study. Comprehensive numerical experiments based on real microarray and RNA sequencing data demonstrate the statistical performance of the method. Availability: A cross-platform open source implementation within the R package sanssouci is available at https://pneuvial.github.io/sanssouci/.

show abstract

Section: Discussionsupporting

confidence: 73%

“…• A closely related approach called pARI has recently been proposed by Andreella et al (2020) for the analysis of neuroimaging data. It is implemented in the R package pARI.…”

Section: Existing Post Hoc Inference Methodsmentioning

confidence: 99%

Powerful and interpretable control of false discoveries in differential expression studies

Enjalbert-Courrech

Neuvial

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The idea is then to sign-flip differences between paired observations. For additional discussions and applications, see Efron 1969;Bekker and Lawford 2008;Davidson and Flachaire 2008;Winkler et al 2014;Andreella et al 2020. This setting is highly convenient to work with, which allows us to illustrate our proposed methodology.…”

Section: Properties Of An Oracle Subgroup-invariance Testmentioning

confidence: 99%

“…The original data are available at https://openfmri. org and we used the pre-prosessed data from Andreella (2021) (for details see Smeets et al, 2013;Andreella et al, 2020). The dataset contains measurements for n = 29 subjects who interchangeably looked a images of food and non-food.…”

Section: • Tests With M = 4nmentioning

confidence: 99%

“…As an illustration, we have focused on sign-flipping transformations. These are often used in practice when the data entries are symmetric about 0 under the null, as is often the case for paired data (Fisher, 1935) and fMRI data (Winkler et al, 2014;Andreella et al, 2020). We have illustrated in detail how to construct (near-)oracle subgroups in this case.…”

Section: • Tests With M = 4nmentioning

confidence: 99%

See 1 more Smart Citation

Faster Exact Permutation Testing: Using a Representative Subgroup

W.¹,

Hemerik²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Non-parametric tests based on permutation, rotation or sign-flipping are examples of so-called group-invariance tests. These tests rely on invariance of the null distribution under a set of transformations that has a group structure, in the algebraic sense. Such groups are often huge, which makes it computationally infeasible to use the entire group. Hence, it is standard practice to test using a randomly sampled set of transformations from the group. This random sample still needs to be substantial to obtain good power and replicability. We improve upon the standard practice by using a well-designed subgroup of transformations instead of a random sample. We show this can yield a more powerful and fully replicable test with the same number of transformations. For a normal location model and a particular design of the subgroup, we show that the power improvement is equivalent to the power difference between a Monte Carlo Z -test and Monte Carlo t-test. In our simulations, we find that we can obtain the same power as a test based on sampling with just half the number of transformations, or equivalently, more power for the same computation time. These benefits come entirely 'for free', as our methodology relies on an assumption of invariance under the subgroup, which is implied by invariance under the entire group.

show abstract

Notip: Non-parametric True Discovery Proportion control for brain imaging

Blain,

Thirion,

Neuvial

2022

Preprint

View full text Add to dashboard Cite

Cluster-level inference procedures are widely used for brain mapping. These methods compare the size of clusters obtained by thresholding brain maps to an upper bound under the global null hypothesis, computed using Random Field Theory or permutations.However, the guarantees obtained by this type of inference -i.e. at least one voxel is truly activated in the cluster -are not informative with regards to the strength of the signal therein. There is thus a need for methods to assess the amount of signal within clusters; yet such methods have to take into account that clusters are defined based on the data, which creates circularity in the inference scheme.This has motivated the use of post hoc estimates that allow statistically valid estimation of the proportion of activated voxels in clusters. In the context of fMRI data, the All-Resolutions Inference framework introduced in [24] provides post hoc estimates of the proportion of activated voxels. However, this method relies on parametric threshold families, which results in conservative inference. In this paper, we leverage randomization methods to adapt to data characteristics and obtain tighter false discovery control. We obtain Notip: a powerful, non-parametric method that yields statistically valid estimation of the proportion of activated voxels in data-derived clusters. Numerical experiments demonstrate substantial power gains compared with state-of-the-art methods on 36 fMRI datasets. The conditions under which the proposed method brings benefits are also discussed.

show abstract

Permutation-based true discovery proportions for fMRI cluster analysis

Cited by 3 publications

References 29 publications

Powerful and interpretable control of false discoveries in differential expression studies

Powerful and interpretable control of false discoveries in differential expression studies

Faster Exact Permutation Testing: Using a Representative Subgroup

Notip: Non-parametric True Discovery Proportion control for brain imaging

Contact Info

Product

Resources

About