Inflated false negative rates undermine reproducibility in task-based fMRI

Lohmann, Gabriele; Stelzer, Johannes; Mueller, Karsten; Lacosse, Eric; Buschmann, Tilo; Kumar, VJ; Grodd, Wolfgang; Scheffler, Klaus

doi:10.1101/122788

Cited by 14 publications

(14 citation statements)

References 38 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beyond providing an accurate qualitative description of response times, this account is also concordant with the general view that actions are organized in a hierarchical manner (Cooper and Shallice, 2006;Fuster, 2008) and with contemporary models suggesting a hierarchical neural organization of control processes in prefrontal cortex (Koechlin and Summerfield, 2007;Badre, 2008). However, experiments supporting the latter models have not examined the interaction between task-and responseswitching processes (Koechlin, et al, 2003;Badre and D'Esposito, 2007). Moreover, the hierarchical task-switching model has remained untested beyond its original remit of accounting for firstoder task-and response-transition effects; no novel predictions have been derived to test the model's general validity.…”

Section: Introductionsupporting

confidence: 74%

Hierarchically Organized Medial Frontal Cortex-Basal Ganglia Loops Selectively Control Task- and Response-Selection

et al. 2017

View full text Add to dashboard Cite

Adaptive behavior requires context-sensitive configuration of task-sets that specify time-varying stimulus-response mappings. Intriguingly, response time costs associated with changing task-sets and motor responses are known to be strongly interactive: switch costs at the task level are small in the presence of a response-switch but large when accompanied by a response-repetition, and vice versa for response-switch costs. The reasons behind this well known interdependence between task- and response-level control processes are currently not well understood. Here, we formalized and tested a model assuming a hierarchical organization of superordinate task-set and subordinate response-set selection processes to account for this effect. The model was found to successfully explain the full range of behavioral task- and response-switch costs across first and second order trial transitions. Using functional magnetic resonance imaging (fMRI) in healthy humans, we then characterized the neural circuitry mediating these effects. We found that presupplementary motor area (preSMA) activity tracked task-set control costs, SMA activity tracked response-set control costs, and basal ganglia (BG) activity mirrored the interaction between task- and response-set regulation processes that characterized participants' response times. A subsequent fMRI-guided transcranial magnetic stimulation experiment confirmed dissociable roles of the preSMA and SMA in determining response costs. Together, these data provide evidence for a hierarchical organization of posterior medial frontal cortex and its interaction with the BG, where a superordinate preSMA-BG loop establishes task-set selection, which imposes a (unidirectional) constraint on a subordinate SMA-BG loop that determines response-selection, resulting in the characteristic interdependence in task- and response-switch costs in behavior. The ability to use context-sensitive task-sets to guide our responses is central to human adaptive behavior. Task and response selection are strongly interactive: it is more difficult to repeat a response in the context of a changing task-set, and vice versa. However, the neurocognitive architecture giving rise to this interdependence is currently not understood. Here we use modeling, neuroimaging, and noninvasive neurostimulation to show that this phenomenon derives from a hierarchical organization of posterior medial frontal cortex and its interaction with the basal ganglia, where a more anterior corticostriatal loop establishes task-set selection, which constrains a more posterior loop responsible for response-selection. These data provide a neural explanation for a key behavioral signature of human cognitive control.

show abstract

Section: Introductionsupporting

confidence: 74%

Hierarchically Organized Medial Frontal Cortex-Basal Ganglia Loops Selectively Control Task- and Response-Selection

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Finally, it is worth noting that the appropriate parameters for the analysis of neuroimaging data is a topic of active, ongoing discussion. For example, although Eklund, Nichols, & Knutsson (2016) advocate more stringent cluster-forming thresholds for functional data, their results were not subsequently replicated (Cox, Chen, Glen, Reynolds, & Taylor, 2017a, 2017b), and may not account for elevated Type II error (Lohmann et al, 2017). The implications of this debate for the analysis of structural MRI data have not been tested.…”

Section: Methodsmentioning

confidence: 99%

Individual differences in decoding skill, print exposure, and cortical structure in young adults

Johns

Jahn

Jones

et al. 2018

Language, Cognition and Neuroscience

View full text Add to dashboard Cite

This exploratory study investigated relations between individual differences in cortical grey matter structure and young adult readers’ cognitive profiles. Whole-brain analyses revealed neuroanatomical correlations with word and nonword reading ability (decoding), and experience with printed matter. Decoding was positively correlated with grey matter volume (GMV) in left superior temporal sulcus, and thickness (GMT) in right superior temporal gyrus. Print exposure was negatively correlated with GMT in left inferior frontal gyrus (pars opercularis) and left fusiform gyrus (including the visual word form area). Both measures also correlated with supramarginal gyrus (SMG), but in spatially distinct subregions: decoding was positively associated with GMV in left anterior SMG, and print exposure was negatively associated with GMT in left posterior SMG. Our comprehensive approach to assessment both confirms and refines our understanding of the novel relation between the structure of pSMG and proficient reading, and unifies previous research relating cortical structure and reading skill.

show abstract

“…They conclude that the average fMRI study is poorly powered for capturing realistic e↵ects. In Lohmann et al (2017) the data was used to investigate the false negative rates in studies with small sample sizes. Similar to our approach, they used a larger cohort of subjects (N = 400) to approximate the true positive e↵ects.…”

Section: Discussionmentioning

confidence: 99%

“…For example, Poldrack et al (2017) used the data to obtain e↵ectsize estimates for a number of common neuroimaging experimental paradigms in certain regions of interest. More recently, Lohmann et al (2017) used it to investigate the false negative rates in studies with small sample sizes. Similar in spirit to our approach, they used a large cohort of 400 subjects to approximate the true positive e↵ects.…”

Section: Introductionmentioning

confidence: 99%

Effect Size and Power in fMRI Group Analysis

Geuter

Welsh

et al. 2018

Preprint

View full text Add to dashboard Cite

Multi-subject functional magnetic resonance imaging (fMRI) analysis is often concerned with determining whether there exists a significant population-wide 'activation' in a comparison between two or more conditions. Typically this is assessed by testing the average value of a contrast of parameter estimates (COPE) against zero in a general linear model (GLM) analysis. In this work we investigate several aspects of this type of analysis. First, we study the e↵ects of sample size on the sensitivity and reliability of the group analysis, allowing us to evaluate the ability of small sampled studies to e↵ectively capture population-level e↵ects of interest. Second, we assess the di↵erence in sensitivity and reliability when using volumetric or surface based data. Third, we investigate potential biases in estimating e↵ect sizes as a function of sample size. To perform this analysis we utilize the task-based fMRI data from the 500-subject release from the Human Connectome Project (HCP). We treat the complete collection of subjects (N = 491) as our population of interest, and perform a single-subject analysis on each subject in the population. We investigate the ability to recover population level e↵ects using a subset of the population and standard analytical techniques. Our study shows that sample sizes of 40 are generally able to detect regions with high e↵ect sizes (Cohen's d > 0.8), while sample sizes closer to 80 are required to reliably recover regions with medium e↵ect sizes (0.5 < d < 0.8). We find little di↵erence in results when using volumetric or surface based data with respect to standard mass-univariate group analysis. Finally, we conclude that special care is needed when estimating e↵ect sizes, particularly for small sample sizes.

show abstract

Inflated false negative rates undermine reproducibility in task-based fMRI

Cited by 14 publications

References 38 publications

Hierarchically Organized Medial Frontal Cortex-Basal Ganglia Loops Selectively Control Task- and Response-Selection

Hierarchically Organized Medial Frontal Cortex-Basal Ganglia Loops Selectively Control Task- and Response-Selection

Individual differences in decoding skill, print exposure, and cortical structure in young adults

Effect Size and Power in fMRI Group Analysis

Contact Info

Product

Resources

About