How representative are neuroimaging samples? Large-scale evidence for trait anxiety differences between fMRI and behaviour-only research participants.

Abstract

Over the past three decades, MRI has become a key tool to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into MRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, if a behavioural effect discovered in a non-imaging context does not replicate in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (n=272), that MRI study participants differ from behaviour-only study participants on one fundamental individual difference variable: trait anxiety. Analysing a large-scale dataset drawn from multiple institutions (n=3317) and controlling for possible confounding variables, we found robust support for lower trait anxiety in MRI study participants, consistent with a sampling bias. Distributions of trait anxiety scores differed most markedly when psychiatric screening was minimal. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures, in an attempt to mitigate this bias.

Individual differences in fear acquisition: multivariate analyses of different emotional negativity scales, physiological responding, subjective measures, and neural activation

“…Our sample size calculation revealed that Study 2 was most likely underpowered to detect an association between CS+/CS− discrimination an STAI-T scores (given the correlation coefficient observed in Study 1) and in addition represents individuals sampled from a different distribution, likely caused by the nature of the study (i.e., fMRI). Hence, we replicate the recent results by a report suggesting samples for fMRI and behavioral studies are drawn from different populations 99 .…”

“…Instead, sample bias-possibly originating from high anxious individuals not signing up for fMRI studies-may in addition to the differences in power between the studies also contribute to different results in both studies. Hence, we replicate a recent report of the existence of a profound sampling bias in MRI studies in a large set of pooled studies, which showed that participants in MRI studies had lower trait anxiety scores compared to participants in behavioral studies 99 . This implies that good characterization and reporting of study populations and experimental parameters is highly important especially in individual difference research 29 .…”

“…There is a plethora of potential and plausible reasons underlying these seeming discrepancies. As discussed in a recent review 29 , these include a number of procedural factors including high vs. low reinforcement rate 100 , potency of the experimental situation 101 instructions 102 , additional triggered outcome measures such as startle that impact on the learning process 65 as well as sample biases or exclusion of specific participants 56,99 to name just a few. Hence, it is possible that neither results may be necessarily 'wrong' as different associations may unfold depending on the specific sample, experimental context or boundary conditions.…”

Determining the effects of training duration on the behavioral expression of habitual control in humans: a multi-laboratory investigation

“…Related to this, a recent large scale investigation (Charpentier et al 2020) demonstrated that participants enrolling in fMRI studies are lower in anxiety than participants enrolling in behavioral only studies perhaps because higher anxiety participants tend to avoid taking part in fMRI studies. Anxiety (and concomitant stress reactions) are thus potential candidates for accounting for the differential manifestation of habitual behavior in studies conducted inside vs outside the scanner.…”

Controllability over stressor decreases responses in key threat-related brain areas

“…The STAI scale is defined such that scores range from 20 to 80. In a recent analysis of over 3,000 individuals across behavioral and fMRI studies, the mode was a trait score of 36, and only 5% of the participants exhibited scores above 60 (Charpentier, Faulkner et al 2020). Given our participant-matching procedure (see next), participants with trait anxiety scores above 60 were removed from the study.…”