Potential Reporting Bias in fMRI Studies of the Brain

Abstract

BackgroundFunctional magnetic resonance imaging (fMRI) studies have reported multiple activation foci associated with a variety of conditions, stimuli or tasks. However, most of these studies used fewer than 40 participants.MethodologyAfter extracting data (number of subjects, condition studied, number of foci identified and threshold) from 94 brain fMRI meta-analyses (k = 1,788 unique datasets) published through December of 2011, we analyzed the correlation between individual study sample sizes and number of significant foci reported. We also performed an analysis where we evaluated each meta-analysis to test whether there was a correlation between the sample size of the meta-analysis and the number of foci that it had identified. Correlation coefficients were then combined across all meta-analyses to obtain a summary correlation coefficient with a fixed effects model and we combine correlation coefficients, using a Fisher’s z transformation.Principal FindingsThere was no correlation between sample size and the number of foci reported in single studies (r = 0.0050) but there was a strong correlation between sample size and number of foci in meta-analyses (r = 0.62, p<0.001). Only studies with sample sizes <45 identified larger (>40) numbers of foci and claimed as many discovered foci as studies with sample sizes ≥45, whereas meta-analyses yielded a limited number of foci relative to the yield that would be anticipated from smaller single studies.ConclusionsThese results are consistent with possible reporting biases affecting small fMRI studies and suggest the need to promote standardized large-scale evidence in this field. It may also be that small studies may be analyzed and reported in ways that may generate a larger number of claimed foci or that small fMRI studies with inconclusive, null, or not very promising results may not be published at all.

Externalizing proneness and brain response during pre-cuing and viewing of emotional pictures

“…While these lines of evidence suggest that current findings for high-disinhibited women would likely extend to men, follow-up studies examining disinhibition-related differences in fMRI brain response in mixed-gender samples of differing ages are clearly needed before firm conclusions can be drawn. While the sample size of this study is above-average for fMRI studies (David et al, 2013) and of sufficient size to detect relevant effects (Desmond and Glover, 2002), it bears noting that due to the correlational nature of this study, a larger-scale study could have found stronger or additional effects.…”

Fibromyalgia Is Associated with Decreased Connectivity Between Pain- and Sensorimotor Brain Areas

“…It is unlikely that these observed discrepancies are due to difference in statistical power, since the sizes of the cohorts essentially were the same. However, increased power of fMRI-studies would generally enhance the overlap of true positive findings (David et al, 2013), and although both studies had cohort sizes within the range of what commonly are used in fMRI studies, the observed discrepancies could reflect a lack of sensitivity that led to the detection of two relatively separate subsets of the true group differences. The divergent findings could possibly also in part be due to the difference in experimental design between the study of Napadow and colleagues (2010) and this study (i.e., here resting-state scans were preceded by task fMRI that potentially introduced spillover effects).…”

Neural signatures of human fear conditioning: an updated and extended meta-analysis of fMRI studies

“…Including original SPMs, as opposed to including only peak regional effects (typically reported in tables of statistics), substantially increases the analyses statistical power 13 and may avoid reporting biases that are likely to affect certain brain regions, especially smaller subcortical regions that are poorly represented in commonly used stereotaxic atlases of the human brain. 15,16 Furthermore, inclusion of this number of SPMs allowed, for the first time, the estimation of the task-specific optimal parameters for processing the peak information of the remaining studies.…”