Progress toward openness, transparency, and reproducibility in cognitive neuroscience

Abstract

Accumulating evidence suggests that many findings in psychological science and cognitive neuroscience may prove difficult to reproduce; statistical power in brain imaging studies is low, and has not improved recently; software errors in common analysis tools are common, and can go undetected for many years; and, a few large scale studies notwithstanding, open sharing of data, code, and materials remains the rare exception. At the same time, there is a renewed focus on reproducibility, transparency, and openness as essential core values in cognitive neuroscience. The emergence and rapid growth of data archives, meta-analytic tools, software pipelines, and research groups devoted to improved methodology reflects this new sensibility. We review evidence that the field has begun to embrace new open research practices, and illustrate how these can begin to address problems of reproducibility, statistical power, and transparency in ways that will ultimately accelerate discovery.

Replicable services for reproducible research: a model for academic libraries

“…Национальный 4 директорат NSF по социальным, поведенческим и экономическим наукам выпустил свои девять рекомендаций [12]. В одних содержался призыв к использованию более надежных методологий и стандартов по отчетности, с тем чтобы доклады о финансируемых исследованиях содержали достаточно деталей с целью «позволить независимому исследователю воспроизвести результаты первого исследователя» [12, p. 5].…”

“…Существуют и другие руководства, которые могут быть использованы библиотеками, особенно специалистами по связи с пользователями (liason librarian) и специалистами, выполняющими различные библиотечные функции и работающими с исследователями по конкретной дисциплине. Как было ранее указано авторами, два примера включают Рекомендации Федерации американских обществ экспериментальной биологии, а также Неврологического общества -«Исследовательские 4 Примечание переводчика. Table 2 TOP Guidelines [11] Цитирование.…”

“…4, pp. 33-45 Введение В последние годы в ряде дисциплин, включая психологию [1], биологию [2], биомедицину [3], неврологию [4], разработку лекарств [5], химию [6], климатологию [7], экономику [8,9] и образование [10], появились свидетельства того, что наука стоит перед лицом кризиса воспроизводимости. Авторы статьи ранее утверждали, что библиотеки несут ответственность за поддержку воспроизводимых исследований и должны играть ведущую роль в научных учреждениях [11].…”

Direct replication of task‐dependent neural activation patterns during sadness introspection in two independent adolescent samples

“…The scarcity of such replication efforts is also evident in social cognitive neuroscience research using child and adolescent samples. Although few direct replications exist, examinations of resting state networks have shown generally stable patterns of blood oxygen‐level‐dependent (BOLD) signal activation and meta‐analyses of task‐based activation patterns have identified reliable neural correlates for various psychological processes measured using cognitive tasks (Gilmore, Diaz, Wyble, & Yarkoni, ). Nonetheless, as Gilmore et al () indicate, “it is one thing to establish that neuroimaging methods can consistently reveal broad mappings between cognitive processes and distributed brain networks, and quite another to establish that the specific pattern of findings generated by any single study can be reproduced with a high degree of fidelity in another study” (p. 9).…”

“…Although few direct replications exist, examinations of resting state networks have shown generally stable patterns of blood oxygen‐level‐dependent (BOLD) signal activation and meta‐analyses of task‐based activation patterns have identified reliable neural correlates for various psychological processes measured using cognitive tasks (Gilmore, Diaz, Wyble, & Yarkoni, ). Nonetheless, as Gilmore et al () indicate, “it is one thing to establish that neuroimaging methods can consistently reveal broad mappings between cognitive processes and distributed brain networks, and quite another to establish that the specific pattern of findings generated by any single study can be reproduced with a high degree of fidelity in another study” (p. 9). The current study sought to address this need by identifying whole‐brain neural activation during a social cognition task involving emotion introspection in one sample of adolescents and testing whether the pattern replicated in an independent second sample of adolescents.…”