#EEGManyLabs: Investigating the Replicability of Influential EEG Experiments

Abstract

There is growing awareness across the neuroscience community that the replicability of findings on the relationship between brain activity and cognitive phenomena can be improved by conducting studies with high statistical power that adhere to well-defined and standardized analysis pipelines. Inspired by efforts from the psychological sciences, and with the desire to examine some of the foundational findings using electroencephalography (EEG), we have launched #EEGManyLabs, a large-scale international collaborative replication effort. Since its discovery in the early 20th century, EEG has had a profound influence on our understanding of human cognition, but there is limited evidence on the replicability of some of the most highly cited discoveries. After a systematic search and selection process, we have identified 27 of the most influential and continually cited studies in the field. We plan to directly test the replicability of key findings from 20 of these studies in teams of at least three independent laboratories. The design and protocol of each replication effort will be submitted as a Registered Report and peer-reviewed prior to data collection. Prediction markets, open to all EEG researchers, will be used as a forecasting tool to examine which findings the community expects to replicate. This project will update our confidence in some of the most influential EEG findings and generate a large open access database that can be used to inform future research practices. Finally, through this international effort, we hope to create a cultural shift towards inclusive, high-powered multi-laboratory collaborations.

Lessons from a catalogue of 6674 brain recordings

“…All our themes are evident in their article. We also draw attention to international efforts to estimate the replicability of influential EEG experiments ( Pavlov et al, 2020 ). These mass replication projects provide a broad overview.…”

Cooperative behavior evokes inter-brain synchrony in the prefrontal and temporoparietal cortex: A systematic review and meta-analysis of fNIRS hyperscanning studies

“…Not only further investigations are required but also attempts to replicate mentioned results. The fNIRS community would highly benefit from a large scale replication project similar to the EEG community (Pavlov et al, 2021). Additionally, it is hoped that editorial teams remain open to publication of studies with variety in terms of language and experimental designs and the publication of research leading to insignificant and negative inter-brain synchrony outcomes, thereby reducing the likelihood of bias.…”

The 4E approach to the human microbiome: Nested interactions between the gut‐brain/body system within natural and built environments

“…Completing this first set of goals will heavily depend on technical and methodological constraints and choices. [80][81][82][83] Therefore, we identify a second set of goals. These are of technical and methodological nature and are centered on reaching methodological maturity reflected in convergence of methods and analyses.…”

“…These are of technical and methodological nature and are centered on reaching methodological maturity reflected in convergence of methods and analyses. [61,68,69,84] Convergence will only be reached through the development, sharing, and use of adequate computational tools [82,[85][86][87] in order to represent the most relevant dimensions of the multi-dimensional/multi-scale embedded gut-brain-behavior interactomes.…”