The Data-Processing Multiverse of Event-Related Potentials (ERPs): A Roadmap for the Optimization and Standardization of ERP Processing and Reduction Pipelines

Abstract

In studies of event-related brain potentials (ERPs), numerous decisions about data processing are required to extract ERP scores from continuous data. Unfortunately, the systematic impact of these choices on the data quality and psychometric reliability of ERP scores or even ERP scores themselves is virtually unknown, which is a barrier to the standardization of ERPs. The aim of the present study was to optimize processing pipelines for the error-related negativity (ERN) and error positivity (Pe) by considering a multiverse of data processing choices. A multiverse analysis of a data processing pipeline examines the impact of a large set of different reasonable choices to determine the robustness of effects, such as the impact of different decisions on between-trial standard deviations (i.e., data quality) and between-condition differences (i.e., experimental effects). ERN and Pe data from 298 healthy young adults were used to determine the impact of different methodological choices on data quality and experimental effects (correct vs. error trials) at several key stages: highpass filtering, lowpass filtering, ocular artifact correction, reference, baseline adjustment, scoring sensors, and measurement procedure. This multiverse analysis yielded 3,456 ERN scores and 576 Pe scores per person. An optimized pipeline for ERN included a .01 Hz highpass filter, 15 Hz lowpass filter, ICA-based ocular artifact correction, and a region of interest (ROI) approach to scoring. For Pe, the optimized pipeline included a .10 Hz highpass filter, 30 Hz lowpass filter, regression-based ocular artifact correction, a -200 to 0 ms baseline adjustment window, and an ROI approach to scoring. The multiverse approach can be used to optimize pipelines for eventual standardization, which would support efforts toward establishing normative ERP databases. The proposed process of analyzing the data-processing multiverse of ERP scores paves the way for better refinement, identification, and selection of data processing parameters, ultimately improving the precision and utility of ERPs.

Introducing RELAX (the Reduction of Electroencephalographic Artifacts): A fully automated pre-processing pipeline for cleaning EEG data - Part 1: Algorithm and Application to Oscillations

“…To address this, we have included artifact reduction metrics that address all common artifacts. Thirdly, perhaps the most important metrics should assess the impact of EEG data cleaning on the practical outcomes of the research (Clayson, Baldwin, et al, 2021). As such, we have included multiple metrics that assess whether cleaning by the different pipelines leads to more variance explained by different experimental manipulations.…”

“…Next, and perhaps most importantly, we assessed the amount of variance explained by a variety of experimental manipulations after cleaning by each pipeline. We chose experimental manipulations that are well established to provide differentiation of neural activity in the comparison of two conditions, which assesses the real-world applicability of the cleaning pipelines (Clayson, Baldwin, et al, 2021). We assessed the variance explained by a comparison of alpha oscillatory power between the working memory delay and probe periods of the Sternberg task (Bailey et al, 2020) and alpha power between EO and EC resting-state data (computation of which is described in the Supplementary Materials, section 4, pages 36-53).…”

“…Demonstrating the superior cleaning of a single artifact type (for example muscle activity) does not necessarily mean the effect of other artifacts (for example blinks) are completely mitigated, so evaluating cleaning efficacy for all artifact types simultaneously is important. Additionally, very few EEG cleaning studies have included an assessment of the most practically important metric - whether cleaning increases the signal-to-noise ratio so that experiments are more likely to detect real effects of interest (Clayson, Baldwin, et al, 2021). While these reasons for the lack of a gold-standard EEG cleaning pipeline are reasonable, the absence of consistent cleaning methods across studies limits our ability to compare findings across different studies, as cleaning methods can vary considerably from study to study.…”

Understanding the Error in Psychopathology: Notable Intraindividual Differences in Neural Variability of Performance Monitoring

“…Similarly, although the ERN-anxiety relationship appears unique to women (72,73), the present study did not observe an improvement in model fit when including biological sex and interactions with biological sex as covariates. The disparate findings of relationships might represent differences in tasks used for recording ERN (26) or in approaches to analyzing ERN (74) and suggest a need to move ERN studies toward standardization to rule out nuisance factors (e.g., paradigm characteristics) that could obscure relationships with clinical characteristics. The present study used a semantic version of the flanker paradigm 2 that has been used in behavioral (75) and EEG research (39), but it is only one of many instantiations of the flanker task.…”

Understanding the Error in Psychopathology: Notable Intraindividual Differences in Neural Variability of Performance Monitoring

“…The present sample included men and women, but the ERN-anxiety relationship appears unique to women (83,84). The disparate findings of relationships between ERN and clinical variables might represent differences in tasks used for recording ERN (26) or in approaches to analyzing ERN (85) and suggest a need to move ERN studies toward standardization to rule out nuisance factors (e.g., paradigm characteristics, data processing decisions) that could obscure relationships with clinical characteristics. It is virtually unknown which task or data processing features yield the most robust psychopathology vs. healthy control differences in ERN or dimensional relationships between ERN and clinical characteristics (86).…”