It’s all about time: precision and accuracy of Emotiv event-marking for ERP research

Abstract

Background The use of consumer-grade electroencephalography (EEG) systems for research purposes has become more prevalent. In event-related potential (ERP) research, it is critical that these systems have precise and accurate timing. The aim of the current study was to investigate the timing reliability of event-marking solutions used with Emotiv commercial EEG systems. Method We conducted three experiments. In Experiment 1 we established a jitter threshold (i.e. the point at which jitter made an event-marking method unreliable). To do this, we introduced statistical noise to the temporal position of event-marks of a pre-existing ERP dataset (recorded with a research-grade system, Neuroscan SynAmps2 at 1,000 Hz using parallel-port event-marking) and calculated the level at which the waveform peaks differed statistically from the original waveform. In Experiment 2 we established a method to identify ‘true’ events (i.e. when an event should appear in the EEG data). We did this by inserting 1,000 events into Neuroscan data using a custom-built event-marking system, the ‘Airmarker’, which marks events by triggering voltage spikes in two EEG channels. We used the lag between Airmarker events and events generated by Neuroscan as a reference for comparisons in Experiment 3. In Experiment 3 we measured the precision and accuracy of three types of Emotiv event-marking by generating 1,000 events, 1 s apart. We measured precision as the variability (standard deviation in ms) of Emotiv events and accuracy as the mean difference between Emotiv events and true events. The three triggering methods we tested were: (1) Parallel-port-generated TTL triggers; (2) Arduino-generated TTL triggers; and (3) Serial-port triggers. In Methods 1 and 2 we used an auxiliary device, Emotiv Extender, to incorporate triggers into the EEG data. We tested these event-marking methods across three configurations of Emotiv EEG systems: (1) Emotiv EPOC+ sampling at 128 Hz; (2) Emotiv EPOC+ sampling at 256 Hz; and (3) Emotiv EPOC Flex sampling at 128 Hz. Results In Experiment 1 we found that the smaller P1 and N1 peaks were attenuated at lower levels of jitter relative to the larger P2 peak (21 ms, 16 ms, and 45 ms for P1, N1, and P2, respectively). In Experiment 2, we found an average lag of 30.96 ms for Airmarker events relative to Neuroscan events. In Experiment 3, we found some lag in all configurations. However, all configurations exhibited precision of less than a single sample, with serial-port-marking the most precise when paired with EPOC+ sampling at 256 Hz. Conclusion All Emotiv event-marking methods and configurations that we tested were precise enough for ERP research as the precision of each method would provide ERP waveforms statistically equivalent to a research-standard system. Though all systems exhibited some level of inaccuracy, researchers could easily account for these during data processing.

Synchronization of ear-EEG and audio streams in a portable research hearing device

“…While the authors point out that these values should not be regarded as absolute thresholds, they can be used as guidelines to narrow down the order of magnitude of temporal requirements for mobile EEG systems. The jitter of the system presented here is well below the jitter thresholds determined in Williams et al (2021). Further, when comparing the results to existing studies that report timing test data, e.g., Debener et al (2015), Blum et al (2017), Mirkovic et al (2019), Williams et al (2021), Hölle et al (2022), the system shows a similar or better accuracy.…”

“…The jitter of the system presented here is well below the jitter thresholds determined in Williams et al (2021). Further, when comparing the results to existing studies that report timing test data, e.g., Debener et al (2015), Blum et al (2017), Mirkovic et al (2019), Williams et al (2021), Hölle et al (2022), the system shows a similar or better accuracy. It can therefore be assumed that the system is suitable to be used for the measurement of temporally sharper peaks, such as the N1.…”

“…Nevertheless, there are some indications that can be used for the evaluation of the system's performance. For instance, Williams et al ( 2021 ) established a jitter threshold, i.e., the point at which jitter made an event-marking method unreliable using an ERP data set recorded using a research-grade EEG system. Their data set contained P100, N100, and P200 peaks.…”

“…For instance, obtaining ERPs requires the extraction of EEG trials by epoching the data using event markers, which indicate a response-evoking feature of the auditory stream. Accurate time synchronization is crucial since it leads to an exact alignment when averaging over the single-trial responses, leading to high and sharp ERP components, i.e., specific peak amplitudes in the ERP (Williams et al, 2021 ). For wireless commercial mobile EEG systems, the problem of event-locking the EEG data has long been challenging, as they are often not designed for it.…”

“…For instance, in early iterations of the wireless Emotiv EEG system, it was found that the built-in event-locking was unstable and did not produce high-quality ERPs (Hairston et al, 2014 ; Ries et al, 2014 ). Nowadays, mobile EEG hardware can also be used for ERP studies, as indicated by recent studies of the Emotiv EEG system (Williams et al, 2021 ). Further, fully mobile smartphone-based systems have already been successfully coupled with mobile Smarting EEG systems (mBrainTrain, Belgrade, Serbia), which could be used for extracting ERPs outside the laboratory (Debener et al, 2015 ; Blum et al, 2017 ; Hölle et al, 2022 ).…”