Measuring attention and vigilance in the laboratory vs. online: The split-half reliability of the ANTI-Vea

Abstract

Over the past few years, there has been growing interest in using online methods for collecting data from large samples. However, only a few studies have administered online behavioral tasks to assess attention outside the lab. In the present study, we assessed the classic attentional functions and two vigilance components using two versions of the Attentional Networks Test for Interactions and Vigilance-executive and arousal vigilance components (ANTI-Vea): (1) a standard version, performed under typical experimental conditions (n = 314), and (2) an online version, completed outside the lab (n = 303). Both versions were equally effective in assessing (1) the main effects and interactions of phasic alertness, orienting, and executive control, and (2) the executive (i.e., a decline in the ability to detect infrequent critical signals) and the arousal (i.e., a progressive slowness and variability in responses to stimuli from the environment) vigilance decrement across time on task. Responses were generally slower in the online than in the standard version. Importantly, the split-half reliability observed for both tasks was (1) higher for executive control (~.67) than for phasic alertness and orienting (< .40), as observed in previous versions of the task, and (2) between .71 and .99 for the executive and arousal vigilance measures. We expect the present study will be of interest to researchers aiming to assess attentional functions with a valid and reliable method that, importantly, is publicly available on an open website (https://www.ugr.es/~neurocog/ANTI/) and is easy to use in applied contexts.

Attentional networks, vigilance, and distraction as a function of attention‐deficit/hyperactivity disorder symptoms in an adult community sample

“…Besides the sampling bias discussed above, a more plausible reason is related to the psychometric properties of the ANTI-Vea indexes. Although our task reliability scores are similar to the ones reported in Luna, Roca, et al (2020), the reliability found for difference scores and slopes tended to be fairly low. This limitation, which is also inherent to most cognitive tasks (Dang, King, & Inzlicht, 2020;Hedge, Powell, & Sumner, 2018), could dramatically attenuate the observed correlations coefficients.…”

“…Although a loss of sensitivity over the task has been reported in our data as well as in studies with high statistical power(Feltmate et al, 2020;Luna, Roca, et al, 2020), this effect size seems to be lower than the effect on the response criterion.…”

“…However, the relatively low rate of false alarms in this literature prevents us from ruling out a floor effect that might be overestimating the role of sensitivity at the expense of underestimating the role of response criterion. Indeed, Luna, Roca, et al (2020) found a drop in sensitivity only among those participants with a percentage of false alarms close to the floor (≤5%) in the first block, but not for the rest of participants. A similar pattern was observed in our data.…”

“…For its part, arousal vigilance (AV) is measured with a salient stimulus (i.e., a red down counter) that participant must stop as fast as possible. Worthy of note, the length of the task (~33 min) enables the analysis of the decrement of both types of vigilance across the six blocks with sufficient precision and adequate reliability for using the task in experimental designs (Luna, Roca, Mart ın-Ar evalo, & Lupi añez, 2020).…”

The ANTI-Vea task: analyzing the executive and arousal vigilance decrements while measuring the three attentional networks

“…Notwithstanding, Thomson et al outcomes were replicated later in a study conducted under the typical lab conditions (Claypoole et al, 2018). Indeed, we have recently analyzed the reliability of the vigilance and attentional components of the ANTI-Vea in a study with a large sample (N = 617), wherein participants either performed the standard ANTI-Vea in the lab or the online version (https://www.ugr.es/~neurocog/ANTI/) outside the lab in a suitable place of their choosing (Luna, Roca, Martín-Arévalo, & Lupiáñez, 2020). In short, we observed that in both task versions, the split-half reliability indices (corrected by the Spearman-Brown prophecy) were higher than .75 for both the EV and AV measures.…”