Neural alpha oscillations and pupil size differentially index cognitive demand under competing audio-visual task conditions

Abstract

Cognitive demand is thought to modulate two often used, but rarely combined, measures: pupil size and neural alpha (8-12 Hz) oscillatory power. However, it is unclear whether these two measures capture cognitive demand in a similar way, and whether a measured cognitive demand response in pupil size or brain activity is dependent on sensory modality by which the demand is induced. Here we recorded pupil size and neural alpha power (using electroencephalography), while human participants of both sexes concurrently performed a visual multiple object-tracking task and an auditory gap-detection task. Difficulties of the two tasks were manipulated independent of each other. Participants' performance decreased in accuracy and speed with increasing cognitive demand. Pupil size increased with increasing demand for both the auditory and the visual task. Alpha power emerging from two cortical sources showed counteracting neural dynamics: Alpha power in visual cortex decreased with increasing demand in the visual task, whereas alpha power in parietal cortex increased with increasing demand in the auditory task. Furthermore, demand-related changes in pupil size and alpha power were uncorrelated, suggesting that they index different aspects of cognitive demand. Overall, the current study demonstrates that the dynamics of the neurophysiological indices of cognitive demand and associated effort are multi-faceted and potentially modality-dependent under complex audio-visual task conditions.

Eye movements decrease during effortful speech listening

“…We show that pupil area increases with increasing effort induced by masked speech during sentence listening (Figures 2C and 5D), in line with a large body of previous work (Zekveld et al, 2010;Kuchinsky et al, 2013;Winn et al, 2015;Wendt et al, 2016;Winn, 2016;Wendt et al, 2017;Zekveld et al, 2019;Kadem et al, 2020;Seifi Ala et al, 2020;Winn and Teece, 2021;Zhang et al, 2022). The current results further show that the masking-related increase in pupil area is also present when fixation on a stationary point is not required (see also Kraus et al, 2022). Fixation to a stationary point reduces the sensitivity of behavioral measures of speech comprehension (Figure 1), and may impair memory and mental imagery for spoken speech (Johansson et al, 2012).…”

“…Because participants were not required to fixate in one location on the screen for most of the experimental conditions, pupil area could be affected by the changing angle of the pupil relative to the eye-tracking camera. In order to mitigate this potential issue, we regressed out any linear and quadratic relationship of x and y with the pupil area before low-pass filtering (Fink et al, 2021; Kinley and Levy, 2021; Kraus et al, 2022). There were no meaningful differences in the results reported below compared to the uncorrected pupil area.…”

“…Whereas the current study was not designed to specifically examine inter-individual variations (i.e., correlation analyses may require a higher number of participants; Bossier et al, 2020;Grady et al, 2021), the absence of a correlation between pupil-area and eye-movement changes may imply distinct processes or mechanisms (although note that the direction of the relation was as expected). Previous works have also failed to find a significant relationship between indices of listening effort, for example among pupil area, neural oscillation, heart rate, and skin conductance (Miles et al, 2017;Strand et al, 2018;Alhanbali et al, 2019;Seifi Ala et al, 2020;Kraus et al, 2022). The absence of a relationship between listening-effort measures might be due to different neural mechanisms that tap into different aspects of effort (Strand et al, 2018;Herrmann and Johnsrude, 2020b;Strand et al, 2020).…”

Pupil size and eye movements differently index effort in both younger and older adults

“…For example, older adults often invest substantial effort to comprehend speech in noise, and this effort is considered an early sign of age-related hearing loss (Eckert et al, 2016; Pichora-Fuller et al, 2016; Peelle, 2018; Herrmann and Johnsrude, 2020). Research is thus increasingly dedicated to quantifying effort (Zekveld et al, 2010; McGarrigle et al, 2014; Moran et al, 2016; Francis and Love, 2020; Shechter and Share, 2020; Slade et al, 2021; Kraus et al, 2023b), but existing effort measures have limitations.…”

“…Research is thus increasingly dedicated to quantifying effort (Zekveld et al, 2010;McGarrigle et al, 2014;Moran et al, 2016;Francis and Love, 2020;Shechter and Share, 2020;Slade et al, 2021;Kraus et al, 2023b), but existing effort measures have limitations.…”