Measures of Listening Effort Are Multidimensional

Abstract

Objectives: Listening effort can be defined as the cognitive resources required to perform a listening task. The literature on listening effort is as confusing as it is voluminous: measures of listening effort rarely correlate with each other and sometimes result in contradictory findings. Here, we directly compared simultaneously recorded multimodal measures of listening effort. After establishing the reliability of the measures, we investigated validity by quantifying correlations between measures and then grouping-related measures through factor analysis. Design: One hundred and sixteen participants with audiometric thresholds ranging from normal to severe hearing loss took part in the study (age range: 55 to 85 years old, 50.3% male). We simultaneously measured pupil size, electroencephalographic alpha power, skin conductance, and self-report listening effort. One self-report measure of fatigue was also included. The signal to noise ratio (SNR) was adjusted at 71% criterion performance using sequences of 3 digits. The main listening task involved correct recall of a random digit from a sequence of six presented at a SNR where performance was around 82 to 93%. Test–retest reliability of the measures was established by retesting 30 participants 7 days after the initial session. Results: With the exception of skin conductance and the self-report measure of fatigue, interclass correlation coefficients (ICC) revealed good test–retest reliability (minimum ICC: 0.71). Weak or nonsignificant correlations were identified between measures. Factor analysis, using only the reliable measures, revealed four underlying dimensions: factor 1 included SNR, hearing level, baseline alpha power, and performance accuracy; factor 2 included pupillometry; factor 3 included alpha power (during speech presentation and during retention); factor 4 included self-reported listening effort and baseline alpha power. Conclusions: The good ICC suggests that poor test reliability is not the reason for the lack of correlation between measures. We have demonstrated that measures traditionally used as indicators of listening effort tap into multiple underlying dimensions. We therefore propose that there is no “gold standard” measure of listening effort and that different measures of listening effort should not be used interchangeably. When choosing method(s) to measure listening effort, the nature of the task and aspects of increased listening demands that are of interest should be taken into account. The findings of this study provide a framework for understanding and interpreting listening effort measures.

Pupil dilation is sensitive to semantic ambiguity and acoustic degradation

“…This is in line with the overall lower performance in Experiment 1 compared to Experiment 2 but is in contrast to the overall larger pupil size (relative to baseline) and larger effect of speech clarity in Experiment 2 compared to Experiment 1. The contrasting effects observed in the current study are consistent with the observation that different measures of listening effort and cognitive load are not or only minimally correlated (Miles et al, 2017;Alhanbali et al, 2019).…”

“…Although pupillometry recordings are increasingly used as a measure of listening effort (Winn et al, 2018;Zekveld et al, 2018), our data complement other data indicating that pupillometric measures do not always correlate with task performance measures or other measures of listening effort, such as subjective ratings or oscillatory neural activity (Hicks and Tharpe, 2002;Zekveld et al, 2010;Mackersie and Cones, 2011;Winn et al, 2015;Miles et al, 2017;Strand et al, 2018;Alhanbali et al, 2019). Part of the inconsistency may be due to the fact that the term 'listening effort' is ambiguous (Herrmann and Johnsrude, 2019) because it may refer to a mental act -associated with the recruitment of resources (Pichora-Fuller et al, 2016;Peelle, 2018) -or to a subjective experience (Johnsrude and Rodd, 2016;Lemke and Besser, 2016;Herrmann and Johnsrude, 2019).…”

Top-down modulation of neural envelope tracking: the interplay with behavioral, self-report and neural measures of listening effort

“…In sum, the consistent decrease in effort versus peak-shape indicate that different methods indeed tap into different aspects of listening effort (Lemke & Besser, 2016;Alhanbali et al, 2019).…”

“…Lastly, several studies have also shown that the pupil size and eye movements can be used as physiological indicators of processing load (Zekveld et al, 2010(Zekveld et al, , 2018Koelewijn et al, 2014;Wendt et al, 2014). In spite of this, the neural mechanisms underlying the changes in pupil size are complex, resulting in different outcomes compared to other effort measures (Zekveld et al, 2018;Alhanbali et al, 2019). In addition, pupillometry is affected by several factors such as age, ambient light, and requires expensive equipment as well as specialized training which makes it difficult to implement it in the clinic or research (Houben et al, 2013;Winn et al, 2018).…”

“…Different measures can thus lead to different outcomes and interpretations. Recent reviews have addressed this issue and suggest that different measures tap into different aspects of listening effort (Lemke & Besser, 2016;Pichora-Fuller et al, 2016;Ohlenforst, Zekveld, & Jansma et al, 2017;Alhanbali et al, 2019). For this reason, we chose to include five measures in this study.…”