Toward a taxonomic model of attention in effortful listening

Abstract

In recent years, there has been increasing interest in studying listening effort. Research on listening effort intersects with the development of active theories of speech perception and contributes to the broader endeavor of understanding speech perception within the context of neuroscientific theories of perception, attention, and effort. Due to the multidisciplinary nature of the problem, researchers vary widely in their precise conceptualization of the catch-all term listening effort. Very recent consensus work stresses the relationship between listening effort and the allocation of cognitive resources, providing a conceptual link to current cognitive neuropsychological theories associating effort with the allocation of selective attention. By linking listening effort to attentional effort, we enable the application of a taxonomy of external and internal attention to the characterization of effortful listening. More specifically, we use a vectorial model to decompose the demand causing listening effort into its mutually orthogonal external and internal components and map the relationship between demanded and exerted effort by means of a resource-limiting term that can represent the influence of motivation as well as vigilance and arousal. Due to its quantitative nature and easy graphical interpretation, this model can be applied to a broad range of problems dealing with listening effort. As such, we conclude that the model provides a good starting point for further research on effortful listening within a more differentiated neuropsychological framework.

Listening effort: Are we measuring cognition or affect, or both?

“…Shinn‐Cunningham (2008) adapted theories of visual object recognition to develop a characterization of the role of attention in speech perception and audition more generally. As discussed in more detail by Strauss and Francis (), Shinn‐Cunningham distinguishes between processes of object formation , object selection , and coping mechanisms such as “filling in” and “recalling from memory.” In terms of Edwards' () revised ELU model, these processes can be assigned roughly to the auditory scene analysis stage (object formation and selection) and the explicit processing stage (recalling from memory), with “filling in” possibly represented by top‐down adjustment of low‐level (i.e., object formation) processes. Strauss and Francis () make a similar binary distinction between externally and internally directed attention, again roughly corresponding respectively to the earlier‐ and later‐occurring resource‐demanding processes depicted in the extended ELU model.…”

Effects of Second Language Proficiency and Linguistic Uncertainty on Recognition of Speech in Native and Nonnative Competing Speech

“…Our goal of supplementing the behavioral SRT measure with a physiological measure was to permit the possibility of distinguishing between conditions exhibiting poor performance that results from listeners "giving up" on the task (cf. Eckert, Teubner-Rhodes, & Vaden, 2016;Hornsby, Naylor, & Bess, 2016;Strauss & Francis, 2017), which would result in a pattern of poor performance but low physiological evidence of effort and conditions that result in listeners simply not being able to perform as well despite trying harder (high effort with poor performance). Pupil dilation has long been used as a tool for investigating cognitive demand (Beatty, 1982;Kahneman, 1973) and has been shown to effectively index a variety of linguistic processing demands (Piquado, Isaacowitz, & Wingfield, 2010) and listening effort .…”

“…I nterest in research on listening effort has increased considerably in recent years (McGarrigle et al, 2014;Pichora-Fuller et al, 2016). One of the dominant perspectives in this developing field is that understanding speech in difficult contexts depends on the application of limited resources such as selective attention or working memory (Pichora-Fuller et al, 2016; see also discussion by Strauss & Francis, 2017). As demand on these resources increases, performance decreases and subjective effort increases concomitantly, accompanied by increases in psychophysiological responses associated with cognitive effort, such as pupil dilation (Zekveld, Heslenfeld, Johnsrude, Versfeld, & Kramer, 2014).…”

“…It is the engagement of these systems that, in turn, results in the well-documented sensation of effort (cf. Ohlenforst et al, 2017;Westbrook & Braver, 2016) and increases in physiological markers of effort such as pupil dilation ) that arise in adverse listening conditions (see also discussion by Strauss & Francis, 2017). However, although it is clear that something in the complex combination of target, masker, and listener characteristics contribute to listening effort, more research is necessary to identify how effort arises.…”

“…It may also arise when listeners must resort to postperceptual mechanisms of inference and deduction to compensate for difficulties related to the ability to accurately process information from the target language (Shinn-Cunningham & Best, 2008). In other words, informational masking arises in contexts in which there is demand on either external attention (i.e., stream segregation, object selection) or (also) on internal attention (see terminological discussion by Strauss & Francis, 2017 and also similar concepts shown in Edwards, 2016, Figure 2;Wingfield & Tun, 2007, Figure 1). Although it is obvious that higher-level information such as linguistic knowledge must play a role in guiding internal attention, it is not yet clear whether or how linguistic knowledge might affect the direction of external attention or the degree to which these influences might interact.…”