Working Memory for Signs with Poor Visual Resolution: fMRI Evidence of Reorganization of Auditory Cortex in Deaf Signers

Abstract

Stimulus degradation adds to working memory load during speech processing. We investigated whether this applies to sign processing and, if so, whether the mechanism implicates secondary auditory cortex. We conducted an fMRI experiment where 16 deaf early signers (DES) and 22 hearing non-signers performed a sign-based n-back task with three load levels and stimuli presented at high and low resolution. We found decreased behavioral performance with increasing load and decreasing visual resolution, but the neurobiological mechanisms involved differed between the two manipulations and did so for both groups. Importantly, while the load manipulation was, as predicted, accompanied by activation in the frontoparietal working memory network, the resolution manipulation resulted in temporal and occipital activation. Furthermore, we found evidence of cross-modal reorganization in the secondary auditory cortex: DES had stronger activation and stronger connectivity between this and several other regions. We conclude that load and stimulus resolution have different neural underpinnings in the visual–verbal domain, which has consequences for current working memory models, and that for DES the secondary auditory cortex is involved in the binding of representations when task demands are low.

The cognitive hearing science perspective on perceiving, understanding, and remembering language: The ELU model

“…These findings have been taken as evidence for resource models of WM in general and the ELU model for language processing in particular ( Obleser et al, 2012 ; Petersen et al, 2015 ; Peelle, 2018 ; Rönnberg et al, 2019 ). Although visual degradation of the language signal resulted in similar effects at the behavioral level, the neural overlap was absent for sign language in deaf early signers ( Andin et al, 2021 ). Hence, while increasing WM load was reflected in increased engagement of the frontoparietal working memory network, as predicted, the degradation of the visual signal instead caused activation of bilateral inferior occipital and temporal cortices.…”

“…His discussion is based on the Rönnberg et al (2008) version of the ELU model, where RAMBPHO processing focuses on how different streams of sensory information are integrated and bound into a phonological representation (see also Stenfelt and Rönnberg, 2009 ). Nevertheless, in Rönnberg et al (2019 , 2022) , it is made more explicit that the system may feedback via postdiction processes, which may prime the prediction process ( Sörqvist and Rönnberg, 2012 ), including fine-tuning of attention ( Holmer and Rudner, 2020 ; Andin et al, 2021 ) and selection processes to specific features of the input ( Rönnberg et al, 2013 ). This seems to be rather close to stream segregation, but the theoretical languages differ.…”

“…In WM studies using sign-language material (e.g., Rönnberg et al, 2004 ; Bola et al, 2017 ; Cardin et al, 2018 ; Andin et al, 2021 ), the superior temporal regions (auditory cortex) and occipito-parietal regions (in speech-sign bilinguals, Rönnberg et al, 2004 ) are activated to a greater extent for deaf compared to hearing individuals. However, in a recent neuroimaging study from our lab, we found that the activation of auditory cortex did not increase with increasing WM load in a sign language-based task, suggesting a general sensory-perceptual processing role in response to visual linguistic material ( Andin et al, 2021 ) in line with functional preservation. Further, we found support of a modality-specific pattern in relation to the degradation of the sign-language signal.…”

Associations Between Sign Language Skills and Resting-State Functional Connectivity in Deaf Early Signers

“…Participants were recruited for a larger project (see Andin et al, 2021) and 15 (out of 17) had complete data on measures of sign language phonological and sentence processing, as well as an fMRI resting-state session (mean age = 35.0, SD = 7.8, min 22, max 48). All were right-handed and had normal or correctedto-normal vision.…”

Sensory experience modulates the reorganisation of temporal auditory regions for executive processing

“…However, the study of deafness and blindness suggests that this preference might be at least partially driven by environmental sensory experience, given that reorganisation for cognitive processing has been observed in sensory areas of deaf and blind individuals (Amedi et al, 2003, 2004; Bedny et al, 2011; Buchsbaum et al, 2005; Cardin et al, 2018; Ding et al, 2015). For example, previous studies have shown recruitment for visual working memory in the posterior superior temporal cortex (pSTC) of deaf individuals (Andin et al, 2021; Buchsbaum et al, 2005; Cardin et al, 2018; Ding et al, 2015), suggesting a change in function in this area from auditory to cognitive processing as a consequence of deafness. While crossmodal plasticity usually refers to the adaptation of sensory brain regions to processing information from a different sensory modality (Cardin et al, 2020a; Cardin et al, 2020b; Frasnelli et al, 2011; Heimler et al, 2015; Kral, 2007; Merabet & Pascual-Leone, 2010; Ricciardi et al, 2020), these working memory responses in pSTC seem to suggest that in the absence of early sensory stimulation, a sensory region can change its function as well as the sensory modality to which it responds (Bedny, 2017; Cardin et al, 2020b).…”