Brain correlates of constituent structure in sign language comprehension

Abstract

During sentence processing, areas of the left superior temporal sulcus, inferior frontal gyrus and left basal ganglia exhibit a systematic increase in brain activity as a function of constituent size, suggesting their involvement in the computation of syntactic and semantic structures. Here, we asked whether these areas play a universal role in language and therefore contribute to the processing of non-spoken sign language. Congenitally deaf adults who acquired French sign language as a first language and written French as a second language were scanned while watching sequences of signs in which the size of syntactic constituents was manipulated. An effect of constituent size was found in the basal ganglia, including the head of the caudate and the putamen. A smaller effect was also detected in temporal and frontal regions previously shown to be sensitive to constituent size in written language in hearing French subjects (Pallier et al., 2011). When the deaf participants read sentences versus word lists, the same network of language areas was observed. While reading and sign language processing yielded identical effects of linguistic structure in the basal ganglia, the effect of structure was stronger in all cortical language areas for written language relative to sign language. Furthermore, cortical activity was partially modulated by age of acquisition and reading proficiency. Our results stress the important role of the basal ganglia, within the language network, in the representation of the constituent structure of language, regardless of the input modality.

Emmorey_Reading_Chapter_2020

“…The participants' task was to respond to occasional probe sentences that explicitly asked them to press a button. Moreno et al (2018) found that reading sentences compared to lists of words engaged a strongly left-lateralized network for deaf readers, including middle and inferior frontal cortices and superior and middle temporal cortices, replicating the results found for hearing readers (Pallier et al, 2011). Consistent with Hirshorn et al (2014), neural activation was also observed in right superior temporal cortex (including auditory regions) for reading sentences compared to word lists.…”

“…Emmorey et al (2016) found that activation within right inferior frontal cortex was positively associated with reading skill when deaf readers made a semantic decision to words. At the sentence level, Moreno et al (2018) found that activation in right (and left) superior temporal cortex (including auditory regions) was positively correlated with lexical ability for deaf readers. Hirshorn et al (2014) also found greater bilateral activation in auditory regions when deaf readers (signers and non-signers) comprehended sentences compared to their reading-matched hearing peers.…”

“…Finally, Moreno, Limousin, Dehaene, and Pallier (2018) used fMRI to compare sentence processing in written French and French Sign Language (LSF) for deaf adults who acquired LSF as their first language. The results with written French were compared to results from a previous The neurobiology of reading 25 study using the same materials with hearing French readers (Pallier, Devauchelle, & Dehaene, 2011).…”

Language and Sensory Neural Plasticity in the Superior Temporal Cortex of the Deaf

“…Cortical structures that are deprived of their normal sensory input may become responsive to the stimulation of adjacent receptors, a process that is generally known as cross-modal plasticity or cross-modal reorganization [ 1 ]. In human brain imaging studies, there is growing evidence showing that, in early bilaterally deaf adults, the superior temporal cortex (STC) may experience cross-modal recruitment of different visual inputs, such as visual motion [ 2 – 8 ], biological motion [ 9 – 11 ], sign language [ 11 – 19 ], and silent speech reading [ 15 , 20 – 23 ]. Animal models have also confirmed the dystrophic change that occurs when the auditory cortex fails to develop typically due to the absence of auditory input [ 24 – 28 ].…”

“…Visual-related responses in the STC of deaf subjects could result from long-term auditory deprivation (e.g., missing auditory sensory input) but could also be caused by other dynamic cognitive functions (e.g., sign language learning) [ 1 , 12 , 16 , 19 , 29 , 30 ]. In the previous studies, STC activity was found to positively correlate with the duration of deafness or the age at cochlear implantation [ 2 , 18 , 31 – 35 ], suggesting that functional reorganization was likely to take place in the auditory cortex over a considerable period of time.…”