Simulating the emergence of the Visual Word Form Area: Recycling a convolutional neural network for reading

Abstract

The visual word form area (VWFA) is a region of human inferotemporal cortex that emerges at a fixed location in occipitotemporal cortex during reading acquisition, and systematically responds to written words in literate individuals. According to the neuronal recycling hypothesis, this region arises through the repurposing, for letter recognition, of a subpart of the ventral visual pathway initially involved in face and object recognition. Furthermore, according to the biased connectivity hypothesis, its universal localization is due to pre-existing connections from this subregion to areas involved in spoken language processing. Here, we evaluate those hypotheses in an explicit computational model. We trained a deep convolutional neural network of the ventral visual pathway, first to categorize pictures, and then to recognize written words invariantly for case, font and size. We show that the model can account for many properties of the VWFA, particularly when a subset of units possesses a biased connectivity to word output units. The network develops a sparse, invariant representation of written words, based on a restricted set of reading-selective units. Their activation mimics several properties of the VWFA, and their lesioning causes a reading-specific deficit. Our simulation fleshes out the neuronal recycling hypothesis, and make several testable predictions concerning the neural code for written words.

Convolutional Neural Networks Trained to Identify Words Provide a Good Account of Visual Form Priming Effects

“…The success of convolutional networks is remarkable as their architectures were not developed to support word recognition (they were designed to perform well on object recognition) and they classify pixel images of words (rather artificial encodings of letter strings). The findings add to the recent work of (Hannagan et al, 2021) suggesting that convolutional networks may capture key aspects of visual word identification.…”

“…It seems that these learned representations are also able to account for a substantial amount of of form priming effects observed in humans. The observation that CNNs not only account for a range of empirical phenomena regarding human visual word identification, but in addition, perform well on various brain benchmarks for visual object identification lends some support to the "recycling hypothesis" according to which a subpart of the ventral visual pathway initially involved in face and object recognition visual is repurposed for letter recognition (Hannagan et al, 2021).…”

“…More relevant for the present context, Hannagan et al (2021) demonstrated that training a CNN (CORnet-Z; Kubilius et al 2018) to recognize words leads the model to reproduce some key findings regarding visual-orthographic processing in the human visual word form area (VWFA) as observed with fMRI, such as case, font, and word length invariance. In addition, the model's word recognition ability were mediated by a restricted set of reading-selective units.…”

“…After pretraining the final classifier layer was removed and models were trained to classify images of 1000 different words (same number of words as used by Hannagan et al 2021), one-hot encoded. All 420 six letter words from From Priming Project were used and the remaining 580 were sourced from Google's Trillion Word Corpus (Google, 2011).…”

Repetition Probability Effects for Chinese Characters and German Words in the Visual Word Form Area

“…In fact, it has been proposed that the VWFA emerges as the result of "cultural recycling" due to the potential re-use of neural structures previously related to face and object processing (Dehaene & Cohen, 2007). As a result, it has also been proposed that many structural, connectivity and functional properties of the VWFA are inherited from evolutionarily older brain circuits, such as the face-processing network (Hannagan et al, 2021). Thus, it is not surprising that similar P(rep) modulation of RS has been found within the VWFA for written words and in the FFA for faces (Summerfield et al, 2008).…”