Reading Increases the Compositionality of Visual Word Representations

Abstract

Reading causes widespread changes in the brain, but its effect on visual word representations is unknown. Learning to read may facilitate visual processing by forming specialized detectors for longer strings or by making word responses more predictable from single letters—that is, by increasing compositionality. We provided evidence for the latter hypothesis using experiments that compared nonoverlapping groups of readers of two Indian languages (Telugu and Malayalam). Readers showed increased single-letter discrimination and decreased letter interactions for bigrams during visual search. Importantly, these interactions predicted subjects’ overall reading fluency. In a separate brain-imaging experiment, we observed increased compositionality in readers, whereby responses to bigrams were more predictable from single letters. This effect was specific to the anterior lateral occipital region, where activations best matched behavior. Thus, learning to read facilitates visual processing by increasing the compositionality of visual word representations.

Simulating the emergence of the Visual Word Form Area: Recycling a convolutional neural network for reading

“…This conclusion is consistent with the fact that the present neural network simulations, comprising solely bottom-up connections, capture the first effect but not the second. It is also consistent with recent empirical results indicating that the main change induced by literacy is a sensitivity to letter shapes and their precise locations [40,41]. As also concluded by others [35], the simulation of recurrent interactions may be needed to capture the full temporal profile of word-related activations and to mimic fMRI signals.…”

“…bigrams. The empirical data is conflicting: there is behavioral and brain-imaging evidence that bigrams are a crucial cue to word identity [29,30,37,38,42,74], but also recent data suggesting that the bulk of bottom-up orthographic coding may be based on a conjunction of single letters and their positions [31,40,41]. The present results reconcile both, as they suggest that, in the course of learning, a neural network will make use of all available statistical cues and will develop both letter X position codes and bigram-sensitive units.…”

“…Our second goal was to see if we could predict, in anticipation of future experiments, some of the properties of the neural code for written words. It is currently controversial whether neurons in the reading pathway are specialized for whole words [36], frequent pairs of letters ('bigrams') [37,38], graphemes that map onto phonemes [39], or individual letters at a specific location [40,41]. Indeed, these possibilities are not mutually exclusive, and multiple codes may coexist, perhaps in different pathways, to support different tasks such as comprehension versus reading aloud [39,42].…”

General object-based features account for letter perception

“…Recent findings using other methodologies also corroborate our computationalbehavioral evidence for this claim. For example, both readers and non-readers of an alphabet have nearly identical representational geometries for letters, as measured using a visual search paradigm over two different Brahmic scripts (Agrawal et al, 2019). Further, in the macaque visual system, letters and words can be linearly decoded from neural responses of inferotemporal cortex in monkeys who have never been trained to distinguish between letters (Rajalingham et al, 2020).…”

“…Specifically, participants (n = 222) located the odd-letter-out as quickly as possible in displays with one letter among five other letters of a different identity (e.g., the letter a among five letter b's; Figure 1a). Visual search is faster when the target is more perceptually distinct from the distractors, and it is slower when the target is more similar to the distractors (Duncan & Humphreys, 1989); in this way, visual search time serves as an implicit measure of the perceptual similarity between stimuli (Agrawal et al, 2019;Arun, 2012;M. A. Cohen et al, 2017;Long et al, 2016Long et al, , 2017Magri & Konkle, 2019).…”