Letter processing in upright bigrams predicts reading fluency variations in children.

Agrawal, Aakash; Nag, Sonali; Hari, K.V.S.; Arun, S. P.

doi:10.1037/xge0001175

Cited by 3 publications

(2 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This was evidenced by weaker within‐bigram interactions in readers, which predicted their reading fluency (Agrawal et al, 2019). More recently, we have found that letter interactions in upright (not inverted) bigrams explained reading fluency variations in children even after accounting for other language based measures (Agrawal et al, 2022).…”

Section: The Blessing Of Compositionalitymentioning

confidence: 85%

Using compositionality to understand parts in whole objects

Arun

2022

Eur J of Neuroscience

Self Cite

View full text Add to dashboard Cite

A fundamental question for any visual system is whether its image representation can be understood in terms of its components. Decomposing any image into components is challenging because there are many possible decompositions with no common dictionary, and enumerating the components leads to a combinatorial explosion. Even in perception, many objects are readily seen as containing parts, but there are many exceptions. These exceptions include objects that are not perceived as containing parts, properties like symmetry that cannot be localized to any single part and special categories like words and faces whose perception is widely believed to be holistic. Here, I describe a novel approach we have used to address these issues and evaluate compositionality at the behavioural and neural levels. The key design principle is to create a large number of objects by combining a small number of pre‐defined components in all possible ways. This allows for building component‐based models that explain neural and behavioural responses to whole objects using a combination of these components. Importantly, any systematic error in model fits can be used to detect the presence of emergent or holistic properties. Using this approach, we have found that whole object representations are surprisingly predictable from their components, that some components are preferred to others in perception and that emergent properties can be discovered or explained using compositional models. Thus, compositionality is a powerful approach for understanding how whole objects relate to their parts.

show abstract

Section: The Blessing Of Compositionalitymentioning

confidence: 85%

Using compositionality to understand parts in whole objects

Arun

2022

Eur J of Neuroscience

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, recently, evidence from psychophysics and time-resolved intracranial recordings suggests that frequent bigrams may not contribute to recognition, at least during the first ~300 ms of word recognition, where only frequent letters are separated from rare letters or non-letters (Agrawal et al, 2020;McCloskey et al, 2013;Woolnough et al, 2020). What develops with literacy is the compositionality of visual word representation, which increases the dissimilarity and independence between individual letters at nearby locations (Agrawal et al, 2019(Agrawal et al, , 2022. Compared to contextual bigram coding, ordinal letter coding provides a better fit to the psychophysical distance between letter strings (Agrawal et al, 2020), the early intracranial responses to written words (Woolnough et al, 2020), and the responses of word selective units in artificial deep networks (Hannagan et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Dissecting the neuronal mechanisms of invariant word recognition

Agrawal,

Dehaene

2023

Preprint

Self Cite

View full text Add to dashboard Cite

Learning to read places a strong challenge on the visual system. Years of expertise lead to a remarkable capacity to separate highly similar letters and encode their relative positions, thus distinguishing words such as FORM and FROM invariantly over a large range of sizes and absolute positions. How neural circuits achieve invariant word recognition remains unknown. Here we address this issue through computational modeling and brain imaging. We first trained deep neural network models to recognize written words, then analyzed the reading-specialized units that emerged in deep layers. With literacy, units became sensitive to specific letter identities and their distance from the blank space at the left or right of a word, thus acting as “space bigrams” encoding ordinal position using an approximate number code. Using 7T functional MRI and magnetoencephalography in adults, we localized the predicted ordinal code anatomically (visual word form area) and temporally (∼220ms). The proposed neuronal mechanism for invariant word recognition can explain reading errors and makes precise predictions about how position-invariant neural codes arise in brains and artificial systems.

show abstract

Cracking the neural code for word recognition in convolutional neural networks

Agrawal,

Dehaene

2024

PLoS Comput Biol

View full text Add to dashboard Cite

Learning to read places a strong challenge on the visual system. Years of expertise lead to a remarkable capacity to separate similar letters and encode their relative positions, thus distinguishing words such as FORM and FROM, invariantly over a large range of positions, sizes and fonts. How neural circuits achieve invariant word recognition remains unknown. Here, we address this issue by recycling deep neural network models initially trained for image recognition. We retrain them to recognize written words and then analyze how reading-specialized units emerge and operate across the successive layers. With literacy, a small subset of units becomes specialized for word recognition in the learned script, similar to the visual word form area (VWFA) in the human brain. We show that these units are sensitive to specific letter identities and their ordinal position from the left or the right of a word. The transition from retinotopic to ordinal position coding is achieved by a hierarchy of “space bigram” unit that detect the position of a letter relative to a blank space and that pool across low- and high-frequency-sensitive units from early layers of the network. The proposed scheme provides a plausible neural code for written words in the VWFA, and leads to predictions for reading behavior, error patterns, and the neurophysiology of reading.

show abstract

Letter processing in upright bigrams predicts reading fluency variations in children.

Cited by 3 publications

References 59 publications

Using compositionality to understand parts in whole objects

Using compositionality to understand parts in whole objects

Dissecting the neuronal mechanisms of invariant word recognition

Cracking the neural code for word recognition in convolutional neural networks

Contact Info

Product

Resources

About