Visual Working Memory of Chinese Characters and Expertise: The Expert’s Memory Advantage Is Based on Long-Term Knowledge of Visual Word Forms

Abstract

People unfamiliar with Chinese characters show poorer visual working memory (VWM) performance for Chinese characters than do literates in Chinese. In a series of experiments, we investigated the reasons for this expertise advantage. Experiments 1 and 2 showed that the advantage of Chinese literates does not transfer to novel material. Experts had similar resolution as novices for material outside of their field of expertise, and the memory of novices and experts did not differ when detecting a big change, e.g., when a character's color was changed. Memorizing appears to function as a rather abstract representation of word forms because memory for characters' fonts was poor independently of expertise (Experiment 3), though still visual. Distractors that were highly similar conceptually did not increase memory errors, but visually similar distractors impaired memory (Experiment 4). We hypothesized that literates in Chinese represent characters in VWM as tokens of visual word forms made available by long-term memory. In Experiment 5, we provided novices with visual word form knowledge. Participants subsequently performed a change detection task with trained and novel characters in a functional magnetic resonance experiment. We analyzed set size-and trainingdependent effects in the intraparietal sulcus (IPS) and the visual word form area. VWM for trained characters was better than for novel characters. Neural activity increased with set size and at a slower rate for trained than for novel characters. All conditions approached the same maximum, but novel characters reached the maximum at a smaller set size than trained characters. The time course of the bold response depended on set size and knowledge status. Starting from the same initial maximum, neural activity at small set sizes returned to baseline more quickly for trained characters than for novel characters. Additionally, high performers showed generally more neural activity in the IPS than low performers. We conclude that experts' better performance in working memory (WM) is caused by the availability of visual long-term representations (word form types) that allow a sparse representation of the perceived stimuli and make even small changes big because they cause a type change that is easily detected.

A model of working memory for latent representations

“…In the proposed framework, these two features are compression of visual information and categorical representations. Later, we show how these aspects of visual knowledge would affect WM capacity and precision of retrieved items for familiar and novel items (Brady, et al, 2008; Yu et al, 1985; Zhang & Simon, 1985; Zimmer & Fischer, 2020).…”

“…In the simulations here, color and shape are treated as distinct attributes but in a larger model, the set of tunable attributes could include any stimulus dimension for which there are distinct representations. This would be a separate latent space in the context of an autoencoder, or a separate cortical area in the context of neuroscience (Konkle & Caramazza; Representing Novel stimuli: WM performance is more efficient for previously learned items (Yu et al, 1985;Zimmer & Fischer, 2020), however, humans can still encode novel configurations that they have not seen before (Lake et al, 2011; see also Experiment 1). Similarly, the MLR model stores and retrieves novel shapes that it has not seen before, although those memory reconstructions are less precise compared to shape categories that the model was trained on.…”

“…For instance, Alvarez & Cavanagh (2004) demonstrated that the number of items stored in WM is affected by stimulus complexity, with particularly poor performance for Chinese characters. Zimmer & Fischer (2020) expanded on this by showing that the difficulty in remembering Chinese characters is specific to individuals who are not readers of the language. That is to say, the WM capacity for Chinese characters is higher if observers have already been trained on those characters.…”

“…Knowledge that emerges from long-term familiarity with particular shapes or statistically common featural combinations enables us to recognize and remember complex objects (i.e., the prototypical shape of a car, or the strokes that comprise a digit). It is widely acknowledged that such information is crucial for building WM representations (Cowan, 1999; Brady, Konkle & Alvarez, 2009; Oberauer, 2009;) but there has been little attempt, if any, to mechanistically implement the role of visual knowledge in WM models in spite of abundant behavioral research in this domain (Alvarez & Cavanagh, 2004; Chen & Cowan, 2005; 2009; Hulme, Maughen & Brown, 1991; Ngiam, et al, 2019; Ngiam, Brissenden & Awh, 2019; Yu et al, 1985; Zhang & Simon, 1985; Zimmer & Fischer, 2020). For instance, Alvarez & Cavanagh (2004) demonstrated that the number of items stored in WM is affected by stimulus complexity, with particularly poor performance for Chinese characters.…”

“…Representing Novel stimuli : WM performance is more efficient for previously learned items (Yu et al, 1985; Zimmer & Fischer, 2020), however, humans can still encode novel configurations that they have not seen before (Lake et al, 2011; see also Experiment 1). Similarly, MLR stores and retrieves novel shapes that it has not seen before, although those memory reconstructions are less precise compared to shape categories that the model was trained on.…”