A Critical Review of Network‐Based and Distributional Approaches to Semantic Memory Structure and Processes

Abstract

Some of the earliest work on understanding how concepts are organized in memory used a network‐based approach, where words or concepts are represented as nodes, and relationships between words are represented by links between nodes. Over the past two decades, advances in network science and graph theoretical methods have led to the development of computational semantic networks. This review provides a modern perspective on how computational semantic networks have proven to be useful tools to investigate the structure of semantic memory as well as search and retrieval processes within semantic memory, to ultimately model performance in a wide variety of cognitive tasks. Regarding representation, the review focuses on the distinctions and similarities between network‐based (based on behavioral norms) approaches and more recent distributional (based on natural language corpora) semantic models, and the potential overlap between the two approaches. Capturing the type of relation between concepts appears to be particularly important in this modeling endeavor. Regarding processes, the review focuses on random walk models and the degree to which retrieval processes demand attention in pursuit of given task goals, which dovetails with the type of relation retrieved during tasks. Ultimately, this review provides a critical assessment of how the network perspective can be reconciled with distributional and machine‐learning‐based perspectives to meaning representation, and describes how cognitive network science provides a useful conceptual toolkit to probe both the structure and retrieval processes within semantic memory.

Child-oriented word associations improve models of early word learning

“…However, corpora of child-directed speech provide only a snapshot of input provided to a child. Indeed, a criticism of distributed semantic models based on text corpora is that the structure of semantic representations is heavily biased by the size of the text corpora and parameter tuning decisions (i.e., the grounding problem; Kumar, 2021;Kumar et al, 2022). To gather semantic data for the comprehensive list of words on the CDI, we also collected word association data by probing which words naturally go together after experimentally establishing a child-oriented context.…”

“…Given this, it is possible that the word association data derived from the child-oriented task also capture aspects of the multimodal learning process that the child may experience, such as learning biases based on perceptual and affective features (e.g., Berman et al, 2013;McDonough et al, 2011;Perry et al, 2015;Perry & Samuelson, 2011). However, it is important to emphasize that this suggestion is only speculative (see Kumar et al, 2022 for a discussion about the utility of semantic network approaches for offering insight into both the structure of knowledge representation and the processes that are in play).…”

Semantic Memory Search and Retrieval in a Novel Cooperative Word Game: A Comparison of Associative and Distributional Semantic Models

“…However, it is also possible that free association represents unique conceptual information that is not contained within linguistic corpora‐based DSMs, and tasks that tap into such conceptual processing (such as the speaker and guesser tasks in Connector) may benefit from this representational overlap. Therefore, although comparing associative models to DSMs may be problematic (for detailed arguments, see Jones et al., 2015), it is important to understand the nature of the information contained within these representations, after controlling for differences in the representational format itself (see Kumar, Steyvers, Balota, 2021 for a discussion). In the present work, we ensured that associative models and DSMs were compared in the fairest way possible by constructing WAS and ensuring all words were represented within a high‐dimensional space across the two classes of models.…”

Semantic network analysis (SemNA): A tutorial on preprocessing, estimating, and analyzing semantic networks.

“…Network science is based on mathematical graph theory, which provides quantitative methods to represent and investigate complex systems as networks (Baronchelli, Ferrer-i-Cancho, Pastor-Satorras, Chater, & Christiansen, 2013;Siew, Wulff, Beckage, & Kenett, 2019). Network science methodologies provide a powerful computational approach for modeling cognitive structures such as semantic memory (i.e., memory of word meanings, categorizations of concepts and facts, and knowledge about the world; Kumar, Steyvers, & Balota, 2021;Steyvers & Tenenbaum, 2005) and mental lexicon (i.e., word meaning, pronunciation, and syntactic characteristics; Stella, Beckage, Brede, & De Domenico, 2018;Wulff et al, 2019; for a review, see Siew, Wulff, Beckage, & Kenett, 2019).…”