Modelling concrete and abstract concepts using brain-constrained deep neural networks

Abstract

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically ‘ground’ concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their ‘shared neurons’, thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Brain correlates of action word memory revealed by fMRI

“…Neurocomputational modelling of word learning show the emergence of action perception circuits distributed across language areas 56 , 57 . These neuronal circuit models explain why language and symbol processing activate classic language areas along with multimodal prefrontal and temporal areas, and even, depending on semantic word type, additional category-preferential areas such as the motor cortex 58 – 61 . Tomasello et al 60 simulated word-learning processes, as documented by language developmental studies (e.g., 62 ), in a frontal-temporal-occipital brain model constrained by connectivity structures at the global and local scales (for more detail about the biological constrain of the model see also 63 ).…”

Influence of language on perception and concept formation in a brain-constrained deep neural network model

“…Building on earlier modelling work [46,[56][57][58], we used a neuroanatomically grounded, neurophysiologically plausible computational model with spiking neurons and 12 model areas representing visual and motor as well as auditory and articulatory areas in frontal, temporal and occipital cortices along with adjacent multimodal hub areas that are known to be important for processing words and their meaning.…”

“…We adopted a model architecture constrained by neurobiological information and previously applied to explore neural mechanisms of semantic learning [46,56,57,59,60]. The following brain constraints were applied to the model (for a recent review of this brain-constrained modelling approach, see [27]):…”

“…In addition, to implement the learning and processing of symbols, i.e. spoken words, the articulatory motor cortex (M1 i ) and the auditory cortex (A1) was modelled, again with secondary and higher areas for each system (PM i , PF i and AB, Adapted from [46]. Schematic illustration of a structural difference between concrete (left) and abstract (right) concepts (semantic feature overlap versus family resemblance).…”

“…Figure 1. Adapted from[46]. Schematic illustration of a structural difference between concrete (left) and abstract (right) concepts (semantic feature overlap versus family resemblance).…”