Interconnected growing self-organizing maps for auditory and semantic acquisition modeling

Abstract

Based on the incremental nature of knowledge acquisition, in this study we propose a growing self-organizing neural network approach for modeling the acquisition of auditory and semantic categories. We introduce an Interconnected Growing Self-Organizing Maps (I-GSOM) algorithm, which takes associations between auditory information and semantic information into consideration, in this paper. Direct phonetic–semantic association is simulated in order to model the language acquisition in early phases, such as the babbling and imitation stages, in which no phonological representations exist. Based on the I-GSOM algorithm, we conducted experiments using paired acoustic and semantic training data. We use a cyclical reinforcing and reviewing training procedure to model the teaching and learning process between children and their communication partners. A reinforcing-by-link training procedure and a link-forgetting procedure are introduced to model the acquisition of associative relations between auditory and semantic information. Experimental results indicate that (1) I-GSOM has good ability to learn auditory and semantic categories presented within the training data; (2) clear auditory and semantic boundaries can be found in the network representation; (3) cyclical reinforcing and reviewing training leads to a detailed categorization as well as to a detailed clustering, while keeping the clusters that have already been learned and the network structure that has already been developed stable; and (4) reinforcing-by-link training leads to well-perceived auditory–semantic associations. Our I-GSOM model suggests that it is important to associate auditory information with semantic information during language acquisition. Despite its high level of abstraction, our I-GSOM approach can be interpreted as a biologically-inspired neurocomputational model.

The emergence of phonetic–phonological features in a biologically inspired model of speech processing

“…an amount of model neurons located in the same cortical area, most likely in a few mm 2 regions within a part of the prefrontal as well as within a part of the temporo-parietal cortex, see representing all trained or learned syllables of the target language by specific model neurons. This neural map is called phonetic map (P-MAP) and is implemented as a growing self-organizing map in our quantitative approach (GSOM, see Cao et al (2014)). Each syllable is represented here by an individual model neuron within the phonetic map ( Fig.…”

“…Our model of speech processing Cao et al, 2014) can be divided into a cognitive lexical and a sensorimotor or phonetic part (Fig. 1).…”

“…Thus we introduced growing self-organizing maps (GSOMs) for modeling parts of the mental lexicon (i.e. semantic map or S-MAP, Cao et al, 2013) as well as for modeling the sensorimotor part of the speech processing system, here called phonetic map or P-MAP (Cao et al, 2014). Moreover a first version of a neural coupling between P-MAP and S-MAP has been described in Cao et al (2014) as well.…”

“…But if we intend to model speech learning, these approaches are not very helpful, because the phonological representation of a target language is not present at the very beginning, but emerges during speech acquisition . It is the goal of this paper to describe an integrative model of speech processing, exhibiting a connection between cognitive and sensorimotor parts, which allows to model these complex aspects of the phonetic-phonological interface including the option of an emerging phonological representation during speech acquisition (Kröger, Birkholz, & Neuschaefer-Rube, 2011;Kröger, Birkholz, Kannampuzha, Kaufmann, & Neuschaefer-Rube, 2011;Cao, Li, Fang, & Kröger, 2013;Cao, Li, Fang, Kaufmann, & Kröger, 2014).…”

“…It is the goal of this paper to describe simulations of speech learning using the current version of the model (Cao et al, 2014) focusing on the emergence of phonetic-phonological features. This occurs on the basis of an ordering of speech items at the level of the self-organizing phonetic map and on the basis of the fact that these speech items are linked unambiguously with meanings (i.e.…”