Computationelle Neurolinguistik

Graben, Peter beim; Drenhaus, Heiner

doi:10.1515/zgl-2012-0006

Cited by 5 publications

(3 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another important scalar observable, e.g. used by beim Graben and Drenhaus (2012) as a neuronal observation model, is Smolensky's harmony (Smolensky 1986…”

Section: Invariants In Dynamical Systemsmentioning

confidence: 99%

“…Similarly, Frank et al (2015) used different ERP components in the EEG, such as the N400 (a deflection of negative polarity appearing about 400 ms after stimulus onset as a marker of lexical-semantic access) for such statistical modeling. Beim Graben and Drenhaus (2012) correlated the temporally integrated ERP during the understanding of negative polarity items (Krifka 1995) with the harmony observable of a recurrent neural network (Smolensky 2006), thereby implementing a formal language processor as a vector symbolic architecture (Gayler 2006, Schlegel et al 2021. Another neural network model of the N400 ERP-component is due to Rabovsky and McRae (2014), and to Rabovsky et al (2018) who related this marker with neural prediction error and semantic updating as observation models.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Invariants for neural automata

Uria-Albizuri¹,

Carmantini²,

Graben³

et al. 2023

Preprint

View full text Add to dashboard Cite

Computational modeling of neurodynamical systems often deploys neural networks and symbolic dynamics. One particular way for combining these approaches within a framework called vector symbolic architectures leads to neural automata. An interesting research direction we have pursued under this framework has been to consider mapping symbolic dynamics (e.g. performed by Turing machines) onto neurodynamics, represented as neural automata. This representation theory, enables us to ask questions, such as, how does the brain implement Turing computations. Specifically, in this representation theory, neural automata result from the assignment of symbols and symbol strings to numbers, known as Gödel encoding. Under this assignment symbolic computation becomes represented by trajectories of state vectors in a real phase space, that allows for statistical correlation analyses with realworld measurements and experimental data. However, these assignments are usually completely arbitrary. Hence, it makes sense to address the problem question of, which aspects of the dynamics observed under such a representation is intrinsic to the dynamics and which are not. In this study, we develop a formally rigorous mathematical framework for the investigation of symmetries and invariants of neural automata under different encodings. As a central concept we define patterns of equality for such systems. We consider different macroscopic observables, such as the mean activation level of the neural network, and ask for their invariance properties. Our main result shows that only step functions that are defined over those patterns of equality are invariant under symbolic recodings, while the mean activation, e.g., is not. Our work could be of substantial importance for related regression studies of real-world measurements with neurosymbolic processors for avoiding confounding results that are dependant on a particular encoding and not intrinsic to the dynamics.

show abstract

“…Another important scalar observable, e.g. used by beim Graben and Drenhaus (2012) as a neuronal observation model, is Smolensky's harmony (Smolensky 1986…”

Section: Invariants In Dynamical Systemsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Invariants for neural automata

Uria-Albizuri¹,

Carmantini²,

Graben³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, researchers in natural language processing and computational psycholinguistics have developed and refined computational models that reflect the mechanisms of language processing and produce human-like language output (Trueswell et al, 1994;Jurafsky, 1996;Hale, 2001;Levy et al, 2009;McRae and Matsuki, 2013;Smith and Levy, 2013;Linzen et al, 2016;Caliskan et al, 2017;Lau et al, 2017). In a relatively new field referred to as computational neurolinguistics, researchers attempt to model the direct link between linguistic features and biological bases in the brain (Arbib and Caplan, 1979;Hagoort, 2003;Beim Graben et al, 2008;Huyck, 2009;Beim Graben and Drenhaus, 2012;Barrès et al, 2013;Rabovsky and McRae, 2014;Frank et al, 2015;Brouwer and Crocker, 2017;Carmantini et al, 2017;Venhuizen et al, 2019;Brouwer et al, 2021). The recent advancement in computational modeling of language based on deep neural networks further adds innovations to those computationally oriented areas.…”

Section: Introductionmentioning

confidence: 99%

Investigating a neural language model’s replicability of psycholinguistic experiments: A case study of NPI licensing

Shin

Song

2023

Front. Psychol.

View full text Add to dashboard Cite

The recent success of deep learning neural language models such as Bidirectional Encoder Representations from Transformers (BERT) has brought innovations to computational language research. The present study explores the possibility of using a language model in investigating human language processes, based on the case study of negative polarity items (NPIs). We first conducted an experiment with BERT to examine whether the model successfully captures the hierarchical structural relationship between an NPI and its licensor and whether it may lead to an error analogous to the grammatical illusion shown in the psycholinguistic experiment (Experiment 1). We also investigated whether the language model can capture the fine-grained semantic properties of NPI licensors and discriminate their subtle differences on the scale of licensing strengths (Experiment 2). The results of the two experiments suggest that overall, the neural language model is highly sensitive to both syntactic and semantic constraints in NPI processing. The model’s processing patterns and sensitivities are shown to be very close to humans, suggesting their role as a research tool or object in the study of language.

show abstract

Invariants for neural automata

Uria-Albizuri

Carmantini²,

Graben

et al. 2023

Cogn Neurodyn

View full text Add to dashboard Cite

Computational modeling of neurodynamical systems often deploys neural networks and symbolic dynamics. One particular way for combining these approaches within a framework called vector symbolic architectures leads to neural automata. Specifically, neural automata result from the assignment of symbols and symbol strings to numbers, known as Gödel encoding. Under this assignment, symbolic computation becomes represented by trajectories of state vectors in a real phase space, that allows for statistical correlation analyses with real-world measurements and experimental data. However, these assignments are usually completely arbitrary. Hence, it makes sense to address the problem which aspects of the dynamics observed under a Gödel representation is intrinsic to the dynamics and which are not. In this study, we develop a formally rigorous mathematical framework for the investigation of symmetries and invariants of neural automata under different encodings. As a central concept we define patterns of equality for such systems. We consider different macroscopic observables, such as the mean activation level of the neural network, and ask for their invariance properties. Our main result shows that only step functions that are defined over those patterns of equality are invariant under symbolic recodings, while the mean activation, e.g., is not. Our work could be of substantial importance for related regression studies of real-world measurements with neurosymbolic processors for avoiding confounding results that are dependant on a particular encoding and not intrinsic to the dynamics.

show abstract

Computationelle Neurolinguistik

Cited by 5 publications

References 50 publications

Invariants for neural automata

Invariants for neural automata

Investigating a neural language model’s replicability of psycholinguistic experiments: A case study of NPI licensing

Invariants for neural automata

Contact Info

Product

Resources

About