The problem of multimodal concurrent serial order in behavior

Kolodny, Oren; Edelman, Shimon

doi:10.1016/j.neubiorev.2015.07.009

Cited by 13 publications

(13 citation statements)

References 152 publications

(184 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the abstract computational level, our view holds that language users aim to influence the cognitive state, and accordingly the behaviour of others (and of themselves, when resorting to internal speech). This implies that the key computational problem that the cognitive system needs to address is choosing what to say next (note that this is a subset of the general problem in the control of behaviour, which is deciding what to do next; [7,48]). On the level of representation and algorithms, language production is supported by a system of options structured like a graph, whose nodes correspond to the discrete combinatorial elements of language-phonemes on one level; morphemes or words on another [49][50][51].…”

Section: Structuring the Open Questions About Language And Its Evolutionmentioning

confidence: 99%

The evolution of the capacity for language: the ecological context and adaptive value of a process of cognitive hijacking

Kolodny

Edelman

2018

Phil. Trans. R. Soc. B

Self Cite

View full text Add to dashboard Cite

Language plays a pivotal role in the evolution of human culture, yet the evolution of the capacity for language-uniquely within the hominin lineage-remains little understood. Bringing together insights from cognitive psychology, neuroscience, archaeology and behavioural ecology, we hypothesize that this singular occurrence was triggered by exaptation, or 'hijacking', of existing cognitive mechanisms related to sequential processing and motor execution. Observed coupling of the communication system with circuits related to complex action planning and control supports this proposition, but the prehistoric ecological contexts in which this coupling may have occurred and its adaptive value remain elusive. Evolutionary reasoning rules out most existing hypotheses regarding the ecological context of language evolution, which focus on ultimate explanations and ignore proximate mechanisms. Coupling of communication and motor systems, although possible in a short period on evolutionary timescales, required a multi-stepped adaptive process, involving multiple genes and gene networks. We suggest that the behavioural context that exerted the selective pressure to drive these sequential adaptations had to be one in which each of the systems undergoing coupling was necessary or highly beneficial, as well as frequent and recurring over evolutionary time. One such context could have been the teaching of tool production or tool use. In the present study, we propose the Cognitive Coupling hypothesis, which brings together these insights and outlines a unifying theory for the evolution of the capacity for language.This article is part of the theme issue 'Bridging cultural gaps: interdisciplinary studies in human cultural evolution'.

show abstract

Section: Structuring the Open Questions About Language And Its Evolutionmentioning

confidence: 99%

The evolution of the capacity for language: the ecological context and adaptive value of a process of cognitive hijacking

Kolodny

Edelman

2018

Phil. Trans. R. Soc. B

Self Cite

View full text Add to dashboard Cite

show abstract

“…From Lashley (1951) to Elman (1990) and Kolodny and Edelman (2015) , understanding the temporal structure of human behavior has been recognized as one of the most fundamental problems in psychology. Recently, advances in technology have allowed us to collect large datasets of high-density behavioral streams in naturalistic scenarios ( de Barbaro, 2019 ).…”

Section: Overall Discussionmentioning

confidence: 99%

“…Analysis of these rich corpora of behavioral data faces three main challenges. The first challenge is that our world is profoundly multimodal (Smith and Gasser, 2005;Kolodny and Edelman, 2015). Behaviors are organized with multiple time-locked sensory-motor systems which influence each other simultaneously (Garrod and Pickering, 2009;Louwerse et al, 2012;Fusaroli and Tylén, 2016).…”

Section: Introductionmentioning

confidence: 99%

Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series

Barbaro

Abney

et al. 2020

Front. Psychol.

View full text Add to dashboard Cite

The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the non-linear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique's application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a "Programming Basics" module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data.

show abstract

“…From Lashley (1951) to Elman (1990) and Kolodny and Edelman (2015), understanding the temporal structure of human behavior has been recognized as one of the most fundamental problems in psychology. Recently, advances in technology have allowed us to collect large datasets of highdensity behavioral streams in naturalistic scenarios (de Barbaro, 2019).…”

Section: Overall Discussionmentioning

confidence: 99%

Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series

Xu¹,

Barbaro²,

Abney³

et al. 2019

Preprint

View full text Add to dashboard Cite

The temporal structure of behavior contains a rich source of information about its dynamic organization, origins, and development. Today, advances in sensing and data storage allow researchers to collect multiple dimensions of behavioral data at a fine temporal scale both in and out of the laboratory, leading to the curation of massive multimodal corpora of behavior. However, along with these new opportunities come new challenges. Theories are often underspecified as to the exact nature of these unfolding interactions, and psychologists have limited ready-to-use methods and training for quantifying structures and patterns in behavioral time series. In this paper, we will introduce four techniques to interpret and analyze high-density multi-modal behavior data, namely, to: (1) visualize the raw time series, (2) describe the overall distributional structure of temporal events (Burstiness calculation), (3) characterize the nonlinear dynamics over multiple timescales with Chromatic and Anisotropic Cross-Recurrence Quantification Analysis (CRQA), (4) and quantify the directional relations among a set of interdependent multimodal behavioral variables with Granger Causality. Each technique is introduced in a module with conceptual background, sample data drawn from empirical studies and ready-to-use Matlab scripts. The code modules showcase each technique’s application with detailed documentation to allow more advanced users to adapt them to their own datasets. Additionally, to make our modules more accessible to beginner programmers, we provide a “Programming Basics” module that introduces common functions for working with behavioral timeseries data in Matlab. Together, the materials provide a practical introduction to a range of analyses that psychologists can use to discover temporal structure in high-density behavioral data.

show abstract

The problem of multimodal concurrent serial order in behavior

Cited by 13 publications

References 152 publications

The evolution of the capacity for language: the ecological context and adaptive value of a process of cognitive hijacking

The evolution of the capacity for language: the ecological context and adaptive value of a process of cognitive hijacking

Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series

Finding Structure in Time: Visualizing and Analyzing Behavioral Time Series

Contact Info

Product

Resources

About