A continuous-time neural model for sequential action

Abstract

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.

Predictive Movements and Human Reinforcement Learning of Sequential Action

“…Complex action sequences are not simple stimulus-response chains, but rather require representing sequential context in order to learn (Lashley, 1951). Moreover, human behavior is often thought of as predictive-indeed, many models of sequential learning operate on a predictionbased error signal (Botvinick & Plaut, 2004;Kachergis, Wyatt, O'Reilly, de Kleijn & Hommel, 2014). Thus, it is problematic that the discrete button-presses in the SRT paradigm cannot distinguish an anticipatory response due to correctly predicting the stimulus (or a slow response due to an incorrect prediction) from reactive responses (although perhaps pre-potentiated) based on the cue (Marcus, Karatekin, & Markiewicz, 2006).…”

The theory of event coding (TEC) as embodied-cognition framework

“…Acquiring these bindings allows the agent to run them internally to simulate the action without actually activating the motor patterns (if intentional weighting deactivates the motor components). The acquisition of multiple sensorimotor events allows the agent to construct more complex event sequences, such as for making coffee ( Kachergis et al, 2014 ). These representations provide information about how to move from one situation to another to reach a distant goal, which can be used to simulate and to compare alternative problem-solving strategies.…”

The principles of goal-directed decision-making: from neural mechanisms to computation and robotics

“…in terms of macro-actions or skills such as 'log into email account') thus disregarding the exact details of action at least initially, potentially extending the scope and complexity of human reasoning. Kachergis et al [29] propose a model for sequential behaviour. The model uses principles from a biology-based computational model (Leabra, [30]) to implement the constructs of a psychological theory of ideomotor action (TEC, [31]), demonstrating how it can work in hierarchically structured tasks such as coffee making.…”

“…habitual) systems. The Special Issue also revolves around distinct computational schemes that are proposed to define goal-directed choice in a normative framework, which include model-based reinforcement learning [26], hierarchical reinforcement learning [28], ideomotor action [29], active inference [27] and game-theoretic approaches in the social domain [33]. Clearly, these and other proposals remain to be assessed empirically, and an important open question is whether they apply to the wide research field in which goal-directed choice is at play: from simple laboratory studies to complex ecological scenarios [39] and malfunctioning [38].…”