Many behavioral tasks require goal-directed actions to obtain delayed reward. The prefrontal cortex appears to mediate many aspects of goal-directed function. This paper presents a model of prefrontal cortex function emphasizing the influence of goal related activity on the choice of the next motor output. The model can be interpreted in terms of key elements of Reinforcement Learning theory. Different neocortical minicolumns represent distinct sensory input states and distinct motor output actions. The dynamics of each minicolumn include separate phases of encoding and retrieval. During encoding, strengthening of excitatory connections forms forward and reverse associations between each state, the following action, and a subsequent state, which may include reward. During retrieval, activity spreads from reward states throughout the network. The interaction of this spreading activity with a specific input state directs selection of the next appropriate action. Simulations demonstrate how these mechanisms can guide performance in a range of goal-directed tasks, and provide a functional framework for some of the neuronal responses previously observed in medial prefrontal cortex during performance of spatial memory tasks in rats.Numerous behavioral tasks involve goal-directed behavior based upon a delayed reward. For example, a rat in an instrumental task must generate lever presses to obtain food reward (Corbit and Balleine, 2003;Killcross and Coutureau, 2003;Wyble et al., 2004), and a rat in a T-maze must run down the stem of the maze to obtain food reward in one arm of the maze (Jung et al., 1998;Wood et al., 2000;Baeg et al., 2003;Ferbinteanu and Shapiro, 2003). Lesions of the prefrontal cortex cause impairments in goal-directed behavior (Fuster, 1995;Miller and Cohen, 2001;Corbit and Balleine, 2003;Killcross and Coutureau, 2003), and prefrontal units show firing dependent upon the association of cues and future responses (Miller, 2000). The model presented here addresses how goal directed behavior can be mediated by populations of neurons.An extensive theoretical framework termed Reinforcement Learning (Sutton, 1988;Sutton and Barto, 1998) describes how an agent can generate behaviors for delayed rewards in its environment. Current sensory input to the agent is represented by a "state" vector, and the output of the agent is represented by "actions" which alter the state vector (i.e., moving the agent to a different location). The selection of actions is guided by value functions (associating states with future reward) and state-action-value functions (associating actions in specific states with future reward). These functions are often learned using variants of temporal difference (TD) learning (Sutton, 1988;Sutton and Barto, 1998).Research has focused on the similarity between the error term in TD learning and the activity of dopaminergic neurons (Montague et al., 1996;Schultz et al., 1997). The basal ganglia have been proposed to provide circuitry for computation of TD learning (Houk et al., 1995). A...