Reward-related cues are an important part of our daily life as they often influence and guide our actions. This paper reviews one of the experimental paradigms used to study the effects of cues, the Pavlovian to Instrumental Transfer paradigm. In this paradigm, cues associated with rewards through Pavlovian conditioning alter motivation and choice of instrumental actions. The first transfer experiments date back to the 1940s, but only in the last decade has it been fully recognised that there are two types of transfer, specific and general. This paper presents a systematic review of both the neural substrates and the behavioral factors affecting both types of transfer. It also examines the recent application of the paradigm to study the effect of cues on human participants, both in normal and pathological conditions, and the interactions of transfer with drugs of abuse. Finally, the paper analyses the theoretical aspects of transfer to build an overall picture of the phenomenon, from early theories to recent hierarchical accounts.

Balancing habitual and deliberate forms of choice entails a comparison of their respective merits-the former being faster but inflexible, and the latter slower but more versatile. Here, we show that arbitration between these two forms of control can be derived from first principles within an Active Inference scheme. We illustrate our arguments with simulations that reproduce rodent spatial decisions in T-mazes. In this context, deliberation has been associated with vicarious trial and error (VTE) behavior (i.e., the fact that rodents sometimes stop at decision points as if deliberating between choice alternatives), whose neurophysiological correlates are "forward sweeps" of hippocampal place cells in the arms of the maze under consideration. Crucially, forward sweeps arise early in learning and disappear shortly after, marking a transition from deliberative to habitual choice. Our simulations show that this transition emerges as the optimal solution to the trade-off between policies that maximize reward or extrinsic value (habitual policies) and those that also consider the epistemic value of exploratory behavior (deliberative or epistemic policies)-the latter requiring VTE and the retrieval of episodic information via forward sweeps. We thus offer a novel perspective on the optimality principles that engender forward sweeps and VTE, and on their role on deliberate choice.Substantial evidence indicates that animal behavior is determined both by deliberative processes (i.e., based on predictions of future outcomes and rewards) and by habitual reflexes (i.e., based on stimulus-response associations; Balleine and Dickinson 1998). The former are more resource intensive and sensitive to changes in task contingencies, while the latter are cheaper but inflexible; hence whether it is optimal to call on deliberative or habitual choice depends on the trade-off between the advantage of flexibility and computational costs (Balleine and Dickinson 1998;Dolan and Dayan 2013;Lee et al. 2014). In this paper, we try to understand the contextualization of behavior and the trade-off between deliberative and habitual choice from first principles, using Active Inference and Markov decision process models of exploitation and exploration (Friston et al. 2013(Friston et al. , 2014Pezzulo et al. 2015).We focus specifically on vicarious trial and error (VTE) behavior, which is considered a hallmark of deliberation (Muenzinger 1938;Tolman 1938Tolman , 1939. This is based on the observation that, when rodents have to remember or search the correct route to a reward in a maze (e.g., a T-maze), they sometimes stop at choice points, to look left and right before choosing which direction to go. This has been interpreted as a signature of cognitive search and deliberation between the two choices (i.e., going right or left). In keeping with a role of VTE behavior for deliberation, it occurs early in learning and decreases or disappears after significant experience (Tolman 1939;van der Meer and Redish 2010;van der Meer et al. 2012) but it can incr...