Spontaneous alternation behavior in animals: A review

Richman, Charles L.; Dember, William N.; Kim, Paul

doi:10.1007/bf02686603

Cited by 121 publications

(99 citation statements)

References 165 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, making choices according to relative arming probabilities (probability matching) (Bitterman 1965) is frequently observed in a TAB task in various animal species including monkeys (Morris et al 2006) and humans (Vulkan 2000). It is also well known that animals tend to alternate between two rewarding choices (Richman et al 1986;Lalonde 2002). It is unclear why such a behavioral strategy, which is suboptimal in the TAB task, is adopted across different animal species.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Model-based reinforcement learning under concurrent schedules of reinforcement in rodents

Huh

Jo²,

Kim³

et al. 2009

Learn. Mem.

View full text Add to dashboard Cite

Reinforcement learning theories postulate that actions are chosen to maximize a long-term sum of positive outcomes based on value functions, which are subjective estimates of future rewards. In simple reinforcement learning algorithms, value functions are updated only by trial-and-error, whereas they are updated according to the decision-maker's knowledge or model of the environment in model-based reinforcement learning algorithms. To investigate how animals update value functions, we trained rats under two different free-choice tasks. The reward probability of the unchosen target remained unchanged in one task, whereas it increased over time since the target was last chosen in the other task. The results show that goal choice probability increased as a function of the number of consecutive alternative choices in the latter, but not the former task, indicating that the animals were aware of time-dependent increases in arming probability and used this information in choosing goals. In addition, the choice behavior in the latter task was better accounted for by a model-based reinforcement learning algorithm. Our results show that rats adopt a decision-making process that cannot be accounted for by simple reinforcement learning models even in a relatively simple binary choice task, suggesting that rats can readily improve their decision-making strategy through the knowledge of their environments.Animals must continually update their behavioral strategies according to changes in an environment in order to optimize their choices. Reinforcement learning (RL) models (Sutton and Barto 1998) provide a powerful theoretical framework for understanding choice behavior in humans and animals in a dynamic environment. In theories of RL, future actions are chosen so as to maximize a long-term sum of positive outcomes, and this can be accomplished by a set of value functions that represent the amount of expected reward that is associated with particular states or actions. The value functions are continually updated based on the reward prediction error, which is the difference between the expected and actual rewards. This way, even without prior knowledge about an uncertain and dynamically changing environment, an animal can discover the structure of the environment that can be exploited for optimal choice by trial-and-error. Not surprisingly, human and monkey choice behaviors in various tasks are well described by reinforcement learning algorithms (e.g., O'Doherty et al. 2003;Barraclough et al. 2004;Lee et al. 2004;Samejima et al. 2005;Daw et al. 2006;Pessiglione et al. 2006).The updating of value functions can be achieved in two fundamentally different ways. In simple or direct RL algorithms, value functions are updated only by trial-and-error. In other words, only the value function that is associated with the chosen action is updated, and those that are associated with uncommitted actions remain unchanged. On the other hand, in indirect or model-based RL algorithms, the value functions might also change according to the decis...

show abstract

Section: Discussionmentioning

confidence: 99%

“…However, it could be because of an animal's innate tendency to alternate its choices (Richman et al 1986;Lalonde 2002). To differentiate between these two possibilities, we examined the relationship between an animal's choice and run length in the TAB task.…”

Section: Run Length Independence Of Choice Behavior In the Tab Taskmentioning

confidence: 99%

Model-based reinforcement learning under concurrent schedules of reinforcement in rodents

Huh

Jo²,

Kim³

et al. 2009

Learn. Mem.

View full text Add to dashboard Cite

show abstract

“…The SAB paradigm is a procedure proposed to reflect exploratory behavior, which also depends on the formation of working memory (Richman et al, 1986). However, anxiety and stress may negatively affect SAB (Bats et al, 2001) to the extent that it does not reflect exploratory behavior (Gerlai, 2001).…”

Section: Discussionmentioning

confidence: 99%

“…The discrete-trial procedure (Richman et al, 1986) was employed, with a daily session consisting of trial 1 (5 s in a start position and entry into an unblocked arm), 60-s intertrial period (mouse restrained in the arm) and trial 2, in which both arms were open and a mouse was expected to choose an unvisited arm after leaving the start position. An arm was considered to have been chosen if all four limbs were within the selected arm.…”

Section: Methodsmentioning

confidence: 99%

“…The initial support for the hypothesis that spatial learning and memory are impaired by lupus-like disease came from studies in which MRL-lpr mice perseverated in their response bias during extinction and reversal learning in the Morris water-maze task (Sakic et al, 1992(Sakic et al, , 1993. The aim of the present study is to evaluate the time course of spontaneous alternation behavior (SAB), a functional trait proposed to be highly sensitive to hippocampal damage (reviewed in Richman et al, 1986). More specifically, it is well documented that spatial working memory and the reliable alternation of rodents in a T-maze in two consecutive trials largely depend on an intact hippocampus (Lalonde, 2002).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Hippocampal damage in mouse and human forms of systemic autoimmune disease

Ballok¹,

Woulfe

Sur

et al. 2004

Hippocampus

View full text Add to dashboard Cite

Systemic lupus erythematosus (SLE) is frequently accompanied by neuropsychiatric (NP) and cognitive deficits of unknown etiology. By using autoimmune MRL-lpr mice as an animal model of NP-SLE, we examine the relationship between autoimmunity, hippocampal damage, and behavioral dysfunction. Fluoro Jade B (FJB) staining and anti-ubiquitin (anti-Ub) immunocytochemistry were used to assess neuronal damage in young (asymptomatic) and aged (diseased) mice, while spontaneous alternation behavior (SAB) was used to estimate the severity of hippocampal dysfunction. The causal relationship between autoimmunity and neuropathology was tested by prolonged administration of the immunosuppressive drug cyclophosphamide (CY). In comparison to congenic MRL +/+ controls, SAB acquisition rates and performance in the "reversal" trial were impaired in diseased MRL-lpr mice, suggesting limited use of the spatial learning strategy. FJBpositive neurons and anti-Ub particles were frequent in the CA3 region. Conversely, CY treatment attenuated the SAB deficit and overall FJB staining. Similarly to mouse brain, the hippocampus from a patient who died from NP-SLE showed reduced neuronal density in the CA3 region and dentate gyrus, as well as increased FJB positivity in these regions. Gliosis and neuronal loss were observed in the gray matter, and T lymphocytes and stromal calcifications were common in the choroid plexus. Taken together, these results suggest that systemic autoimmunity induces significant hippocampal damage, which may underlie affective and cognitive deficits in NP-SLE.

show abstract