Suboptimal choice, reward-predictive signals, and temporal information.

Abstract

Suboptimal choice refers to preference for an alternative offering a low probability of food (suboptimal alternative) over an alternative offering a higher probability of food (optimal alternative). Numerous studies have found that stimuli signaling probabilistic food play a critical role in the development and maintenance of suboptimal choice. However, there is still much debate about how to characterize how these stimuli influence suboptimal choice. There is substantial evidence that the temporal information conveyed by a food-predictive signal governs its function as both a Pavlovian conditioned stimulus and as an instrumental conditioned reinforcer. Thus, we explore the possibility that food-predictive signals influence suboptimal choice via the temporal information they convey. Application of this temporal information-theoretic approach to suboptimal choice provides a formal, quantitative framework that describes how food-predictive signals influence suboptimal choice in a manner consistent with related phenomena in Pavlovian conditioning and conditioned reinforcement. Our reanalysis of previous data on suboptimal choice suggests that, generally speaking, preference in the suboptimal choice procedure tracks relative temporal information conveyed by food-predictive signals for the suboptimal and optimal alternatives. The model suggests that suboptimal choice develops when the food-predictive signal for the suboptimal alternative conveys more temporal information than that for the optimal alternative. Finally, incorporating a role for competition between temporal information provided by food-predictive signals and relative primary reinforcement rate provides a reasonable account of existing data on suboptimal choice. (PsycINFO Database Record

Suboptimal choice and initial‐link requirement

“…corroborates past findings using VI schedules with signaled optimal alternatives Kendall, 1974Kendall, , 1985 as well as the basic ordinal predictions of the SiGN model (McDevitt et al, 2016), the information theoretic approach (Cunningham & Shahan, 2018), and the HVA model (Mazur, 2001). Future research would do well to test a wider range of schedule values so that a more explicit quantitative assessment of these models' suboptimal choice predictions can occur.…”

“…A different, yet conceptually similar, explanation to the SiGN model is found in the information-theoretic approach (Cunningham & Shahan, 2018). Under this framework, a terminal-link stimulus will function as a conditioned reinforcer if it conveys temporal information that reduces uncertainty about when the next primary reinforcer is likely to occur.…”

“…However, this only applies to instances where the terminal links on the optimal alternative are unsignaled (i.e., nonpredictive), which is not the case in either the Kendall (1985) or Dunn and Spetch (1990) studies, or even in more recent work by Case and Zentall (2018). Under such conditions, the information-theoretic approach, by itself, predicts only indifference (irrespective of initial-link length) and requires the inclusion of a matching relation that competes with temporal information-via a specific weighting function that formalizes the competition between conditioned and primary reinforcement (see Cunningham & Shahan, 2018)-to make predictions consistent with obtained findings. By contrast, the HVA model's principle of value addition makes ordinal predictions similar to SiGN without a strict need to appeal to an additional matching function on primary reinforcement.…”

The Δ–∑ hypothesis: How contrast and reinforcement rate combine to generate suboptimal choice

“…Cunningham and Shahan () also took an informational approach. However, instead of information about whether food will be delivered, their model stresses information about when food will be delivered.…”

Number and time in acquisition, extinction and recovery

“…Subjects in associative learning experiments make a temporal map of the conditioning experience (Balsam & Gallistel, 2009;Honig, 1981;Taylora, Joseph, Zhaoc, & Balsam, 2014). The temporal information in the map affects every aspect of Pavlovian and operantly conditioned behavior (Arcediano & Miller, 2002;Barnet, Grahame, & Miller, 1993;Barnet & Miller, 1996;Blaisdell, Denniston, & Miller, 1998;Burger, Denniston, & Miller, 2001;Cole, Barnet, & Miller, 1995;Cunningham & Shahan, 2018;Denniston, Blaisdell, & Miller, 1998, 2004Gallistel, Craig, & Shahan, 2019;Gür, Duyan, & Balci, 2017;Shahan & Cunningham, 2015;Theunissen & Miller, 1995;Thrailkill & Shahan, 2014).…”