Switching Costs and the Gittins Index

Banks, Jeffrey S.; Sundaram, Rangarajan K.

doi:10.2307/2951664

Cited by 111 publications

(82 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ainslie 1975). Additionally, if there is a cost associated with switching from one behaviour to another, then not only is the Gittins index no longer optimal, but also there is no optimal index that may be calculated independently for each bandit (Banks & Sundaram 1994). It is well recognized that, under many conditions, humans exhibit costs when switching from one task to another (e.g.…”

Section: Optimal Performance In Stationary Environments: the Gittins mentioning

confidence: 99%

“…the pay-off structure of the environment must be stationary), all options must be available at all decision points (i.e. there cannot be any 'side paths') and agents must discount the value of rewards exponentially into the future (Gittins 1979;Berry & Fristedt 1985;Banks & Sundaram 1994). Real-world problems typically violate one or more of these assumptions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

Cohen

McClure

2007

Phil. Trans. R. Soc. B

951

846

View full text Add to dashboard Cite

Many large and small decisions we make in our daily lives-which ice cream to choose, what research projects to pursue, which partner to marry-require an exploration of alternatives before committing to and exploiting the benefits of a particular choice. Furthermore, many decisions require re-evaluation, and further exploration of alternatives, in the face of changing needs or circumstances. That is, often our decisions depend on a higher level choice: whether to exploit well known but possibly suboptimal alternatives or to explore risky but potentially more profitable ones. How adaptive agents choose between exploitation and exploration remains an important and open question that has received relatively limited attention in the behavioural and brain sciences. The choice could depend on a number of factors, including the familiarity of the environment, how quickly the environment is likely to change and the relative value of exploiting known sources of reward versus the cost of reducing uncertainty through exploration. There is no known generally optimal solution to the exploration versus exploitation problem, and a solution to the general case may indeed not be possible. However, there have been formal analyses of the optimal policy under constrained circumstances. There have also been specific suggestions of how humans and animals may respond to this problem under particular experimental conditions as well as proposals about the brain mechanisms involved. Here, we provide a brief review of this work, discuss how exploration and exploitation may be mediated in the brain and highlight some promising future directions for research.

show abstract

Section: Optimal Performance In Stationary Environments: the Gittins mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

Cohen

McClure

2007

Phil. Trans. R. Soc. B

951

846

View full text Add to dashboard Cite

show abstract

“…Mobility costs impact the wage that will be accepted in the new city, which in turn impacts future wage growth prospects in this new city; this difference in terms of option value will have an additional impact on indifference wages, and so forth. As shown in Equation 7, this dynamic feedback 24 See, for instance, the impossibility result of Banks & Sundaram (1994).…”

Section: Proposition 1 Optimal Strategiesmentioning

confidence: 99%

Job Search and Migration in a System of Cities

Schmutz

Sidibé

2015

SSRN Journal

View full text Add to dashboard Cite

We build an equilibrium job search model, where workers engage in both off-and on-the-job search over a set of cities, to quantify the impact of spatial matching frictions and mobility costs on the job search process.Migration decisions, based on a dynamic utility trade-off between locations, can rationalize diverse wage dynamics as part of forward-looking spatial strategies. Our estimation results allow us to characterize each of the largest 200 French cities by a set of city-specific matching and amenity parameters and to measure the impact of distance on spatial constraints. We find that after controlling for frictions, mobility cost parameters are significantly lower than previously reported in the literature. Additional results include a robust positive correlation between on-the-job arrival rates and local wage dispersion, which provides new empirical support to the wage-posting framework and suggests an alternative explanation for the city size wage gap.Keywords: local labor market; frictions; on-the-job search; migration JEL Classification: J2; J3; J6 * We thank Jim Albrecht, Pat Bayer, Bruno Decreuse, John Kennan, Thomas Le Barbanchon, Sébastien Roux and John Rust, as well as participants to many seminars and conferences, for helpful comments and discussions. This project was initiated when both authors were junior researchers at CREST and pursued when Schmutz was visiting Georgetown University and Sidibé was a post-doctoral fellow at Duke University. The generosity of these three institutions is gratefully acknowledged. Data was made available through the Centre d'Accès Sécurisé à Distance.

show abstract

“…First, Banks and Sundaram (1994) argued that any bandit problem where there are both costs of switching "away from" and "to" an arm is equivalent to another bandit problem where there is only the cost of switching to. Hence for simplicity, assume that there is only cost of switching to.…”

Section: Switching Costsmentioning

confidence: 99%

“…Banks and Sundaram (1994) showed that no index policy is optimal when switching between arms incurs costs. This survey will first discuss the properties of the bandit problem that make the Gittins index policy optimal, and show how these properties break down upon the introduction of costs on switching arms.…”

Section: Introductionmentioning

confidence: 99%

A survey on the bandit problem with switching costs

Jun

2004

De Economist

View full text Add to dashboard Cite

SummaryThe paper surveys the literature on the bandit problem, focusing on its recent development in the presence of switching costs. Switching costs between arms makes not only the Gittins index policy suboptimal, but also renders the search for the optimal policy computationally infeasible. This survey will first discuss the decomposability properties of the arms that make the Gittins index policy optimal, and show how these properties break down upon the introduction of costs on switching arms. Having established the failure of the simple index policy, the survey focus on the recent efforts to overcome the difficulty of finding the optimal policy in the bandit problem with switching costs: characterization of the optimal policy, exact derivation of the optimal policy in the restricted environments, and lastly approximation of optimal policy. The advantages and disadvantages of the above approaches are discussed.

show abstract

Switching Costs and the Gittins Index

Cited by 111 publications

References 6 publications

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

Job Search and Migration in a System of Cities

A survey on the bandit problem with switching costs

Contact Info

Product

Resources

About