Robust decomposable Markov decision processes motivated by allocating school budgets

Dimitrov, Nedialko B.; Dimitrov, Stanko; Chukova, Stefanka

doi:10.1016/j.ejor.2014.05.003

Cited by 9 publications

(7 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We begin this section by reporting the computational time required to compute all the robust indices of a single arm using the restart problem (11). The results are shown in Table 1.…”

Section: Computational Resultsmentioning

confidence: 99%

Robust control of the multi-armed bandit problem

Caro

Gupta

2015

Ann Oper Res

View full text Add to dashboard Cite

We study a robust model of the multi-armed bandit (MAB) problem in which the transition probabilities are ambiguous and belong to subsets of the probability simplex. We first show that for each arm there exists a robust counterpart of the Gittins index that is the solution to a robust optimal stopping-time problem and can be computed effectively with an equivalent restart problem. We then characterize the optimal policy of the robust MAB as a project-by-project retirement policy but we show that arms become dependent so the policy based on the robust Gittins index is not optimal. For a project selection problem, we show that the robust Gittins index policy is near optimal but its implementation requires more computational effort than solving a non-robust MAB problem. Hence, we propose a Lagrangian index policy that requires the same computational effort as evaluating the indices of a non-robust MAB and is within 1% of the optimum in the robust project selection problem.

show abstract

“…We begin this section by reporting the computational time required to compute all the robust indices of a single arm using the restart problem (11). The results are shown in Table 1.…”

Section: Computational Resultsmentioning

confidence: 99%

Robust control of the multi-armed bandit problem

Caro

Gupta

2015

Ann Oper Res

View full text Add to dashboard Cite

show abstract

“…This is equivalent to each arm being played by the controller and an independent copy of nature. We note that this approach resembles the approximate method in [11]. More broadly, this Lagrangian technique has shown to be effective in solving weakly coupled MDPs (see for instance [14] and the references therein).…”

Section: Lagrangian Index Policymentioning

confidence: 99%

“…There is a dearth of approximate methods, even for particular applications, to the point that Iyengar [15] concludes his paper calling for more research in this area. One of the few applications we are aware of is Dimitrov et al [11] that study a robust MDP formulation for school budget allocation. Their approximate method is based on a Lagrangian decomposition that is similar to ours.…”

Section: Introductionmentioning

confidence: 99%

Robust Control of the Multi-Armed Bandit Problem

Caro

Gupta

2013

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…As such, the probabilities are unknown and so a transition set (of possible values) is used instead. The resulting MDP problem is non-linear and non-convex, but a simplification leads to a tractable problem (Dimitrov et al 2014). This type of model has the advantage that various possible objectives of the funding bodies, such as equity in distribution of educational proficiency across schools (BenDavid-Hadar and Ziderman 2011), could be incorporated.…”

Section: Resource Allocationmentioning

confidence: 99%

“…For example, viewing the resource allocation from school districts to individual schools as a Markov Decision Process (MDP) (Howard 2002) can lead to allocations which are more aligned to other aspirations of the educational system (such as the 'No Child Left Behind' policy) than allocation methods currently in use (Dimitrov et al 2014). In this application, a school district comprises a set of schools each of which has a proficiency level (measured by student performance on standardized tests) and the school district aims to maximize aggregate proficiency across schools using its funding allocation model.…”

Section: Resource Allocationmentioning

confidence: 99%

Operational Research in education

Johnes

2015

European Journal of Operational Research

120

View full text Add to dashboard Cite

Operational Research (OR) techniques have been applied, from the early stages of the discipline, to a wide variety of issues in education. At the government level, these include questions of what resources should be allocated to education as a whole and how these should be divided amongst the individual sectors of education and the institutions within the sectors. Another pertinent issue concerns the efficient operation of institutions, how to measure it, and whether resource allocation can be used to incentivise efficiency savings. Local governments, as well as being concerned with issues of resource allocation, may also need to make decisions regarding, for example, the creation and location of new institutions or closure of existing ones, as well as the day-to-day logistics of getting pupils to schools. Issues of concern for managers within schools and colleges include allocating the budgets, scheduling lessons and the assignment of students to courses. This survey provides an overview of the diverse problems faced by government, managers and consumers of education, and the OR techniques which have typically been applied in an effort to improve operations and provide solutions.

show abstract

Robust decomposable Markov decision processes motivated by allocating school budgets

Cited by 9 publications

References 20 publications

Robust control of the multi-armed bandit problem

Robust control of the multi-armed bandit problem

Robust Control of the Multi-Armed Bandit Problem

Operational Research in education

Contact Info

Product

Resources

About