Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Tottori, Takehiro; Kobayashi, Tetsuya J.

doi:10.3390/e25020208

Cited by 4 publications

(28 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Future work could include expanding our formalism to noisy dynamics, noisy control [25,[65][66][67][68], imperfect information or to study the robustness of the control, similar to [40,69,70]. There is also the intriguing possibility of allowing individuals to directly influence government [71] in the same way that ε allows the government to influence individuals.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Rational social distancing policy during epidemics with limited healthcare capacity

Schnyder,

Molina,

Yamamoto

et al. 2023

PLoS Comput Biol

View full text Add to dashboard Cite

Epidemics of infectious diseases posing a serious risk to human health have occurred throughout history. During recent epidemics there has been much debate about policy, including how and when to impose restrictions on behaviour. Policymakers must balance a complex spectrum of objectives, suggesting a need for quantitative tools. Whether health services might be ‘overwhelmed’ has emerged as a key consideration. Here we show how costly interventions, such as taxes or subsidies on behaviour, can be used to exactly align individuals’ decision making with government preferences even when these are not aligned. In order to achieve this, we develop a nested optimisation algorithm of both the government intervention strategy and the resulting equilibrium behaviour of individuals. We focus on a situation in which the capacity of the healthcare system to treat patients is limited and identify conditions under which the disease dynamics respect the capacity limit. We find an extremely sharp drop in peak infections at a critical maximum infection cost in the government’s objective function. This is in marked contrast to the gradual reduction of infections if individuals make decisions without government intervention. We find optimal interventions vary less strongly in time when interventions are costly to the government and that the critical cost of the policy switch depends on how costly interventions are.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Rational social distancing policy during epidemics with limited healthcare capacity

Schnyder,

Molina,

Yamamoto

et al. 2023

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…These effects can result in the existence of (multiple) stationary states [47,49]. This and other complexities could be included in our approach in principle, such as multiple agent types with different risk and behaviour profiles [19,[50][51][52][53], seasonal effects [54], stochastic control [13,[55][56][57][58], or decentralised optimal behaviour via interventions orchestrated by a benevolent social planner [27][28][29][30][31]. We reserve these developments for future studies.…”

Section: Introductionmentioning

confidence: 99%

Rational social distancing in epidemics with uncertain vaccination timing

et al. 2023

View full text Add to dashboard Cite

During epidemics people may reduce their social and economic activity to lower their risk of infection. Such social distancing strategies will depend on information about the course of the epidemic but also on when they expect the epidemic to end, for instance due to vaccination. Typically it is difficult to make optimal decisions, because the available information is incomplete and uncertain. Here, we show how optimal decision-making depends on information about vaccination timing in a differential game in which individual decision-making gives rise to Nash equilibria, and the arrival of the vaccine is described by a probability distribution. We predict stronger social distancing the earlier the vaccination is expected and also the more sharply peaked its probability distribution. In particular, equilibrium social distancing only meaningfully deviates from the no-vaccination equilibrium course if the vaccine is expected to arrive before the epidemic would have run its course. We demonstrate how the probability distribution of the vaccination time acts as a generalised form of discounting, with the special case of an exponential vaccination time distribution directly corresponding to regular exponential discounting.

show abstract

“…In order to address these issues, we propose an alternative theoretical framework to DSC, memory-limited DSC (ML-DSC), which is the decentralized version of memorylimited POSC (ML-POSC) [23,24]. The two major difficulties in DSC originate from the ignorance of constraints over controllers when we derive the optimal estimation and control solution.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, we address the optimization problem by converting ML-DSC in the state space into the deterministic optimal control problem in the probability density function space. This technique has recently been used in mean-field stochastic control [25,26] and ML-POSC [23,24], and is also effective for ML-DSC. Following that, we can solve ML-DSC in a similar way to the deterministic optimal control problem on the probability density function space; the optimal control function of ML-DSC was obtained by jointly solving the Hamilton-Jacobi-Bellman (HJB) equation and the Fokker-Planck (FP) equation.…”

Section: Introductionmentioning

confidence: 99%

“…Following that, we can solve ML-DSC in a similar way to the deterministic optimal control problem on the probability density function space; the optimal control function of ML-DSC was obtained by jointly solving the Hamilton-Jacobi-Bellman (HJB) equation and the Fokker-Planck (FP) equation. HJB-FP equations also appear in mean-field stochastic game and control [25][26][27][28][29] and ML-POSC [23,24], and numerous numerical algorithms have been proposed [24,[30][31][32]. Using these numerical algorithms, ML-DSC may be solved effectively, even in general problems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Decentralized Stochastic Control with Finite-Dimensional Memories: A Memory Limitation Approach

Tottori

Kobayashi

2023

Entropy

Self Cite

View full text Add to dashboard Cite

Decentralized stochastic control (DSC) is a stochastic optimal control problem consisting of multiple controllers. DSC assumes that each controller is unable to accurately observe the target system and the other controllers. This setup results in two difficulties in DSC; one is that each controller has to memorize the infinite-dimensional observation history, which is not practical, because the memory of the actual controllers is limited. The other is that the reduction of infinite-dimensional sequential Bayesian estimation to finite-dimensional Kalman filter is impossible in general DSC, even for linear-quadratic-Gaussian (LQG) problems. In order to address these issues, we propose an alternative theoretical framework to DSC—memory-limited DSC (ML-DSC). ML-DSC explicitly formulates the finite-dimensional memories of the controllers. Each controller is jointly optimized to compress the infinite-dimensional observation history into the prescribed finite-dimensional memory and to determine the control based on it. Therefore, ML-DSC can be a practical formulation for actual memory-limited controllers. We demonstrate how ML-DSC works in the LQG problem. The conventional DSC cannot be solved except in the special LQG problems where the information the controllers have is independent or partially nested. We show that ML-DSC can be solved in more general LQG problems where the interaction among the controllers is not restricted.

show abstract

Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control

Cited by 4 publications

References 58 publications

Rational social distancing policy during epidemics with limited healthcare capacity

Rational social distancing policy during epidemics with limited healthcare capacity

Rational social distancing in epidemics with uncertain vaccination timing

Decentralized Stochastic Control with Finite-Dimensional Memories: A Memory Limitation Approach

Contact Info

Product

Resources

About