Infinite horizon optimal impulsive control with applications to Internet congestion control

Abstract

We investigate infinite horizon deterministic optimal control problems with both gradual and impulsive controls, where any finitely many impulses are allowed simultaneously. Both discounted and long run time average criteria are considered. We establish very general and at the same time natural conditions, under which the dynamic programming approach results in an optimal feedback policy. The established theoretical results are applied to the Internet congestion control, and by solving analytically and nontrivially the underlying optimal control problems, we obtain a simple threshold-based active queue management scheme, which takes into account the main parameters of the transmission control protocols, and improves the fairness among the connections in a given network.

Optimal impulse control of a SIR epidemic

“…Sometimes, such control is called “singular control.” The goal is to minimize the total (expected) accumulated cost, which may be discounted or not . The popular method of attack to such problems is dynamic programming . In other works, versions of the Pontryagin maximum principle are used.…”

“…Impulse control of various dynamical systems attracts the attention of many researchers, eg, those in other works 8,9,[11][12][13][14][15][16][17][18][19][20][21][22][23][24] to mention the most relevant and the most recent works. The underlying system can be described in terms of ordinary 8,9,[11][12][13]20,[22][23][24] or stochastic 17,21 differential equations. In other works, [14][15][16]19 along with the given deterministic drift, there are spontaneous (or natural) Markov jumps of the state.…”

“…Sometimes, such control is called "singular control." 21 The goal is to minimize the total (expected) accumulated cost, which may be discounted 12,[14][15][16]18,19,21 or not. 8,12,13,17,20,22 The popular method of attack to such problems is dynamic programming.…”

“…21 The goal is to minimize the total (expected) accumulated cost, which may be discounted 12,[14][15][16]18,19,21 or not. 8,12,13,17,20,22 The popular method of attack to such problems is dynamic programming. 12,[14][15][16]18,19,21,23 In other works, 11,20,22,24 versions of the Pontryagin maximum principle are used.…”

Whittle index policy for crawling ephemeral content

“…This has been a very successful heuristic for restless bandits, which, while suboptimal in general, is provably optimal in an asymptotic sense [26], [27] and has good empirical performance. It and its variants have been used extensively in logistical and engineering applications, some recent instances of the latter in communications and control being for sensor scheduling [19], multi-UAV coordination [20], congestion control [3], [4], [13], channel allocation in wireless networks [14], cognitive radio [17] and real-time wireless multicast [23]. Book length treatments of indexable restless bandits appear in [12], [24].…”