Fast convergence to Wardrop equilibria by adaptive sampling methods

Abstract

We study rerouting policies in a dynamic round-based variant of a well known game theoretic traffic model due to Wardrop. Previous analyses (mostly in the context of selfish routing) based on Wardrop's model focus mostly on the static analysis of equilibria. In this paper, we ask the question whether the population of agents responsible for routing the traffic can jointly compute or better learn a Wardrop equilibrium efficiently. The rerouting policies that we study are of the following kind. In each round, each agent samples an alternative routing path and compares the latency on this path with its current latency. If the agent observes that it can improve its latency then it switches with some probability depending on the possible improvement to the better path.We can show various positive results based on a rerouting policy using an adaptive sampling rule that implicitly amplifies paths that carry a large amount of traffic in the Wardrop equilibrium. For general asymmetric games, we show that a simple replication protocol in which agents adopt strategies of more successful agents reaches a certain kind of bicriteria equilibrium within a time bound that is independent of the size and the structure of the network but only depends on a parameter of the latency functions, that we call the relative slope. For symmetric games, this result has an intuitive interpretation: Replication approximately satisfies almost everyone very quickly.In order to achieve convergence to a Wardrop equilibrium * besides replication one also needs an exploration component discovering possibly unused strategies. We present a sampling based replication-exploration protocol and analyze its convergence time for symmetric games. For example, if the latency functions are defined by positive polynomials in coefficient representation, the convergence time is polynomial in the representation length of the latency functions. To the best of our knowledge, all previous results on the speed of convergence towards Wardrop equilibria, even when restricted to linear latency functions, were pseudopolynomial. In addition to the upper bounds on the speed of convergence, we can also present a lower bound demonstrating the necessity of adaptive sampling by showing that static sampling methods result in a slowdown that is exponential in the size of the network. A further lower bound illustrates that the relative slope is, in fact, the relevant parameter that determines the speed of convergence.

Minimum-Delay Load-Balancing through Non-parametric Regression

“…It is then necessary to specify an algorithm that when independently ran in every router, the equilibrium is achieved as fast as possible and without oscillations. In [9] the authors present such mechanism and use it to design a load-balancing scheme in [3]. Below, we can see the algorithm that each commodity executes in turn (where p si is the portion of demand d s routed through path P si ).…”

“…The algorithm converges to the WE as long as λ ≤ k/r, where k > 0 is a suitable constant and r is an upper-bound to the relative slope of all φ l (ρ l ), which is defined as follows [9]: Intuitively, migration from one path to the other should be slow if the cost function has abrupt changes. On the other hand, if the cost function is relatively "soft", changes may be faster.…”

The Stability-Featured Dynamic Multi-Path Routing

“…The exploration-replication (REPLEX) policy based traffic engineering protocol [2] also focuses on the hop-byhop routing considering the path cost as the maximum link utilization along each path. However, the theory behind the REPLEX protocol can only guarantee the fast convergence property under the additive path cost [7]- [9]. Compared with the existing studies, TeXCP adopts sophisticated measures to ensure the stability of the network.…”

Exploiting Myopic Learning

“…2 See, for example, Bomze (1986), Fischer and Vocking (2004), Fischer, Racke, and Vocking (2006), and the survey form of learning dynamic that nicely straddles the line between behavioral and rational models.…”