Infinitesimal Perturbation Analysis for Quasi-Dynamic Traffic Light Controllers

Abstract

We consider the traffic light control problem for a single intersection modeled as a stochastic hybrid system. We study a quasi-dynamic policy based on partial state information defined by detecting whether vehicle backlogs are above or below certain controllable thresholds. Using Infinitesimal Perturbation Analysis (IPA), we derive online gradient estimators of a cost metric with respect to these threshold parameters and use these estimators to iteratively adjust the threshold values through a standard gradient-based algorithm so as to improve overall system performance under various traffic conditions. Results obtained by applying this methodology to a simulated urban setting are also included.

Infinitesimal perturbation analysis of stochastic hybrid systems: Application to congestion management in traffic-light intersections

“…One of the ways to characterize congestion is by the traffic buildup in front of a traffic light, and as in [13], [15], [10], we model its dynamics by a fluid queue. Such a queue is driven by two random processes: the arrival rate and the service rate.…”

“…Consider the integral terms in the RHS of (17). Since Γ is a green-signal period contained in a nonempty period of the queue, Equation (10) implies that, for every τ ∈ Γ, β(θ; τ ) = b(τ − kC − θ) for some k = 1, 2, . .…”

“…References [7], [22] provide detailed discussions of these points as well as unified frameworks for IPA in the setting of stochastic hybrid systems. Until recently the main areas mentioned for potential applications of IPA were in manufacturing and telecommunications, but lately there has been a growing interest in transportation networks as well, and especially in traffic-light control (see [10] for a survey).…”

“…The main objective of traffic-light control is to reduce, or minimize congestion at traffic intersections. Early techniques developed for these control and optimization problems include dynamic programming [20] and linear complementary algorithms [8], while more recent approaches are based on Markov-decision processes [21], game theory [3], and mixedinteger programming [9]; [19] contains an early survey, and a recent one can be found in [10]. Regarding applications of IPA, early results were presented in [12], [18], and a recent systematic approach has been developed in [13], [14], [15], [10].…”

“…Early techniques developed for these control and optimization problems include dynamic programming [20] and linear complementary algorithms [8], while more recent approaches are based on Markov-decision processes [21], game theory [3], and mixedinteger programming [9]; [19] contains an early survey, and a recent one can be found in [10]. Regarding applications of IPA, early results were presented in [12], [18], and a recent systematic approach has been developed in [13], [14], [15], [10]. This approach defines the traffic-light control problem in the aforementioned setting of stochastic hybrid systems [7], and develops for it effective IPA-based algorithms.…”