EMVLight: A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles

Abstract

Emergency vehicles (EMVs) play a crucial role in responding to time-critical events such as medical emergencies and fire outbreaks in an urban area. The less time EMVs spend traveling through the traffic, the more likely it would help save people's lives and reduce property loss. To reduce the travel time of EMVs, prior work has used route optimization based on historical traffic-flow data and traffic signal pre-emption based on the optimal route. However, traffic signal pre-emption dynamically changes the traffic flow which, in turn, modifies the optimal route of an EMV. In addition, traffic signal pre-emption practices usually lead to significant disturbances in traffic flow and subsequently increase the travel time for non-EMVs. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for simultaneous dynamic routing and traffic signal control. EMVLight extends Dijkstra's algorithm to efficiently update the optimal route for the EMVs in real-time as it travels through the traffic network. The decentralized RL agents learn network-level cooperative traffic signal phase strategies that not only reduce EMV travel time but also reduce the average travel time of non-EMVs in the network. This benefit has been demonstrated through comprehensive experiments with synthetic and real-world maps. These experiments show that EMVLight outperforms benchmark transportation engineering techniques and existing RL-based signal control methods.

State-of-the-Art Review on Traffic Control Strategies for Emergency Vehicles

“…To reduce the travel time of EVs while reducing the average travel time of regular vehicles, Ogunwolu et al optimize the optimal route of EVs by using the Dijkstra's algorithm and preempting the signals along the route with radio frequency signals [91]. Su et al propose a dynamic route optimization based on an improved the Dijkstra's algorithm and a signal coordination control hybrid strategy based on reinforced learning [92]. Min et al search for a reliable route with the shortest estimated arrival time under changing traffic conditions by path search and adopt elastic signal preemption to reduce the negative impact on the overall traffic flow due to prioritizing EVs [93].…”

“…[81]- [83], [85], [90], [91], [92], [94], [96] Response time + Road occupancy rate [24], [27]- [29] \ \ \…”

“…In terms of hybrid strategies, most studies are no longer limited to reducing the response time of EVs. Scholars consider more practical facts, such as reducing the negative impact on regular vehicles [91], [92], [94], [96], and choosing highly reliable rescue routes [97]. Min et al aim to find reliable arrival time estimates under changing traffic conditions and reduce the negative impact of EVs on overall traffic flow [93].…”

The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

“…For prioritization, [69] upweighted buses and emergency vehicles in their reward formulation based on throughput; [131] used a state representation based on the cell transmission traffic model and modelled priority as a binary variable; [132] adopted an implicit approach based on minimizing delay per person instead of per vehicle; [133] and [134] both considered prioritization for trams, with the former's rewards being based on tram schedule adherence and the latter using model predictive control to model driver behaviour; and [135] adaptively altered vehicle priorities depending on queue length, waiting time, and emergency vehicle presence. For preemption, [136] learned TSC policies for emergency vehicle routing with rewards that encourage low vehicle density, and [137] used RL to learn policies for notifying connected vehicles to clear out lanes for emergency vehicles to pass.…”

A traffic light control method based on multi-agent deep reinforcement learning algorithm

“…It is based on reinforcement learning algorithm and transportation theories. EMVLight algorithm 33 can perform dynamic routing and traffic light control simultaneously. MARDDPG algorithm 34 is proposed based on deep deterministic policy gradient algorithm.…”