Mixed-Autonomy Traffic Control with Proximal Policy Optimization

Wei, Haoran; Liu, Xuanzhang; Mashayekhy, Lena; Decker, Keith

doi:10.1109/vnc48660.2019.9062809

Cited by 15 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Trainable policies for villagers.. In our work, we chose Proximal Policy Optimization (PPO) as a training algorithm for its widespread success in multiagent environment [19,53], but future research could also focus on other algorithms such as TRPO [39] or MADDPG [31]. PPO uses a surrogate loss function to keep the difference between the old and the new policy within a safe range 9 .…”

Section: Policiesmentioning

confidence: 99%

RLupus: Cooperation through emergent communication in The Werewolf social deduction game

Brandizzi

Grossi

Iocchi

2022

View full text Add to dashboard Cite

This paper focuses on the emergence of communication to support cooperation in environments modeled as social deduction games (SDG), that are games where players communicate freely to deduce each others’ hidden intentions. We first state the problem by giving a general formalization of SDG and a possible solution framework based on reinforcement learning. Next, we focus on a specific SDG, known as The Werewolf, and study if and how various forms of communication influence the outcome of the game. Experimental results show that introducing a communication signal greatly increases the winning chances of a class of players. We also study the effect of the signal’s length and range on the overall performance showing a non-linear relationship.

show abstract

Section: Policiesmentioning

confidence: 99%

RLupus: Cooperation through emergent communication in The Werewolf social deduction game

Brandizzi

Grossi

Iocchi

2022

View full text Add to dashboard Cite

show abstract

“…On the other hand, CAV has the potential to improve traffic efficiency but cannot guarantee a higher average speed than traditional vehicles depending on the network type and traffic conditions. [26] conducted experiments in a ring scenario that shows more than 20% CAV allow all vehicles to reach a higher speed and stabilize the flow. The penetration rate in 20% to 40% is possible to result in the near-maximum improvements [25].…”

Section: Efficiency and Safety For Mixed-autonomy Trafficmentioning

confidence: 99%

“…The penetration rate in 20% to 40% is possible to result in the near-maximum improvements [25]. Overall, a high penetration rate of CAV can bring traffic efficiency and safety improvement on mix-autonomy in various scenarios [25], [26].…”

Section: Efficiency and Safety For Mixed-autonomy Trafficmentioning

confidence: 99%

CoTV: Cooperative Control for Traffic Light Signals and Connected Autonomous Vehicles using Deep Reinforcement Learning

Guo¹,

Cheng²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

The target of reducing travel time only is insufficient to support the development of future smart transportation systems. To align with the United Nations Sustainable Development Goals (UN-SDG), a further reduction of fuel and emissions, improvements of traffic safety, and the ease of infrastructure deployment and maintenance should also be considered. Different from existing work focusing on the optimization of the control in either traffic light signal (to improve the intersection throughput), or vehicle speed (to stabilize the traffic), this paper presents a multi-agent deep reinforcement learning (DRL) system called CoTV, which Cooperatively controls both Traffic light signals and connected autonomous Vehicles (CAV). Therefore, our CoTV can well balance the achievement of the reduction of travel time, fuel, and emissions. In the meantime, CoTV can also be easy to deploy by cooperating with only one CAV that is the nearest to the traffic light controller on each incoming road. This enables more efficient coordination between traffic light controllers and CAV, thus leading to the convergence of training CoTV under the large-scale multi-agent scenario that is traditionally difficult to converge. We give the detailed system design of CoTV and demonstrate its effectiveness in a simulation study using SUMO under various grid maps and realistic urban scenarios with mixed-autonomy traffic.

show abstract

“…So far, several works have been demonstrated that use RL to reduce traffic congestion. Some of these projects works are centralized, 8,12,13 while others are decentralized. [14][15][16] The validation of the proposed algorithms in References 12 and 14 has been conducted in simplified hypothetical intersection scenarios.…”

Section: Introductionmentioning

confidence: 99%

Deep reinforcement learning based cooperative control of traffic signal for multi‐intersection network in intelligent transportation system using edge computing

Paul

2022

Trans Emerging Tel Tech

View full text Add to dashboard Cite

In the current era, the coordination of traffic flow is hindered by the discrepancy between road infrastructure and the number of vehicles which leads to traffic congestion. One of the widely used strategies to mitigate traffic congestion is to control traffic signals with the help of deep reinforcement learning (DRL) in edge computing based intelligent transportation system. This article provides a comprehensive analysis of the most recent DRL algorithms, advantage actor‐critic and proximal policy optimization in multiple deep neural networks (DNNs), including a state‐of‐the‐art transformer model for effective traffic signal management. Here, a single DRL agent is used, which obtains the spatio‐temporal information of the traffic to identify traffic patterns from complex intersection environments. The agent uses this information as the input to the DNNs and then applies the algorithms to retrieve the essential parameters of DNN to seek an optimal action selection policy to mitigate congestion. Different real‐time maps and small city networks are explored here to determine which DNN is best suited for traffic congestion management. The simulation study reveals that both the algorithms significantly outperform the baseline. The transformer model gives the best result when compared to other DNNs. The transformer model decreases average waiting time by 96.16%, implying that it has a higher capability of dealing with congested environments.

show abstract

Mixed-Autonomy Traffic Control with Proximal Policy Optimization

Cited by 15 publications

References 16 publications

RLupus: Cooperation through emergent communication in The Werewolf social deduction game

RLupus: Cooperation through emergent communication in The Werewolf social deduction game

CoTV: Cooperative Control for Traffic Light Signals and Connected Autonomous Vehicles using Deep Reinforcement Learning

Deep reinforcement learning based cooperative control of traffic signal for multi‐intersection network in intelligent transportation system using edge computing

Contact Info

Product

Resources

About