Using Reinforcement Learning to Control Traffic Signals in a Real-World Scenario: An Approach Based on Linear Function Approximation

Alegre, Lucas N.; Ziemke, Theresa; Bazzan, Ana L. C.

doi:10.1109/tits.2021.3091014

Cited by 18 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although SUMO is popular among researchers and practitioners in the industry, another software called Multi-Agent Transport Simulation (MATSim) 7 is often used in academic research. While SUMO focuses on macroscopic traffic flow modeling, MATSim uses an agent-based approach to model individual travel behavior [145]. As a result, MATSim can capture more complex individual decision processes, while SUMO is better suited for overall traffic flow modeling.…”

Section: ) Mobility and Autonomous Vehiclesmentioning

confidence: 99%

Machine Learning With Computer Networks: Techniques, Datasets, and Models

Afifi,

Pochaba,

Boltres

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

Section: ) Mobility and Autonomous Vehiclesmentioning

confidence: 99%

Machine Learning With Computer Networks: Techniques, Datasets, and Models

Afifi,

Pochaba,

Boltres

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…To choose a vehicle phase to display green with a specific duration 18 [9,[11][12][13]15,21,[26][27][28][29][31][32][33][34]36,37,39,40] To choose the green time for current vehicle phase 4 [10,17,23,41] To determine whether or not to end current vehicle phase 8 [7,16,[18][19][20]24,25,38] To adjust the green time for all vehicle phases in next cycle 5 [8,14,22,30,35] Vehicle-specific performance measure used to construct rewards Number of already served vehicles 14 [12,13,17,18,[20][21][22][23]28,31,33,34,38,39] Wait time of already ...…”

Section: Action Taken By An Agentmentioning

confidence: 99%

Double Deep Q-Network with Dynamic Bootstrapping for Real-Time Isolated Signal Control: A Traffic Engineering Perspective

Zheng

Chen

et al. 2022

Applied Sciences

View full text Add to dashboard Cite

Real-time isolated signal control (RISC) at an intersection is of interest in the field of traffic engineering. Energizing RISC with reinforcement learning (RL) is feasible and necessary. Previous studies paid less attention to traffic engineering considerations and under-utilized traffic expertise to construct RL tasks. This study profiles the single-ring RISC problem from the perspective of traffic engineers, and improves a prevailing RL method for solving it. By qualitative applicability analysis, we choose double deep Q-network (DDQN) as the basic method. A single agent is deployed for an intersection. Reward is defined with vehicle departures to properly encourage and punish the agent’s behavior. The action is to determine the remaining green time for the current vehicle phase. State is represented in a grid-based mode. To update action values in time-varying environments, we present a temporal-difference algorithm TD(Dyn) to perform dynamic bootstrapping with the variable interval between actions selected. To accelerate training, we propose a data augmentation based on intersection symmetry. Our improved DDQN, termed D3ynQN, is subject to the signal timing constraints in engineering. The experiments at a close-to-reality intersection indicate that, by means of D3ynQN and non-delay-based reward, the agent acquires useful knowledge to significantly outperform a fully-actuated control technique in reducing average vehicle delay.

show abstract

“…Boukerche et al [13], to address the problem that existing methods ignore the impact of transmission delay on the system exchanging traffic flow information, proposed a traffic state detection method, and proved to solve the data transmission delay problem by an experimental comparison. Alegre et al [14] proposed the TOS(λ)-FB algorithm and proved its efficiency by combining the Fourier basis function and the reinforcement learning SARSA(λ) algorithm in order to solve the dimensional explosion problem due to the large state space. Wang et al [15] investigated multi-intelligent reinforcement learning for large-scale traffic signal optimization control problems.…”

Section: Introductionmentioning

confidence: 99%

Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning

Cao,

Wang,

Zhang

et al. 2024

Electronics

View full text Add to dashboard Cite

The optimization and control of traffic signals is very important for logistics transportation. It not only improves the operational efficiency and safety of road traffic, but also conforms to the direction of the intelligent, green, and sustainable development of modern cities. In order to improve the optimization effect of traffic signal control, this paper proposes a traffic signal optimization method based on deep reinforcement learning and Simulation of Urban Mobility (SUMO) software for urban traffic scenarios. The intersection training scenario was established using SUMO micro traffic simulation software, and the maximum vehicle queue length and vehicle queue time were selected as performance evaluation indicators. In order to be more relevant to the real environment, the experiment uses Weibull distribution to simulate vehicle generation. Since deep reinforcement learning takes into account both perceptual and decision-making capabilities, this study proposes a traffic signal optimization control model based on the deep reinforcement learning Deep Q Network (DQN) algorithm by considering the realism and complexity of traffic intersections, and first uses the DQN algorithm to train the model in a training scenario. After that, the G-DQN (Grouping-DQN) algorithm is proposed to address the problems that the definition of states in existing studies cannot accurately represent the traffic states and the slow convergence of neural networks. Finally, the performance of the G-DQN algorithm model was compared with the original DQN algorithm model and Advantage Actor-Critic (A2C) algorithm model. The experimental results show that the improved algorithm increased the main indicators in all aspects.

show abstract

Using Reinforcement Learning to Control Traffic Signals in a Real-World Scenario: An Approach Based on Linear Function Approximation

Cited by 18 publications

References 28 publications

Machine Learning With Computer Networks: Techniques, Datasets, and Models

Machine Learning With Computer Networks: Techniques, Datasets, and Models

Double Deep Q-Network with Dynamic Bootstrapping for Real-Time Isolated Signal Control: A Traffic Engineering Perspective

Optimization Control of Adaptive Traffic Signal with Deep Reinforcement Learning

Contact Info

Product

Resources

About