IEDQN: Information Exchange DQN with a Centralized Coordinator for Traffic Signal Control

Xie, Donghan; Wang, Zhi; Chen, Chunlin; Dong, Dezun

doi:10.1109/ijcnn48605.2020.9206820

Cited by 14 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…POMDPs are useful for when the same state may be observed differently by agents depending on some randomness. Works in RL-based TSC that use a POMDP framework [29,34,35] either use observations to represent local intersection states in a road network (e.g., [34]), or to represent incomplete inputs from detectors (as is done by [35] for connected vehicle data).…”

Section: Partially Observable Mdpsmentioning

confidence: 99%

The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

Chen¹,

Fang²,

Sadeh³

2022

Preprint

View full text Add to dashboard Cite

Traffic signal control (TSC) is a high-stakes domain that is growing in importance as traffic volume grows globally. An increasing number of works are applying reinforcement learning (RL) to TSC; RL can draw on an abundance of traffic data to improve signalling efficiency. However, RL-based signal controllers have never been deployed. In this work, we provide the first review of challenges that must be addressed before RL can be deployed for TSC. We focus on four challenges involving (1) uncertainty in detection, (2) reliability of communications, (3) compliance and interpretability, and (4) heterogeneous road users. We show that the literature on RL-based TSC has made some progress towards addressing each challenge. However, more work should take a systems thinking approach that considers the impacts of other pipeline components on RL.

show abstract

Section: Partially Observable Mdpsmentioning

confidence: 99%

The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

Chen¹,

Fang²,

Sadeh³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In [76], Donghan et al embedded SADRL in a centralized agent, which is the centralized controller, to coordinate the traffic phases of distributed agents, which are traffic lights (A.2), based on traffic conditions in order to manage the congestion level. The agents maximize their individual rewards in a competitive (X.2.1) manner.…”

Section: Donghan's Sadrl Approach For a Hierarchical Multi-agent Environmentmentioning

confidence: 99%

“…Similarly to [33,36,76,79], in [83], Tian et al embedded MADRL in distributed agents, which are the traffic lights (A.2), to enhance their exploration strategy. A multi-agent environment is collaborative (X.2.2) in nature.…”

Section: Tian's Madrl Approach With Bootstrapping In a Multi-agent Environmentmentioning

confidence: 99%

“…Communication between agents faces the challenges of ultradensification (C.1) due to the large number of agents in the environment, high dynamicity (C.2) due to the rapid changes in the operating environment, and high dimensionality (C.3) due to the large number of state and action spaces characterizing the operating environment of the large number of agents. Communication can be reduced by enhancing the learning mechanism [76] that normally stores knowledge exchanged with neighboring agents in a memory device [91]. However, a memory device has a limited capacity which is suitable for a small number of agents only.…”

Section: Enhancing Communication Between Agents In Large Scale Madrlmentioning

confidence: 99%

See 1 more Smart Citation

Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms

et al. 2021

View full text Add to dashboard Cite

Recent advancements in deep reinforcement learning (DRL) have led to its application in multi-agent scenarios to solve complex real-world problems, such as network resource allocation and sharing, network routing, and traffic signal controls. Multi-agent DRL (MADRL) enables multiple agents to interact with each other and with their operating environment, and learn without the need for external critics (or teachers), thereby solving complex problems. Significant performance enhancements brought about by the use of MADRL have been reported in multi-agent domains; for instance, it has been shown to provide higher quality of service (QoS) in network resource allocation and sharing. This paper presents a survey of MADRL models that have been proposed for various kinds of multi-agent domains, in a taxonomic approach that highlights various aspects of MADRL models and applications, including objectives, characteristics, challenges, applications, and performance measures. Furthermore, we present open issues and future directions of MADRL.

show abstract

“…The number of hidden layers in a DNN can improve learning ability and task performance. Deep Reinforcement Learning (DRL) technique [5][6][7] is one such technique that has improved significantly with the implementation of DNN, which has been used in various applications such as autonomous voltage control for power grid operations [8,9], battery management system [10], network traffic signal control [11] and human-machine collaborations [12] One of the commonly used DRL is Deep Q-network (DQN), which approximates Q-value function using DNN [13,14]. In this paper, DQN is used to train the agent of the autonomous self-driving vehicle in two simulated environments.…”

Section: Introductionmentioning

confidence: 99%

Deep Q‐network implementation for simulated autonomous vehicle control

Quek

Koh

et al. 2021

IET Intelligent Trans Sys

View full text Add to dashboard Cite

Deep reinforcement learning is poised to be a revolutionised step towards newer possibilities in solving navigation and autonomous vehicle control tasks. Deep Q‐network (DQN) is one of the more popular methods of deep reinforcement learning that allows the agent that controls the vehicle to learn through its mistakes based on its actions and interactions with the environment. This paper presents the implementation of DQN to an autonomous self‐driving vehicle control in two different simulated environments; first environment is in Python which is a simple 2D environment and then advanced to Unity software separately which is a 3D environment. Based on the scores and pixel inputs, the agent in the vehicle learns and adapts to its surrounding. It develops the best solution strategy to direct itself in the environment where its task is to manoeuvre the vehicle from point to point on a simulated highway scenario. The implemented DQN technique approximates the action value function with convolutional neural network. This evaluates the Q‐function for the Q‐learning architecture and updates the action value function. This paper shows that DQN is an effective learning method for the agent of an autonomous vehicle. In both simulated environments, the autonomous vehicle gradually learnt the manoeuvre operations and progressively gained the ability to successfully navigate itself and avoid obstacles without prior information of the surrounding.

show abstract

IEDQN: Information Exchange DQN with a Centralized Coordinator for Traffic Signal Control

Cited by 14 publications

References 24 publications

The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

The Real Deal: A Review of Challenges and Opportunities in Moving Reinforcement Learning-Based Traffic Signal Control Systems Towards Reality

Applications of Multi-Agent Deep Reinforcement Learning: Models and Algorithms

Deep Q‐network implementation for simulated autonomous vehicle control

Contact Info

Product

Resources

About