A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network

Dong, Jiqian; Chen, Sikai; Ha, Paul Young Joun; Li, Yujie; Labi, Samuel

doi:10.48550/arxiv.2010.05437

Cited by 12 publications

(28 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to power allocation [9]- [12], GNNs have been used to address cellular [25] and satellite [26] traffic prediction, link scheduling [27], channel control [28], and localization [29]. Due to their localized nature, GNNs have also been applied to cooperative [30] and decentralized [31] control problems in networked systems. A review of the use of GNNs in wireless communication can be found in [32].…”

Section: Prior Workmentioning

confidence: 99%

“…During meta-testing, we consider the obtained module set M as fixed. Using the training portion of the meta-test data set D tr τtest , we only optimize the parameters of the distribution Pτtest ( Sτtest |M, D tr τtest ) using (30), or, more practically, multiple gradient descent steps. The final REGNN is constructed by using the mode of the assignment distribution as…”

Section: Optimization During Runtimementioning

confidence: 99%

“…Update module parameters M using data set D te τ using (31) return module set M Update the distribution parameters η τtest,l (M) using data set D tr τtest according to (30) 12:…”

Section: Permutation Equivariance and Invariancementioning

confidence: 99%

See 2 more Smart Citations

Modular Meta-Learning for Power Control via Random Edge Graph Neural Networks

Nikoloska¹,

Simeone²

2021

Preprint

View full text Add to dashboard Cite

In this paper, we consider the problem of power control for a wireless network with an arbitrarily time-varying topology, including the possible addition or removal of nodes. A data-driven design methodology that leverages graph neural networks (GNNs) is adopted in order to efficiently parametrize the power control policy mapping the channel state information (CSI) to transmit powers. The specific GNN architecture, known as random edge GNN (REGNN), defines a non-linear graph convolutional filter whose spatial weights are tied to the channel coefficients. While prior work assumed a joint training approach whereby the REGNN-based policy is shared across all topologies, this paper targets adaptation of the power control policy based on limited CSI data regarding the current topology. To this end, we propose both black-box and modular meta-learning techniques. Black-box meta-learning optimizes a general-purpose adaptation procedure via (stochastic) gradient descent, while modular meta-learning finds a set of reusable modules that can form components of a solution for any new network topology.Numerical results validate the benefits of meta-learning for power control problems over joint training schemes, and demonstrate the advantages of modular meta-learning when data availability is extremely limited.

show abstract

Section: Prior Workmentioning

confidence: 99%

Section: Optimization During Runtimementioning

confidence: 99%

See 1 more Smart Citation

Modular Meta-Learning for Power Control via Random Edge Graph Neural Networks

Nikoloska¹,

Simeone²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Experiment is carried out on the "MAgent" platform, and results show that the proposed methods substantially outperform existing methods in a variety of cooperative scenarios. In [12], GNN and DQN are combined (named GCQ) for multiagent cooperative controlling of CAVs in merging scenarios. Specifically, GNN is proposed to aggregate the information acquired from collaborative sensing, while cooperative lanechanging decisions are generated from DQN.…”

Section: Introductionmentioning

confidence: 99%

“…This paper proposes an innovative modular framework to analyze the performance of different GRL methods in interactive traffic scenarios.The traffic scenarios adopted in our work is constructed based on [12]. The designed algorithm in this paper is based on GRL, which consists of two modules: GNN and DRL.…”

Section: Introductionmentioning

confidence: 99%

Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments

Liu¹,

Li²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

An efficient and reliable multi-agent decisionmaking system is highly demanded for the safe and efficient operation of connected autonomous vehicles in intelligent transportation systems. Current researches mainly focus on the Deep Reinforcement Learning (DRL) methods. However, utilizing DRL methods in interactive traffic scenarios is hard to represent the mutual effects between different vehicles and model the dynamic traffic environments due to the lack of interactive information in the representation of the environments, which results in low accuracy of cooperative decisions generation. To tackle these difficulties, this research proposes a framework to enable different Graph Reinforcement Learning (GRL) methods for decision-making, and compares their performance in interactive driving scenarios. GRL methods combinate the Graph Neural Network (GNN) and DRL to achieve the better decisions generation in interactive scenarios of autonomous vehicles, where the features of interactive scenarios are extracted by the GNN, and cooperative behaviors are generated by DRL framework. Several GRL approaches are summarized and implemented in the proposed framework. To evaluate the performance of the proposed GRL methods, an interactive driving scenarios on highway with two ramps is constructed, and simulated experiment in the SUMO platform is carried out to evaluate the performance of different GRL approaches. Finally, results are analyzed in multiple perspectives and dimensions to compare the characteristic of different GRL approaches in intelligent transportation scenarios. Results show that the implementation of GNN can well represents the interaction between vehicles, and the combination of GNN and DRL is able to improve the performance of the generation of lanechange behaviors. The source code of our work can be found at https://github.com/Jacklinkk/TorchGRL.

show abstract

Hardware‐in‐the‐loop validation of an adaptive model predictive control on a connected and automated vehicle

Landolfi

Salvi²,

Troiano³

et al. 2023

Adaptive Control & Signal

View full text Add to dashboard Cite

Summary Connected and automated vehicles will characterize the future of the mobility, featured by Smart Roads and Internet of Things technologies. Vehicles will behave as mobile nodes of a network enabling communication between them and with respect to the road infrastructure. In order to provide advanced driver assistance and automated driving along a Smart Road, control strategies need to take into account both parametric variations due to environmental and/or weather conditions (e.g., wind speed, presence of ice, and/or rain), and constraints related to road safety and vehicular traffic. For this reason, advanced adaptive control strategies result particularly suitable to provide a complete control of the vehicle. This article proposes an extension to a previous conference paper, concerning a Processor in the Loop testing of an Adaptive Model Predictive Control for coupled longitudinal/lateral vehicle dynamics, where the ego vehicle was simulated through a bicycle model. In the present work, a low‐cost real‐time Hardware In the Loop test bench, featured by Raspberry boards, is used to verify the control strategy in a way closer to a real application. A four‐wheel vehicle dynamic model is considered to simulate the ego vehicle, then robustness and sensitivity tests with respect to parametric variations related to road friction, wind speed and vehicle mass, and a comparison with an existing adaptive MPC are performed. The achieved results show good robustness properties, safety performances and confirm the real‐time execution of the control strategy, paving the way for future developments, where the goal will be the testing in a real road scenario by using an IoT gateway purposefully designed by the NetCom company.

show abstract

A DRL-based Multiagent Cooperative Control Framework for CAV Networks: a Graphic Convolution Q Network

Cited by 12 publications

References 13 publications

Modular Meta-Learning for Power Control via Random Edge Graph Neural Networks

Modular Meta-Learning for Power Control via Random Edge Graph Neural Networks

Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments

Hardware‐in‐the‐loop validation of an adaptive model predictive control on a connected and automated vehicle

Contact Info

Product

Resources

About