Multi-Kernel Online Reinforcement Learning for Path Tracking Control of Intelligent Vehicles

Liu, Jiahang; Huang, Zhenhua; Xu, Xin; Zhang, Xinglong; Sun, Shiliang; Li, Dazi

doi:10.1109/tsmc.2020.2966631

Cited by 31 publications

(15 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Driven by rapid developments of emerging technologies such as artificial intelligence, extended reality, and blockchain, metaverse is becoming an attainable reality. As a promising artificial intelligence technique, deep reinforcement learning (DRL) recently achieves remarkable success in both video games of virtuality [2], [3] and many real-world scenes, such as robotic manipulation [4], [5], mobile robot control [6], [7], [8], [9], [10], [11], [12], [13], and manufacturing process [14], [15], which makes it ideally suited for the realization of metaverse intelligence. Multiagent DRL (MARL) is a multiagent extension of DRL that concentrates on the relation and interaction of multiple agents in mixed cooperativecompetitive environments [16].…”

Section: Introductionmentioning

confidence: 99%

MARL Sim2real Transfer: Merging Physical Reality With Digital Virtuality in Metaverse

Shi

Liu

Zhang

et al. 2023

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

Metaverse is an artificial virtual world mapped from and interacting with the real world. In metaverse, digital entities coexist with their physical counterparts. Powered by deep learning, metaverse is inevitably becoming more intelligent in the interactions between reality and virtuality. However, it is confronted with a nontrivial problem known as sim2real transfer when deep learning techniques try to bridge the reality gap between the physical world and simulations. In this article, we use multiagent deep reinforcement learning (MARL) to implement collective intelligence for digital entities as well as their physical counterparts. To model the immersive environments in metaverse, we define a nonstationary variant of Markov games and propose a recurrent MARL solution to it. Based on the solution, MARL sim2real transfer that bridges real and virtual multiple unmanned aerial vehicle (multi-UAV) systems is successfully conducted by employing recurrent multiagent deep deterministic policy gradient (R-MADDPG) with the domain randomization technique. Additionally, we use perception-control modularization to improve the generalization performance of MARL policies and make training more efficient.

show abstract

Section: Introductionmentioning

confidence: 99%

MARL Sim2real Transfer: Merging Physical Reality With Digital Virtuality in Metaverse

Shi

Liu

Zhang

et al. 2023

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

show abstract

“…In autonomous driving, prescriptive radars calculate the specific feedback scheme based on the optimal strategy obtained through computational experiments [62], [63], [64], [65], [66]. For example, prescriptive radars can adjust important parameters in radar systems such as scanning frequency and distribution of laser emitters to focus on the key area during operation [67].…”

Section: Prescriptive Radarsmentioning

confidence: 99%

RadarVerses in Metaverses: A CPSI-Based Architecture for 6S Radar Systems in CPSS

Liu

Shen

Tian

et al. 2023

IEEE Trans. Syst. Man Cybern, Syst.

View full text Add to dashboard Cite

Metaverses have caused significant changes in the industry and their academic foundation can be traced back to the term cyber-physical-social systems (CPSS), which was proposed in 2010. Radar is an important sensor in sensing systems that are widely applied in many fields, especially, in autonomous driving. To deal with the complex environment, smart radars with realtime information processing capabilities are required. Human factors play a critical role in the operation and management of radar systems, thus, digital twins' radars in cyber-physical systems (CPS) are unable to achieve intelligence in CPSS due to an incomplete consideration of human involvement. For this consideration, we propose a novel framework of RadarVerses for smart radars in metaverses based on ACP-based parallel intelligence, which is also known as cyber-physical-social intelligence (CPSI). RadarVerses consist of five main parts which are physical radars, descriptive radars, predictive radars, prescriptive radars, and deep radars. To construct RadarVerses at the technical level, we introduce four main technical foundations: 1) communication technology; 2) scenarios engineering; 3) foundation models; and 4) digital workers. In addition, we also provide a case study about LiDARs' predictive maintenance of accumulated snow in RadarVerses.

show abstract

“…Several techniques are present in the literature for the purpose of path tracking [18]. These include MPC-based methods [19], learning-based methods [20] and other control theory methods such as sliding mode control (SMC) [8]. SMC is a nonlinear control technique that drives the target system to a designed surface in the state space, then keeps the system in a close neighborhood of this surface in a sliding (switching) manner.…”

Section: A Local Steering Control: Path Followingmentioning

confidence: 99%

Proactive And Smooth Maneuvering For Navigation Around Pedestrians

Kabtoul

Spalanzani

Martinet

2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Navigation in close proximity with pedestrians is a challenge on the way to fully automated vehicles. Pedestrianfriendly navigation requires an understanding of pedestrian reaction and intention. Merely safety based reactive systems can lead to sub-optimal navigation solutions resulting in the freezing of the vehicle in many scenarios. Moreover, a strictly reactive method can produce unnatural driving patterns which cannot guarantee the legibility or social acceptance of the automated vehicle. This work presents a proactive maneuvering method adapted to navigation in close interaction with pedestrians using a dynamic channel approach. The method allows to proactively explore the navigation options based on anticipating pedestrians cooperation. The navigation is tested in frontal and lateral crossing scenarios with variable space density. The system is implemented under ROS, and compared with the probabilistic Risk-RRT planning method. The results are evaluated based on the safety and comfort of the pedestrians, and the quality of the vehicle's trajectory.

show abstract

Multi-Kernel Online Reinforcement Learning for Path Tracking Control of Intelligent Vehicles

Cited by 31 publications

References 43 publications

MARL Sim2real Transfer: Merging Physical Reality With Digital Virtuality in Metaverse

MARL Sim2real Transfer: Merging Physical Reality With Digital Virtuality in Metaverse

RadarVerses in Metaverses: A CPSI-Based Architecture for 6S Radar Systems in CPSS

Proactive And Smooth Maneuvering For Navigation Around Pedestrians

Contact Info

Product

Resources

About