Design and Experimental Validation of a Cooperative Adaptive Cruise Control System Based on Supervised Reinforcement Learning

Wei, Shouyang; Zou, Yuan; Zhang, Tao; Zhang, Xudong; Wang, Wenwei

doi:10.3390/app8071014

Cited by 32 publications

(18 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Incorporating a supervised network (trained by real driving data) into the actor-critical framework, improves the success rate of training. Furthermore, the driver's characteristics can be learned by the actor to achieve a human-like CACC controller (Wei et al, 2018). There are several validation challenges for adaptive control, including proving convergence over long durations, guaranteeing controller stability, using new tools to compute statistical error bounds, identifying problems in faulttolerant software, and testing in the presence of adaptation (Jacklin et al, 2004).…”

Section: Run-time Monitoring Of Operation Of Control Sub-systemsmentioning

confidence: 99%

From Validation of Medical Devices towards Validation of Adaptive Cyber-Physical Systems

Tavčar

Duhovnik

Horváth

2020

JID

View full text Add to dashboard Cite

Conventionally, designing of a technical system is completed in the product development phase in terms of all details and aspects. This can be done if the expected behavior and the conditions of operation are known in advance and can be validated before the product is launched on the market. Validation is typically based on a predictive analysis or simulation of the designed operation. However, this contradicts with the case of smart systems, such as smart cyber-physical systems (CPSs), which self-manage their operation or at least a part of it. Being able to adapt at run-time and evolve over time, smart CPSs cannot be validated using conventional deterministic approaches. This is especially true for smart CPSs used as instrumentation in the medical field. This gave the stimulation of our background research, the results of which are concisely summarized and critically concluded in this paper. The literature has been found fairly narrow in terms of novel validation approaches for self-managing smart systems. The main finding is that the tasks of operational and behavioral validation should be shared among the system designers and the designed systems. Designers need prognostic approaches, while systems need to construct validation plans and execute them at run-time. This needs validation-specific functionalities and context dependent mechanisms such as run-time validation frameworks or meta-models, objective-sensitive self-monitoring mechanisms, self-constraining and self-supporting mechanisms, and other enablers. Extensive foundational research and system prototype testing are deemed to be indispensable. To make the first small step in this direction, this paper proposes a concept for validation of smart medical CPSs. This relies on the following hypothesis: If a system has freedom for self-adaptation, then it should also be equipped with selfcontrol mechanism, meta-knowledge and a supervisory controller. It all together enables a purpose-and context dependent semantic reasoning about the operational and behavioral objectives. The paper suggests a number of topics for future research towards a run-time validation engine.

show abstract

Section: Run-time Monitoring Of Operation Of Control Sub-systemsmentioning

confidence: 99%

From Validation of Medical Devices towards Validation of Adaptive Cyber-Physical Systems

Tavčar

Duhovnik

Horváth

2020

JID

View full text Add to dashboard Cite

show abstract

“…In [18], a longitudinal dynamic model is considered for predictive speed control using DDPG. In [19], a carfollowing controller is developed with acceleration command dynamics considered using DDPG by learning from naturalistic human-driving data. However, both studies did not investigate the impact of acceleration delay, which could degrade the control performance.…”

Section: Introductionmentioning

confidence: 99%

Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

Lin

McPhee

Azad

2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

The majority of current studies on autonomous vehicle control via deep reinforcement learning (DRL) utilize point-mass kinematic models, neglecting vehicle dynamics which includes acceleration delay and acceleration command dynamics. The acceleration delay, which results from sensing and actuation delays, results in delayed execution of the control inputs. The acceleration command dynamics dictates that the actual vehicle acceleration does not rise up to the desired command acceleration instantaneously due to dynamics. In this work, we investigate the feasibility of applying DRL controllers trained using vehicle kinematic models to more realistic driving control with vehicle dynamics. We consider a particular longitudinal car-following control, i.e., Adaptive Cruise Control (ACC), problem solved via DRL using a pointmass kinematic model. When such a controller is applied to car following with vehicle dynamics, we observe significantly degraded car-following performance. Therefore, we redesign the DRL framework to accommodate the acceleration delay and acceleration command dynamics by adding the delayed control inputs and the actual vehicle acceleration to the reinforcement learning environment state, respectively. The training results show that the redesigned DRL controller results in near-optimal control performance of car following with vehicle dynamics considered when compared with dynamic programming solutions.

show abstract

“…Thus, to solve traffic congestion problems, this paper proposes an intelligent traffic light system to initiate a better traffic light cycle configuration and to simulate the schemes by which traffic officers control traffic lights by analyzing the historical data. To achieve this system, a smart system is needed to adjust the cycle configuration of traffic lights by applying reinforcement learning [1][2][3] in machine learning [4].…”

Section: Introductionmentioning

confidence: 99%

Traffic Light Cycle Configuration of Single Intersection Based on Modified Q-Learning

et al. 2019

View full text Add to dashboard Cite

In recent years, within large cities with a high population density, traffic congestion has become more and more serious, resulting in increased emissions of vehicles and reducing the efficiency of urban operations. Many factors have caused traffic congestion, such as insufficient road capacity, high vehicle density, poor urban traffic planning and inconsistent traffic light cycle configuration. Among these factors, the problems of traffic light cycle configuration are the focal points of this paper. If traffic lights can adjust the cycle dynamically with traffic data, it will reduce degrees of traffic congestion significantly. Therefore, a modified mechanism based on Q-Learning to optimize traffic light cycle configuration is proposed to obtain lower average vehicle delay time, while keeping significantly fewer processing steps. The experimental results will show that the number of processing steps of this proposed mechanism is 11.76 times fewer than that of the exhaustive search scheme, and also that the average vehicle delay is only slightly lower than that of the exhaustive search scheme by 5.4%. Therefore the proposed modified Q-learning mechanism will be capable of reducing the degrees of traffic congestions effectively by minimizing processing steps.

show abstract

Design and Experimental Validation of a Cooperative Adaptive Cruise Control System Based on Supervised Reinforcement Learning

Cited by 32 publications

References 40 publications

From Validation of Medical Devices towards Validation of Adaptive Cyber-Physical Systems

From Validation of Medical Devices towards Validation of Adaptive Cyber-Physical Systems

Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning

Traffic Light Cycle Configuration of Single Intersection Based on Modified Q-Learning

Contact Info

Product

Resources

About