Automated eco-driving in urban scenarios using deep reinforcement learning

Wegener, Marius; Koch, Lucas; Andert, Jakob

doi:10.1016/j.trc.2021.102967

Cited by 75 publications

(22 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar work has been done by Guo et al with an auxiliary model to factor in lane changing [28]. Wegner et al propose a twin-delayed deep deterministic policy gradient method for eco-driving that takes only the information of the traffic light timing and minimal sensor data as inputs [29]. Although these works model multiple signalized intersections, they focus on reducing the fuel consumption for an ego agent; in contrast, our work seeks to optimize the full system of vehicles.…”

Section: B Reinforcement Learning For Autonomous Traffic Controlmentioning

confidence: 92%

Learning Eco-Driving Strategies at Signalized Intersections

Jayawardana¹,

Wu²

2022

Preprint

View full text Add to dashboard Cite

Signalized intersections in arterial roads result in persistent vehicle idling and excess accelerations, contributing to fuel consumption and CO2 emissions. There has thus been a line of work studying eco-driving control strategies to reduce fuel consumption and emission levels at intersections. However, methods to devise effective control strategies across a variety of traffic settings remain elusive. In this paper, we propose a reinforcement learning (RL) approach to learn effective ecodriving control strategies. We analyze the potential impact of a learned strategy on fuel consumption, CO2 emission, and travel time and compare with naturalistic driving and model-based baselines. We further demonstrate the generalizability of the learned policies under mixed traffic scenarios. Simulation results indicate that scenarios with 100% penetration of connected autonomous vehicles (CAV) may yield as high as 18% reduction in fuel consumption and 25% reduction in CO2 emission levels while even improving travel speed by 20%. Furthermore, results indicate that even 25% CAV penetration can bring at least 50% of the total fuel and emission reduction benefits.

show abstract

Section: B Reinforcement Learning For Autonomous Traffic Controlmentioning

confidence: 92%

Learning Eco-Driving Strategies at Signalized Intersections

Jayawardana¹,

Wu²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In some cases, an energy simulator is not sufficient. For example, in self-driving vehicles that need to consider other vehicles and traffic lights, Wegener et al [28] include a traffic simulator to the environment.…”

Section: General Conceptual Overview For Reinforcement Learning Agent...mentioning

confidence: 99%

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

2022

View full text Add to dashboard Cite

The transition to renewable production and smart grids is driving a massive investment to battery storages, and reinforcement learning (RL) has recently emerged as a potentially disruptive technology for their control and optimization of battery storage systems. A surge of papers has appeared in the last two years applying reinforcement learning to the optimization of battery storages in buildings, energy communities, energy harvesting Internet of Things networks, renewable generation, microgrids, electric vehicles and plug-in hybrid electric vehicles. This article reviews these applications through 4 different perspectives. Firstly, the type of optimization problem is analyzed; the literature can be divided to approaches that optimize either financial targets or energy efficiency. Secondly, the approaches for handling user comfort are analyzed for applications that may impact a human user. Thirdly, this paper discusses the approach to model and reduce battery degradation. Fourthly, the articles are categorized by application context and applications likely to attract a high amount of research are identified. The paper concludes with a list of unresolved challenges.

show abstract

“…Controlling of CV is one representative in this field, as the vehicle can move longitudinally and laterally on the road. While the majority of the existing studies are confined to car following motion (Shi et al, 2018;Mousa et al, 2020;Zhou et al, 2020;Wegener et al, 2021), this paper develops a parameterized action space to naturally describe the control problem with hybrid actions and thereby implement joint optimization of car-following and lane-changing movement. Figure 3 presents an example of parameterized action space.…”

Section: Agent Frameworkmentioning

confidence: 99%

“…Guo et al (2021) combined DDPG and DQN to formulate a hybrid reinforcement learning framework, and the developed approach can not only control the longitudinal motion, but also optimize the lateral decision of the ego vehicle. Wegener et al (2021) studied the application of twin-delayed deep deterministic policy gradient (TD3) algorithm, which conformed to the similar basic idea of DDPG, but introduced a series of tricks to tackle the Q function overestimate problem of DDPG. Their simulation study proclaimed that the DRL algorithm can adapt to the eco-driving scenarios with the presence of other surrounding vehicles.…”

Section: Introductionmentioning

confidence: 99%

Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach

Lü¹,

Zhang²,

Li³

2022

Preprint

View full text Add to dashboard Cite

This paper proposes an eco-driving framework for electric connected vehicles (CVs) based on reinforcement learning (RL) to improve vehicle energy efficiency at signalized intersections. The vehicle agent is specified by integrating the model-based car-following policy, lane-changing policy, and the RL policy, to ensure safe operation of a CV. Subsequently, a Markov Decision Process (MDP) is formulated, which enables the vehicle to perform longitudinal control and lateral decisions, jointly optimizing the car-following and lane-changing behaviors of the CVs in the vicinity of intersections. Then, the hybrid action space is parameterized as a hierarchical structure and thereby trains the agents with two-dimensional motion patterns in a dynamic traffic environment. Finally, our proposed methods are evaluated in SUMO software from both a single-vehicle-based perspective and a flow-based perspective. The results show that our strategy can significantly reduce energy consumption by learning proper action schemes without any interruption of other human-driven vehicles (HDVs).

show abstract

Automated eco-driving in urban scenarios using deep reinforcement learning

Cited by 75 publications

References 27 publications

Learning Eco-Driving Strategies at Signalized Intersections

Learning Eco-Driving Strategies at Signalized Intersections

Exploiting Battery Storages With Reinforcement Learning: A Review for Energy Professionals

Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach

Contact Info

Product

Resources

About