Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle

Xu, Bin; Shi, Junzhe; Li, Sixu; Li, Huayi; Wang, Zhe

doi:10.1016/j.energy.2021.120705

Cited by 43 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several formulations of the reinforcement learning mechanism exist, and these have been applied in the battery energy management domain. The simplest is q-learning, which involves a table for mapping states and actions [46]. As our problem formulation involves a small state space, q-learning could have been used in this work.…”

Section: Battery Trading Systemmentioning

confidence: 99%

A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage

et al. 2021

View full text Add to dashboard Cite

Battery storages are an essential element of the emerging smart grid. Compared to other distributed intelligent energy resources, batteries have the advantage of being able to rapidly react to events such as renewable generation fluctuations or grid disturbances. There is a lack of research on ways to profitably exploit this ability. Any solution needs to consider rapid electrical phenomena as well as the much slower dynamics of relevant electricity markets. Reinforcement learning is a branch of artificial intelligence that has shown promise in optimizing complex problems involving uncertainty. This article applies reinforcement learning to the problem of trading batteries. The problem involves two timescales, both of which are important for profitability. Firstly, trading the battery capacity must occur on the timescale of the chosen electricity markets. Secondly, the real-time operation of the battery must ensure that no financial penalties are incurred from failing to meet the technical specification. The trading-related decisions must be done under uncertainties, such as unknown future market prices and unpredictable power grid disturbances. In this article, a simulation model of a battery system is proposed as the environment to train a reinforcement learning agent to make such decisions. The system is demonstrated with an application of the battery to Finnish primary frequency reserve markets.

show abstract

Section: Battery Trading Systemmentioning

confidence: 99%

A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Tey also showed that if an EV continues to operate after the 30% battery degradation limit, the greenhouse gas emissions and energy consumption can be signifcantly increased. Xu et al [33] proposed a Q-learning-based strategy to minimize Li-ion battery degradation and energy consumption. Te Q-learning method is an adaptive optimal control algorithm that uses the Bellman equation of dynamic programming.…”

Section: Introductionmentioning

confidence: 99%

Multiparadigm Modeling Framework to Evaluate the Impacts of Travel Patterns on Electric Vehicle Battery Lifespan

Agrawal

Peeta

Miralinaghi

2023

Journal of Advanced Transportation

View full text Add to dashboard Cite

The widespread adoption of electric vehicles (EVs) can help attain economic and environmental sustainability by reducing oil dependency and greenhouse gas emissions. However, several issues need to be addressed before EVs can become a popular vehicle choice among the general public. A key issue is the perpetual reduction in EV battery capacity caused by battery degradation over time with usage. This can lead to a reduced driving range and cause “range anxiety” for EV drivers. This becomes even more critical in developing countries where consumers are highly sensitive to battery replacement costs. Thus, to promote EVs in developing economies, policymakers and vehicle manufacturers need to develop attractive incentive schemes and warranty strategies preceded by a thorough assessment of the useable EV battery lifespan for a wide range of users. This paper develops a multiparadigm modeling framework to compute battery degradation for a large population of EVs by capturing the effects of travel patterns, traffic conditions, and ambient temperature. The proposed framework consists of four different building blocks: (i) a microscopic traffic simulation model to generate speed profiles, (ii) an EV power consumption model, (iii) a battery equivalent circuit model, and (iv) a semiempirical battery degradation model. The proposed framework can also be used to assess the battery life-cycle of electric-powered automated vehicles by adjusting their travel patterns accordingly. A case study is presented using travel diary data of around 700 households from the U.S. National Household Travel Survey of 2009 to simulate household travel patterns and corresponding battery lifespan distribution.

show abstract

“…Research addressing the transition to EVs is extensive and has focused on a wide variety of technical and social issues. Regarding the first, for instance, Zhang et al [10], discuss the advantages and barriers for the use of thermal management methods of lithium-ion batteries in EVs, while Xu et al [11], investigated EVs' batteries aging and degradation. Others have explored how the deployment of EVs, in tandem with the large-scale penetration of variable renewable energy sources, generates technical challenges for power grid balancing.…”

Section: Introductionmentioning

confidence: 99%

Measuring the equity impacts of government subsidies for electric vehicles

Caulfield

Rio

Stefaniec

et al. 2022

Energy

View full text Add to dashboard Cite

Energy consumption and battery aging minimization using a Q-learning strategy for a battery/ultracapacitor electric vehicle

Cited by 43 publications

References 27 publications

A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage

A Simulation Environment for Training a Reinforcement Learning Agent Trading a Battery Storage

Multiparadigm Modeling Framework to Evaluate the Impacts of Travel Patterns on Electric Vehicle Battery Lifespan

Measuring the equity impacts of government subsidies for electric vehicles

Contact Info

Product

Resources

About