Spatial-Temporal Self-Attention Transformer Networks for Battery State of Charge Estimation

Abstract

Over the past ten years, breakthroughs in battery technology have dramatically propelled the evolution of electric vehicle (EV) technologies. For EV applications, accurately estimating the state-of-charge (SOC) is critical for ensuring safe operation and prolonging the lifespan of batteries, particularly under complex loading scenarios. Despite progress in this area, modeling and forecasting the evaluation of multiphysics and multiscale electrochemical systems under realistic conditions using first-principles and atomistic calculations remains challenging. This study proposes a solution by designing a specialized Transformer-based network architecture, called Bidirectional Encoder Representations from Transformers for Batteries (BERTtery), which only uses time-resolved battery data (i.e., current, voltage, and temperature) as an input to estimate SOC. To enhance the Transformer model’s generalization, it was trained and tested under a wide range of working conditions, including diverse aging conditions (ranging from 100% to 80% of the nominal capacity) and varying temperature windows (from 35 °C to −5 °C). To ensure the model’s effectiveness, a rigorous test of its performance was conducted at the pack level, which allows for the translation of cell-level predictions into real-life problems with hundreds of cells in-series conditions possible. The best models achieve a root mean square error (RMSE) of less than 0.5 test error and approximately 0.1% average percentage error (APE), with maximum absolute errors (MAE) of 2% on the test dataset, accurately estimating SOC under dynamic operating and aging conditions with widely varying operational profiles. These results demonstrate the power of the self-attention Transformer-based model to predict the behavior of complex multiphysics and multiscale battery systems.

Temporal Attention Mechanism Based Indirect Battery Capacity Prediction Combined with Health Feature Extraction

“…Transformer networks are often combined with other deep learning networks due to their advantages of parallel processing and attention mechanisms [22]. Although this combination method performs well in some applications, this method may require a large amount of training data to achieve high accuracy [23].…”

Comparative Study-Based Data-Driven Models for Lithium-Ion Battery State-of-Charge Estimation

“…Unlike recurrent networks, transformers only use self-attention to identify global dependencies between inputs. This allows implicit learning of complex cell dynamics over long charge/discharge profiles for battery SoC modeling [134]. Compared to RNN alternatives, research has developed transformer structures to capture long-range SoC correlations with fewer training samples [135].…”