A Hybrid End-to-End Control Strategy Combining Dueling Deep Q-network and PID for Transient Boost Control of a Diesel Engine with Variable Geometry Turbocharger and Cooled EGR

Abstract

Deep reinforcement learning (DRL), which excels at solving a wide variety of Atari and board games, is an area of machine learning that combines the deep learning approach and reinforcement learning (RL). However, to the authors’ best knowledge, there seem to be few studies that apply the latest DRL algorithms on real-world powertrain control problems. If there are any, the requirement of classical model-free DRL algorithms typically for a large number of random exploration in order to realize good control performance makes it almost impossible to implement directly on a real plant. Unlike most of the other DRL studies, whose control strategies can only be trained in a simulation environment—especially when a control strategy has to be learned from scratch—in this study, a hybrid end-to-end control strategy combining one of the latest DRL approaches—i.e., a dueling deep Q-network and traditional Proportion Integration Differentiation (PID) controller—is built, assuming no fidelity simulation model exists. Taking the boost control of a diesel engine with a variable geometry turbocharger (VGT) and cooled (exhaust gas recirculation) EGR as an example, under the common driving cycle, the integral absolute error (IAE) values with the proposed algorithm are improved by 20.66% and 9.7% respectively for the control performance and generality index, compared with a fine-tuned PID benchmark. In addition, the proposed method can also improve system adaptiveness by adding another redundant control module. This makes it attractive to real plant control problems whose simulation models do not exist, and whose environment may change over time.

EGR Intelligent Control of Diesel Engine Based on Deep Reinforcement Learning

“…A critic is used for evaluating the policy function estimated by the actor according to the temporal TD error (see Fig. 2) [13][14][15].…”

Data-driven control of automotive diesel engines and after-treatment systems: State of the art and future challenges

“…126 In this work, the algorithm of RL is implemented by the proximal policy optimization method and achieved a good control performance. Hu et al 127 and Hu and Li 128 investigated several end-to-end control methods based on deep RL for transient boost control of diesel engines. The proposed algorithms proved efficient for performance improvement and learning efficiency.…”

Investigation of the Lean Stable Limit of a Barrier Discharge Igniter and of a Streamer-Type Corona Igniter at Different Engine Loads in a Single-Cylinder Research Engine

“…CSI Pressure interval (from-to), bar 3.4-4. 5 Considering that TON = 1500 μs at 1000 rpm corresponds to 9 CAD, the in-cylinder pressure conditions considerably change during the test. For this reason, the mean in-cylinder pressure signal (Pcyl) was divided into 15 intervals (corresponding to an activation time of 100 μs), in which the pressure can be considered "almost-constant" (Figure 8), to be then reproducible in the calorimeter.…”

“…Nowadays, automotive research is focused on developing innovative solutions such as engine downsizing [1][2][3], cooled external exhaust gas recirculation (EGR) [4,5], water injection [6], and lean mixture operations [7,8], to satisfy both the stringent regulations of pollutant emissions and customer requirements [9,10].…”