Deep reinforcement learning for dynamic control of fuel injection timing in multi-pulse compression ignition engines

Abstract

Conventional compression-ignition (CI) engines have long offered high thermal efficiencies and torque across a wide range of loads, but often require extensive exhaust gas treatment that decreases efficiency to meet ever-increasing emissions regulations. One strategy to decrease emissions is to split the fuel injection into a series of smaller injections. In this paper, we explore a new way of discovering optimal control strategies for the next generation of CI engines using deep reinforcement learning (DRL). We outline a DRL procedure to maximize the weighted reward of engine work while minimizing end-of-cycle NO x emissions. Through the procedure outlined in this paper, we show that the DRL agent is able to reduce NO x emissions threefold while only decreasing network by 2%. We demonstrate the use of transfer learning (TL) across hierarchies of physical models to accelerate the learning process, making this approach feasible for a range of control problems within this space. This paper presents a framework and demonstration for using DRL to design control systems in technology areas such as multi-pulse engine control where a hierarchy of models combined with multi-objective rewards are used for optimal operation.

Deep reinforcement learning for turbulent drag reduction in channel flows

“…Furthermore, DRL has been applied to a number of other control tasks, ranging from simple onedimensional falling-fluid instabilities [40], convection problems [41], chaotic turbulent combustion systems [42] to a variety of engineering cases [43][44][45][46].…”

Deep reinforcement learning for turbulent drag reduction in channel flows

“…Furthermore, DRL has been applied to a number of other control tasks, ranging from simple one-dimensional falling-fluid instabilities [34], convection problems [35], chaotic turbulent combustion systems [36] to a variety of engineering cases [37][38][39][40].…”

Data-driven control of automotive diesel engines and after-treatment systems: State of the art and future challenges

“…The control policy of the agent would be iteratively optimized by trial and error using feedback from its actions and experiences. Motivated by the feature of RL, the authors in Henry de Frahan et al 125 developed a new control strategy based on RL to reduce the N O x emission for diesel engines. Similarly, RL was also used for emission and energy management in diesel HEV.…”

“…Henry de Frahan et al, 125 Hofstetter et al, 126 Hu et al, 127 Hu and Li, 128 Xu and Li 129 SVM Liu et al, 94 Wong et al 103 Liu et al 112 Niu et al 130…”