Safe deep reinforcement learning in diesel engine emission control

Abstract

A deep reinforcement learning application is investigated to control the emissions of a compression ignition diesel engine. The main purpose of this study is to reduce the engine-out nitrogen oxide [Formula: see text] emissions and to minimize fuel consumption while tracking a reference engine load. First, a physics-based engine simulation model is developed in GT-Power and calibrated using experimental data. Using this model and a GT-Power/Simulink co-simulation, a deep deterministic policy gradient is developed. To reduce the risk of an unwanted output, a safety filter is added to the deep reinforcement learning. Based on the simulation results, this filter has no effect on the final trained deep reinforcement learning; however, during the training process, it is crucial to enforce constraints on the controller output. The developed safe reinforcement learning is then compared with an iterative learning controller and a deep neural network–based nonlinear model predictive controller. This comparison shows that the safe reinforcement learning is capable of accurately tracking an arbitrary reference input while the iterative learning controller is limited to a repetitive reference. The comparison between the nonlinear model predictive control and reinforcement learning indicates that for this case reinforcement learning is able to learn the optimal control output directly from the experiment without the need for a model. However, to enforce output constraint for safe learning reinforcement learning, a simple model of system is required. In this work, reinforcement learning was able to reduce [Formula: see text] emissions more than the nonlinear model predictive control; however, it suffered from slightly higher error in load tracking and a higher fuel consumption.

Integrated catalytic systems for simultaneous NOx and PM reduction: A comprehensive evaluation of synergistic performance and combustion waste energy utilization

“…In terms of emerging technologies, the integration of non-thermal plasma (NTP) with catalytic aftertreatment systems has shown potential for improving low-temperature performance and reducing the dependence on precious metals (Norouzi, Armin, et al 2023). NTP can generate highly reactive species, such as oxidizing radicals and excited molecules, which can enhance the catalytic reactions at lower temperatures.…”

Gas-Lift Optimization Using Physics-Informed Deep Reinforcement Learning

“…Then, the deep reinforcement learning algorithm is trained on the improved process to tune control errors continuously. Norouzi et al 23 developed a similar strategy for controlling diesel engine emissions. This involved adding a safety layer that used optimization techniques to ensure that the engine's output did not exceed the maximum value specified while minimizing the difference between the actions generated by the deep deterministic policy gradient (DDPG) algorithm and those produced by a linear controller.…”

Cloud-Based Reinforcement Learning in Automotive Control Function Development

“…These use cases are usually characterized by the optimization of the control trajectory under external boundary conditions, e.g., a specific torque that needs to be produced. The literature provides the control of various actuators inside the powertrain, such as an active turbocharger or an exhaust gas recirculation (EGR) valve [30,31,33] of a combustion engine or the inverter of an electric motor [34]. Furthermore, applications outside the powertrain, such as an active steering system [19] or a semi-active suspension [35] control, can be found in the literature.…”