Addressing infinite-horizon optimization in MPC via Q-learning

Beckenbach, Lukas; Osinenko, Pavel; Streif, Stefan

doi:10.1016/j.ifacol.2018.10.175

Cited by 10 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is suggested to constrain the RL-method accordingly. Similar philosophy was pursued in the previous work of the authors (Beckenbach et al, 2018;Göhrt et al, 2019). However, the current work greatly generalizes the previous derivations.…”

Section: Introductionsupporting

confidence: 63%

A reinforcement learning method with closed-loop stability guarantee

et al. 2020

Self Cite

View full text Add to dashboard Cite

Section: Introductionsupporting

confidence: 63%

A reinforcement learning method with closed-loop stability guarantee

et al. 2020

Self Cite

View full text Add to dashboard Cite

“…In Section IV, Theorem 2, we have established an estimation scheme which delivers asymptotically accurate predictor maps for any lifted signals (14) assuming that these signals can be embedded in a linear signal model of the form (18). Utilizing the estimation scheme (15) in the control scheme (3) means now that the signal model ( 18) replaces the system model ( 2) and the predictor maps in (21) are used to define the predictor (5). However, it needs to be clarified how the output and input sequence {y(k)} k∈N , {v(k)} k∈N of (1) under Assumption 1 can be related to a signal model of the form (18) such that (A, B) is stabilizable and such that the (observable part of the) state x(k) is available.…”

Section: The Overall Schemementioning

confidence: 99%

“…See for example [11], [12], [13], [14] and references therein on this topic. More recent related research on (partially) unknown systems and receding horizon control are for example discussed in [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], to mention only a few out of the rapidly growing literature. A complete review of the state of the art of controlling (partially) unknown systems with receding horizon schemes is out of the scope of this work.…”

Section: Introductionmentioning

confidence: 99%

Control of Unknown (Linear) Systems with Receding Horizon Learning

Ebenbauer,

Pfitz,

2020

Preprint

View full text Add to dashboard Cite

A receding horizon learning scheme is proposed to transfer the state of a discrete-time dynamical control system to zero without the need of a system model. Global state convergence to zero is proved for the class of stabilizable and detectable linear time-invariant systems, assuming that only input and output data is available and an upper bound of the state dimension is known. The proposed scheme consists of a receding horizon control scheme and a proximity-based estimation scheme to estimate and control the closed-loop trajectory. Simulations are presented for linear and nonlinear systems.

show abstract

“…Studies have been conducted to compare MPC and DQN [35]. In addition, there are studies in which the combination of MPC-DQN is used to increase the performance of MPC and to select design parameters, such as the prediction horizon and weight [36][37][38].…”

Section: Introductionmentioning

confidence: 99%

Energy Management of P2 Hybrid Electric Vehicle Based on Event-Triggered Nonlinear Model Predictive Control and Deep Q Network

Haspolat

Yalçın

2023

WEVJ

View full text Add to dashboard Cite

Hybrid electric vehicles (HEVs) are used as a bridge during the transition to battery electric vehicles (BEVs) and to make energy consumption more efficient. The main problem in improving the efficiency of HEV energy consumption is torque management. In this study, a novel approach based on a nonlinear model predictive controller to solve the reference tracking and torque distribution problem is proposed. That is to say, in order to increase the efficiency of torque distribution, the weights of nonlinear model predictive control (NMPC) are trained with a Deep Q Network (DQN), and an event-triggered mechanism is designed with DQN to reduce the computational cost of MPC. The considered torque distribution problem varies according to the type and structure of the HEV. In this study, a parallel type 2 hybrid electric vehicle (P2 HEV) is considered and modeled via publicly shared passenger vehicle data of the engine, motor, high-voltage battery, transmission, clutch, differential, and wheel characteristics. NMPC is formulated so that the torque values remain within the physical limits of the engine, and the battery also operates at its physical limits. Namely, it is guaranteed that the battery works according to a certain state of charge (SOC) window and current limits. The state of health (SOH) of the battery is also considered in the optimization. The motor and engine efficiencies increase by 3.61% and 2.86%, respectively, with the proposed control structure, while the computational cost is reduced by 52.01% when utilizing the proposed event-triggering mechanism in the NMPC controller.

show abstract

Addressing infinite-horizon optimization in MPC via Q-learning

Cited by 10 publications

References 20 publications

A reinforcement learning method with closed-loop stability guarantee

A reinforcement learning method with closed-loop stability guarantee

Control of Unknown (Linear) Systems with Receding Horizon Learning

Energy Management of P2 Hybrid Electric Vehicle Based on Event-Triggered Nonlinear Model Predictive Control and Deep Q Network

Contact Info

Product

Resources

About