Model-free Learning to Avoid Constraint Violations: An Explicit Reference Governor Approach

Liu, Kaiwen; Li, Nan; Rizzo, Denise; Garone, Emanuele; Kolmanovsky, Ilya; Girard, Anouck

doi:10.23919/acc.2019.8814772

Cited by 18 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The above assumptions (A1)-(A3) characterize the class of systems to be treated. Although the system (1) is stable (or pre-stabilized), the nominal controller may not have the ability to enforce the imposed constraints (2). In addition, for many practical systems, the functions f and g in their corresponding models (1) can be highly complex or not given explicitly (e.g., when the model is given as a black-box simulation code).…”

Section: Problem Formulationmentioning

confidence: 99%

“…Typical RG designs are model-based. For the case when the model is unknown, a model-free learning-based approach to enforcing state/output constraints based on RG scheme has been proposed [2] for non safety-critical control systems, where constraint violations are undesirable but do not lead to catastrophic consequences. The learning algorithm proposed in [2] evolves RG based on observed constraint violations during the learning phase and gradually eliminates these violations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Safe Learning Reference Governor: Theory and Application to Fuel Truck Rollover Avoidance

Liu¹,

Li²,

Kolmanovsky³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a learning reference governor (LRG) approach to enforce state and control constraints in systems for which an accurate model is unavailable; and this approach enables the reference governor to gradually improve command tracking performance through learning while enforcing the constraints during learning and after learning is completed. The learning can be performed either on a black-box type model of the system or directly on the hardware. After introducing the LRG algorithm and outlining its theoretical properties, this paper investigates LRG application to fuel truck rollover avoidance. Through simulations based on a fuel truck model that accounts for liquid fuel sloshing effects, we show that the proposed LRG can effectively protect fuel trucks from rollover accidents under various operating conditions.

show abstract

Section: Problem Formulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Safe Learning Reference Governor: Theory and Application to Fuel Truck Rollover Avoidance

Liu¹,

Li²,

Kolmanovsky³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, Liu et al 33 introduced the concept of model-free learning reference governor applied to systems where constraint violations are undesirable but not catastrophic. Under such an assumption, the system will initially exhibit occasional constraint violations (e.g.…”

Section: Learning Reference Governor For Avoiding Misfire Eventsmentioning

confidence: 99%

“…Such a learning procedure is formally presented in Algorithm 1. Theoretical properties of the learning algorithm, including its convergence, are addressed in Liu et al 33 Note that the algorithm is contractive in nature and is not able to increase the point-wise values of Γ ( v ) to “hunt” for the misfire limit. Such limitations will be a subject of future studies.…”

Section: Learning Reference Governor For Avoiding Misfire Eventsmentioning

confidence: 99%

Learning reference governor for cycle-to-cycle combustion control with misfire avoidance in spark-ignition engines at high exhaust gas recirculation–diluted conditions

Maldonado

Kolmanovsky

et al. 2020

International Journal of Engine Research

Self Cite

View full text Add to dashboard Cite

Cycle-to-cycle feedback control is employed to achieve optimal combustion phasing while maintaining high levels of exhaust gas recirculation by adjusting the spark advance and the exhaust gas recirculation valve position. The control development is based on a control-oriented model that captures the effects of throttle position, exhaust gas recirculation valve position, and spark timing on the combustion phasing. Under the assumption that in-cylinder pressure information is available, an adaptive extended Kalman filter approach is used to estimate the exhaust gas recirculation rate into the intake manifold based on combustion phasing measurements. The estimation algorithm is adaptive since the cycle-to-cycle combustion variability (output covariance) is not known a priori and changes with operating conditions. A linear quadratic regulator controller is designed to maintain optimal combustion phasing while maximizing exhaust gas recirculation levels during load transients coming from throttle tip-in and tip-out commands from the driver. During throttle tip-outs, however, a combination of a high exhaust gas recirculation rate and an overly advanced spark, product of the dynamic response of the system, generates a sequence of misfire events. In this work, an explicit reference governor is used as an add-on scheme to the closed-loop system in order to avoid the violation of the misfire limit. The reference governor is enhanced with model-free learning which enables it to avoid misfires after a learning phase. Experimental results are reported which illustrate the potential of the proposed control strategy for achieving an optimal combustion process during highly diluted conditions for improving fuel efficiency.

show abstract

“…This assumption is often a complicated or impossible task to conduct. Very little has been done to propose data-driven outer-loop strategies in a trajectory tracking framework for systems where, besides the infeasibility to change its local controller, there is not an accurate model available that characterizes its dynamics [8,9]. We address this problem in this paper.…”

Section: Introductionmentioning

confidence: 99%

Data-driven Outer-Loop Control Using Deep Reinforcement Learning for Trajectory Tracking

Arroyo,

Giraldo

2020

Preprint

View full text Add to dashboard Cite

Reference tracking systems involve a plant that is stabilized by a local feedback controller and a command center that indicates the reference set-point the plant should follow. Typically, these systems are subject to limitations such as disturbances, systems delays, constraints, uncertainties, underperforming controllers, and unmodeled parameters that do not allow them to achieve the desired performance. In situations where it is not possible to redesign the inner-loop system, it is usual to incorporate an outer-loop control that instructs the system to follow a modified reference path such that the resultant path is close to the ideal one. Typically, strategies to design the outer-loop control need to know a model of the system, which can be an unfeasible task. In this paper, we propose a framework based on deep reinforcement learning that can learn a policy to generate a modified reference that improves the system's performance in a non-invasive and model-free fashion. To illustrate the effectiveness of our approach, we present two challenging cases in engineering: a flight control with a pilot model that includes human reaction delays, and a mean-field control problem for a massive number of space-heating devices. The proposed strategy successfully designs a reference signal that works even in situations that were not seen during the learning process.

show abstract

Model-free Learning to Avoid Constraint Violations: An Explicit Reference Governor Approach

Cited by 18 publications

References 14 publications

Safe Learning Reference Governor: Theory and Application to Fuel Truck Rollover Avoidance

Safe Learning Reference Governor: Theory and Application to Fuel Truck Rollover Avoidance

Learning reference governor for cycle-to-cycle combustion control with misfire avoidance in spark-ignition engines at high exhaust gas recirculation–diluted conditions

Data-driven Outer-Loop Control Using Deep Reinforcement Learning for Trajectory Tracking

Contact Info

Product

Resources

About