Run-Time Assurance for Learning-Based Aircraft Taxiing

Cofer, Darren; Amundson, Isaac; Sattigeri, Ramachandra; Passi, Arjun; Boggs, Christopher; Smith, Eric; Gilham, Limei; Byun, Taejoon; Rayadurgam, Sanjai

doi:10.1109/dasc50938.2020.9256581

Cited by 22 publications

(13 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Verification Tightness: To compute tighter verification (i.e., tighter reachable set computation) in ReachNN, we set the CutoffThreshold as 1e − 10, QueueSize as 1000, and degree bound of Bernstein polynomials as [2,2]. For less tight verification, we set 1e − 7, 300, and [1,1] for CutoffThreshold, QueueSize, and degree bound, respectively. For Wasserstein distance on the oscillator system, the tighter reachable set computation in average takes around 40 steps with about 115 seconds for each step to learn a neural network controller, compared to the less tight computation that takes about 55 iterations with 86 seconds for each step.…”

Section: Additional Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Verification in the Loop: Correct-by-Construction Control Learning with Reach-avoid Guarantees

Wang,

Huang,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

In the current control design of safety-critical autonomous systems, formal verification techniques are typically applied after the controller is designed to evaluate whether the required properties (e.g., safety) are satisfied. However, due to the increasing system complexity and the fundamental hardness of designing a controller with formal guarantees, such an open-loop process of design-then-verify often results in many iterations and fails to provide the necessary guarantees. In this paper, we propose a correct-by-construction control learning framework that integrates the verification into the control design process in a closed-loop manner, i.e., design-while-verify. Specifically, we leverage the verification results (computed reachable set of the system state) to construct feedback metrics for control learning, which measure how likely the current design of control parameters can meet the required reach-avoid property for safety and goal-reaching. We formulate an optimization problem based on such metrics for tuning the controller parameters, and develop an approximated gradient descent algorithm with a difference method to solve the optimization problem and learn the controller. The learned controller is formally guaranteed to meet the required reach-avoid property. By treating verifiability as a first-class objective and effectively leveraging the verification results during the control learning process, our approach can significantly improve the chance of finding a control design with formal property guarantees. This is demonstrated via a set of experiments on both linear and non-linear systems that use model-based or neural network based controllers.Preprint. Under review.

show abstract

Section: Additional Discussionmentioning

confidence: 99%

“…Safety-critical autonomous systems, such as avionics systems [1] and self-driving vehicles [2], often operate in highly dynamic environments with significant uncertainties and disturbances. It is critical yet challenging to formally ensure their safety, especially for the control and decision making modules.…”

Section: Introductionmentioning

confidence: 99%

Verification in the Loop: Correct-by-Construction Control Learning with Reach-avoid Guarantees

Wang,

Huang,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…A first family of approaches predicts misbehavior of a ML model by analyzing its inputs. For instance, reconstruction errors from auto encoders can be used to determine if an input is within the validity domain of the model [17], [18], [19]. Another kind of Neural Network (NN) monitor models activation patterns of internal layers during training, and uses it to detect out-of-distribution (OOD) data [20].…”

Section: B Runtime Monitoring Of MLmentioning

confidence: 99%

“…For example, the highest softmax activation can be used as an anomaly score [21], and when possible, softmax values can even be calibrated using OOD data [22]. We also mention that some works have implemented MLRM mechanism within the context of practical safety applications [19], [18], [23].…”

Section: B Runtime Monitoring Of MLmentioning

confidence: 99%

Evaluation of Runtime Monitoring for UAV Emergency Landing

Guerin,

Delmas,

Guiochet

2022

Preprint

View full text Add to dashboard Cite

To certify UAV operations in populated areas, risk mitigation strategies -such as Emergency Landing (EL)must be in place to account for potential failures. EL aims at reducing ground risk by finding safe landing areas using onboard sensors. The first contribution of this paper is to present a new EL approach, in line with safety requirements introduced in recent research. In particular, the proposed EL pipeline includes mechanisms to monitor learning based components during execution. This way, another contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM) approaches within the context of a real-world critical system. A new evaluation methodology is introduced, and applied to assess the practical safety benefits of three MLRM mechanisms. The proposed approach is compared to a default mitigation strategy (open a parachute when a failure is detected), and appears to be much safer.

show abstract

“…Traditional methods from control theory provide powerful tools for safety and performance analysis but lack the expressiveness to deal with rich sensing models such as vision or Lidar. On the other hand, learning-based methods have been used successfully on visual-feedback control tasks including autonomous driving [1] and aircraft taxiing [2], but ensuring the safety of these controllers remains an open question.…”

Section: Introductionmentioning

confidence: 99%

Learning Safe, Generalizable Perception-based Hybrid Control with Certificates

Dawson¹,

Lowenkamp²,

Goff³

et al. 2022

Preprint

View full text Add to dashboard Cite

Many robotic tasks require high-dimensional sensors such as cameras and Lidar to navigate complex environments, but developing certifiably safe feedback controllers around these sensors remains a challenging open problem, particularly when learning is involved. Previous works have proved the safety of perception-feedback controllers by separating the perception and control subsystems and making strong assumptions on the abilities of the perception subsystem. In this work, we introduce a novel learning-enabled perception-feedback hybrid controller, where we use Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) to show the safety and liveness of a full-stack perception-feedback controller. We use neural networks to learn a CBF and CLF for the full-stack system directly in the observation space of the robot, without the need to assume a separate perception-based state estimator. Our hybrid controller, called LOCUS (Learning-enabled Observationfeedback Control Using Switching), can safely navigate unknown environments, consistently reach its goal, and generalizes safely to environments outside of the training dataset. We demonstrate LOCUS in experiments both in simulation and in hardware, where it successfully navigates a changing environment using feedback from a Lidar sensor.

show abstract

Run-Time Assurance for Learning-Based Aircraft Taxiing

Cited by 22 publications

References 7 publications

Verification in the Loop: Correct-by-Construction Control Learning with Reach-avoid Guarantees

Verification in the Loop: Correct-by-Construction Control Learning with Reach-avoid Guarantees

Evaluation of Runtime Monitoring for UAV Emergency Landing

Learning Safe, Generalizable Perception-based Hybrid Control with Certificates

Contact Info

Product

Resources

About