The Black-Box Simplex Architecture for Runtime Assurance of Autonomous CPS

Abstract

In this paper, we explore using runtime verification to design safe cyber-physical systems (CPS). We build upon the Simplex Architecture, where control authority may switch from an unverified and potentially unsafe advanced controller to a backup baseline controller in order to maintain system safety. New to our approach, we remove the requirement that the baseline controller is statically verified. This is important as there are many types of powerful control techniques-model-predictive control, rapidly-exploring random trees and neural network controllersthat often work well in practice, but are difficult to statically prove correct, and therefore could not be used before as baseline controllers. We prove that, through more extensive runtime checks, such an approach can still guarantee safety. We call this approach the Black-Box Simplex Architecture, as both high-level controllers are treated as black boxes. We present case studies where model-predictive control provides safety for multi-robot coordination, and neural networks provably prevent collisions for groups of F-16 aircraft, despite occasionally outputting unsafe actions.

Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems

“…A practical consequence of this is that high performing black-box controllers can safely be used to generate backup trajectories. This concept has been used in practice [17] and is thoroughly explored for the context of Simplex-based systems in [108]. Furthermore, the idea extends to other methods of RTA.…”