Runtime Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems

Abstract

More than three miles above the Arizona desert, an F-16 student pilot experienced a gravityinduced loss of consciousness (GLOC), passing out while turning at nearly 9Gs (nine times the force of gravity) flying over 400 knots (over 460 miles per hour). With its pilot unconscious, the aircraft turn devolved into a dive, dropping from over 17,000 feet to less than 8,000 feet in altitude in less than 10 seconds. An auditory warning in the cockpit called out to the pilot "altitude, altitude" just before he crossed through 11,000 feet, switching to a command to "pull up" around 8,000 feet. Meanwhile, the student's instructor was watching the event unfold from his own aircraft. As the student's aircraft passed through 12,500 feet, the instructor called over the radio "two recover," commanding the student ("two") to end the dive. As the student's aircraft passed through 11,000 feet the instructor's "two recover!" came with increased urgency. At 9,000 feet, and with terror rising in his voice the instructor yelled "TWO RECOVER!" Fortunately, at the same time as the instructor's third panicked radio call, a new Run Time Assurance (RTA) system kicked in to automatically recover the aircraft. The Automatic Ground Collision Avoidance System (Auto GCAS), an RTA system integrated on the jets less than two years earlier in the Fall of 2014, detected that the aircraft was about to collide, commanded a roll to wings level and pull up maneuver, and recovered the aircraft less than 3,000 feet above the

Responsible (use of) AI

“…Run time assurance (RTA) is the process of augmenting a complex system, such as a trained neural network, to ensure that its output meets some desired properties (safety, ethics, performance, etc.). RTA theory is sourced from the control theory community, where it is often implemented as a wrapper that monitors the output of a complex controller (e.g., neural network control system) and modifies its output when necessary to assure safety (Hobbs et al, 2023 ). An example RTA system is the Auto-GCAS system mentioned earlier.…”