Counterexample-Guided Synthesis of Perception Models and Control

Ghosh, Sandip; Pant, Yash Vardhan; Ravanbakhsh, Hadi; Seshia, Sanjit A.

doi:10.23919/acc50511.2021.9482896

Cited by 10 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, Dean et al [12] considers synthesizing robust perception based controller. Ghosh et al [10] uses VerifAI [9] for counter-example guided controller synthesis with perception models. Plenty of recent works focus on verification [3], [24]-[26], reachability analysis [5], [27]- [30], statistical model checking [31], and synthesis [32] on neural feedback systems with neural network controllers.…”

Section: Related Workmentioning

confidence: 99%

“…Our main claim is that this is one of the first 2 approaches to provide safety assurances for realistic vision-based control systems with abstractions, approximate or otherwise. We use a piece-wise affine template for M : Suppose the ground truth perception input to the control system is m * (x), at a given 2 The only other closely related works are [10] and [11]. state x.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Verifying Controllers With Vision-Based Perception Using Safe Approximate Abstractions

Hsieh

Sun

et al. 2022

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Fully formal verification of perception models is likely to remain challenging in the foreseeable future, and yet these models are being integrated into safety-critical control systems. We present a practical method for reasoning about the safety of such systems. Our method is based on systematically constructing approximations of perception models from system-level safety requirements, data, and program analysis of the modules that are downstream from perception. These approximations have some desirable properties like being lowdimensional, intelligible, and tractable. The closed-loop system, with the approximation substituting the actual perception model, is verified to be safe. Establishing formal relationship between the actual and the approximate perception models remains well beyond available verification techniques. However, we do provide a useful empirical measure of their closeness called precision. Overall, our method can trade off the size of the approximation against precision. We apply the method to two significant case studies (a) a vision-based lane tracking controller for an autonomous vehicle and (b) a controller for an agricultural robot. We show how the generated approximations for each system can be composed with the downstream modules and be verified using program analysis tools like CBMC. Detailed evaluations of the impacts of size, and the environmental parameters (e.g., lighting, road surface, plant type) on the precision of the generated approximations suggest that the approach can be useful for realistic control systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Verifying Controllers With Vision-Based Perception Using Safe Approximate Abstractions

Hsieh

Sun

et al. 2022

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…This has led to development of approaches that find closed-loop perception failures through forward simulation (also referred to as falsification) [18,19]. Even though promising, these approaches often rely on heuristics to effectively search over the image space for finding the failures, including low-dimensional feature encoding and adaptive Bayesian sampling in the feature space [20,21]. However, it is not immediately apparent how to obtain informative low-dimensional feature encoding for complex RGB images.…”

Section: Related Workmentioning

confidence: 99%

Discovering Closed-Loop Failures of Vision-Based Controllers via Reachability Analysis

Chakraborty¹,

Bansal²

2022

Preprint

View full text Add to dashboard Cite

Machine learning driven image-based controllers allow robotic systems to take intelligent actions based on the visual feedback from their environment. Understanding when these controllers might lead to system safety violations is important for their integration in safety-critical applications and engineering corrective safety measures for the system. Existing methods leverage simulation-based testing (or falsification) to find the failures of vision-based controllers, i.e., the visual inputs that lead to closed-loop safety violations. However, these techniques do not scale well to the scenarios involving high-dimensional and complex visual inputs, such as RGB images. In this work, we cast the problem of finding closedloop vision failures as a Hamilton-Jacobi (HJ) reachability problem. Our approach blends simulation-based analysis with HJ reachability methods to compute an approximation of the backward reachable tube (BRT) of the system, i.e., the set of unsafe states for the system under vision-based controllers. Utilizing the BRT, we can tractably and systematically find the system states and corresponding visual inputs that lead to closed-loop failures. These visual inputs can be subsequently analyzed to find the input characteristics that might have caused the failure. Besides its scalability to high-dimensional visual inputs, an explicit computation of BRT allows the proposed approach to capture non-trivial system failures that are difficult to expose via random simulations. We demonstrate our framework on two case studies involving an RGB imagebased neural network controller for (a) autonomous indoor navigation, and (b) autonomous aircraft taxiing.

show abstract

“…Inductive synthesis is heavily used in the context of programming languages but can also be used for perception modules and control [20]. Rather than learning a program, we learn a monitor.…”

Section: Monitor Refinement and Synthesismentioning

confidence: 99%

Formal Analysis of AI-Based Autonomy: From Modeling to Runtime Assurance

Torfah

Junges

Fremont

et al. 2021

Runtime Verification

Self Cite

View full text Add to dashboard Cite

Autonomous systems are increasingly deployed in safetycritical applications and rely more on high-performance AI/ML-based components. Runtime monitors play an important role in raising the level of assurance in AI/ML-based autonomous systems by ensuring that the autonomous system stays safe within its operating environment. In this tutorial, we present VerifAI, an open-source toolkit for the formal design and analysis of systems that include AI/ML components. Ver-ifAI provides features supporting a variety of use cases including formal modeling of the autonomous system and its environment, automatic falsification of system-level specifications as well as other simulationbased verification and testing methods, automated diagnosis of errors, and automatic specification-driven parameter and component synthesis. In particular, we describe the use of VerifAI for generating runtime monitors that capture the safe operational environment of systems with AI/ML components. We illustrate the advantages and applicability of VerifAI in real-life applications using a case study from the domain of autonomous aviation.

show abstract

Counterexample-Guided Synthesis of Perception Models and Control

Cited by 10 publications

References 18 publications

Verifying Controllers With Vision-Based Perception Using Safe Approximate Abstractions

Verifying Controllers With Vision-Based Perception Using Safe Approximate Abstractions

Discovering Closed-Loop Failures of Vision-Based Controllers via Reachability Analysis

Formal Analysis of AI-Based Autonomy: From Modeling to Runtime Assurance

Contact Info

Product

Resources

About