Improving Neural Network Verification through Spurious Region Guided Refinement

Abstract

We propose a spurious region guided refinement approach for robustness verification of deep neural networks. Our method starts with applying the DeepPoly abstract domain to analyze the network. If the robustness property cannot be verified, the result is inconclusive. Due to the over-approximation, the computed region in the abstraction may be spurious in the sense that it does not contain any true counterexample. Our goal is to identify such spurious regions and use them to guide the abstraction refinement. The core idea is to make use of the obtained constraints of the abstraction to infer new bounds for the neurons. This is achieved by linear programming techniques. With the new bounds, we iteratively apply DeepPoly, aiming to eliminate spurious regions. We have implemented our approach in a prototypical tool DeepSRGR. Experimental results show that a large amount of regions can be identified as spurious, and as a result, the precision of DeepPoly can be significantly improved. As a side contribution, we show that our approach can be applied to verify quantitative robustness properties.

nnenum: Verification of ReLU Neural Networks with Optimized Abstraction Refinement

“…Other optimizations may provide further improvements to performance, such as using the spurious region to guide refinement [40], similar to counter-example guided abstraction refinement [7]. We did not find a way to use this in our algorithm without hurting overall performance, although it may be an avenue for further investigation.…”

Scalable Verification of GNN-based Job Schedulers

“…[44] proposes a procedure specific to fairness properties which uses a forward pre-analysis to partition the input region and a post-condition guided backward analysis to prove the properties for all activation patterns in the input region. [57] proposes a fundamentally different abstraction-refinement loop where the backward analysis iteratively refines the pre-condition for DeepPoly analysis. In contrast, we develop a general framework for forwardbackward analysis on neural networks that can be instantiated with different abstract domains.…”

BDD4BNN: A BDD-Based Quantitative Analysis Framework for Binarized Neural Networks

“…Two sampling-based approaches were proposed to certify the robustness for both DNNs and BNNs [5,65]. Yang et al [69] proposed a spurious region-guided refinement approach for real-numbered DNN verification, claiming to be the first work of the quantitative robustness verification of DNNs with soundness guarantees.…”

“…Quantitative analysis of general neural networks, however, is challenging, hence received little attention and for which the results are rather limited so far. DeepSRGR [69] presented an abstract interpretation based quantitative robustness verification approach for DNNs which is sound but incomplete. For BNNs, approximate SAT model-counting solvers ( SAT) are leveraged [6,47] based on the SAT encoding for the qualitative counterpart.…”