Maximum Resilience of Artificial Neural Networks

Abstract

The deployment of Artificial Neural Networks (ANNs) in safety-critical applications poses a number of new verification and certification challenges. In particular, for ANN-enabled self-driving vehicles it is important to establish properties about the resilience of ANNs to noisy or even maliciously manipulated sensory input. We are addressing these challenges by defining resilience properties of ANN-based classifiers as the maximum amount of input or sensor perturbation which is still tolerated. This problem of computing maximum perturbation bounds for ANNs is then reduced to solving mixed integer optimization problems (MIP). A number of MIP encoding heuristics are developed for drastically reducing MIP-solver runtimes, and using parallelization of MIP-solvers results in an almost linear speed-up in the number (up to a certain limit) of computing cores in our experiments. We demonstrate the effectiveness and scalability of our approach by means of computing maximum resilience bounds for a number of ANN benchmark sets ranging from typical image recognition scenarios to the autonomous maneuvering of robots.

Certified Robustness of Community Detection against Adversarial Structural Perturbation via Randomized Smoothing

“…However, these defenses were often soon broken by adaptive attacks [1,7]. In response, various certified defenses (e.g., [10,15,32,33,37]) against adversarial examples have been developed. Among these methods, randomized smoothing is state-of-the-art.…”

nn-dependability-kit: Engineering Neural Networks for Safety-Critical Autonomous Driving Systems

“…The goal is to provide evidence of uncertainty reduction in key phases of the product life cycle, ranging from data collection, training & validation, testing & generalization, to operation. nn-dependability-kit is built upon our previous research work [2]- [6], where (a) novel dependability metrics [3], [5] are introduced to indicate uncertainties being reduced in the engineering life cycle, (b) formal reasoning engine [2], [6] is used to ensure that the generalization does not lead to undesired behaviors, and (c) runtime neuron activation pattern monitoring [4] is applied to reason whether a decision of a neural network in operation time is supported by prior similarities in the training data.…”

Quantitative Projection Coverage for Testing ML-enabled Autonomous Systems

“…The key motivation of this work is that, apart from recent formal verification efforts [5,7,8,10] where scalability and lack of specification are obvious concerns, the most plausible approach, from a certification perspective, remains to be testing. As domain experts or authorities in autonomous driving may suggest n (incomplete) weighted criteria for describing the operating conditions such as weather, landscape, or partially occluding pedestrians, with these criteria one can systematically partition the domain and weight each partitioned class based on its relative importance.…”

“…For sound-and-complete approaches, Reluplex and Planet developed specialized rules for managing the 0-1 activation in the proof system [7,10]. Our previous work [4,5] focused on the reduction to mixed integer liner programming (MILP) and applied techniques to compute tighter bounds such that in MILP, the relaxation bound is closer to the real bound. Exact approaches suffer from combinatorial explosion and currently the verification speed is not satisfactory.…”