Robustness Verification of Semantic Segmentation Neural Networks Using Relaxed Reachability

Abstract

This paper introduces robustness verification for semantic segmentation neural networks (in short, semantic segmentation networks [SSNs]), building on and extending recent approaches for robustness verification of image classification neural networks. Despite recent progress in developing verification methods for specifications such as local adversarial robustness in deep neural networks (DNNs) in terms of scalability, precision, and applicability to different network architectures, layers, and activation functions, robustness verification of semantic segmentation has not yet been considered. We address this limitation by developing and applying new robustness analysis methods for several segmentation neural network architectures, specifically by addressing reachability analysis of up-sampling layers, such as transposed convolution and dilated convolution. We consider several definitions of robustness for segmentation, such as the percentage of pixels in the output that can be proven robust under different adversarial perturbations, and a robust variant of intersection-over-union (IoU), the typical performance evaluation measure for segmentation tasks. Our approach is based on a new relaxed reachability method, allowing users to select the percentage of a number of linear programming problems (LPs) to solve when constructing the reachable set, through a relaxation factor percentage. The approach is implemented within NNV, then applied and evaluated on segmentation datasets, such as a multi-digit variant of MNIST known as M2NIST. Thorough experiments show that by using transposed convolution for up-sampling and average-pooling for down-sampling, combined with minimizing the number of ReLU layers in the SSNs, we can obtain SSNs with not only high accuracy (IoU), but also that are more robust to adversarial attacks and amenable to verification. Additionally, using our new relaxed reachability method, we can significantly reduce the verification time for neural networks whose ReLU layers dominate the total analysis time, even in classification tasks.

An End-to-End Data-Adaptive Pancreas Segmentation System with an Image Quality Control Toolbox

“…Robustness typically denotes the property of a system to maintain its primary performance in the presence of fuctuations in some parameters [52,53]. Normally, robustness is used to evaluate how stable a system is against uncertain utilization environments.…”

First three years of the international verification of neural networks competition (VNN-COMP)

“…Team Diego Manzanas Lopez (Vanderbilt University), Neelanjana Pal (Vanderbilt University), Samuel Sasaki (Vanderbilt University), Hoang-Dung Tran (University of Nebraska-Lincoln), Taylor T. Johnson (Vanderbilt University) Description The Neural Network Verification (NNV) Tool [41,24] is a formal verification software tool for deep learning models and cyber-physical systems with neural network components written in MATLAB and available at https://github.com/verivital/nnv. NNV uses a star-set state-space representation and reachability algorithm that allows for a layer-by-layer computation of exact or overapproximate reachable sets for feed-forward [39], convolutional [36], semantic segmentation (SSNN) [40], and recurrent (RNN) [38] neural networks, as well as neural network control systems (NNCS) [37,41] and neural ordinary differential equations (Neural 5 Thanks to the authors of the α − β-CROWN team, an unsoundness issue of the competition version of Marabou on the ViT benchmarks was discovered. The networks in that benchmark contain bilinear and softmax connections.…”

“…If no counterexamples are found (i.e., demonstrate that the property is SAT), then we utilize an iterative refinement approach using reachability analysis to verify the property (UNSAT). this consists of performing reachability analysis using a relax-approximation method [40], if not verified, then a less conservative approximation based on zonotope pre-filtering approach [35], and finally using the exact analysis when possible [36] until the specification is verified or there is a timeout. Link https://github.com/verivital/nnv Commit fb858b070d7c2fb1f036ac7fc374e1b9dfb5055e Hardware and licenses CPU, MATLAB license.…”

Work In Progress: Safety and Robustness Verification of Autoencoder-Based Regression Models using the NNV Tool

“…Many researchers are focused on developing new formal method based tools to address this vastly evolving field, some examples being satisfiability modulo theories (SMT) [16], polyhedron [35,44], mixed-integer linear program (MILP) [8], interval arithmetic [41,42], zonotope [30], input partition [46], linearization [43], abstract-domain [31] and star set based [36] methods. So far, several safety verification methodologies have been proposed for different neural network (NN) architectures, the focus mostly being on Feed-Forward Networks [7,10,36,46], Convolutional Neural networks [3,9,17,20,31,34], Semantic segmentation networks [4,12,19,22,25,37,50] and some on Recurrent Neural Networks [2,18].…”