SyReNN: A Tool for Analyzing Deep Neural Networks

Abstract

Deep Neural Networks (DNNs) are rapidly gaining popularity in a variety of important domains. Formally, DNNs are complicated vector-valued functions which come in a variety of sizes and applications. Unfortunately, modern DNNs have been shown to be vulnerable to a variety of attacks and buggy behavior. This has motivated recent work in formally analyzing the properties of such DNNs. This paper introduces SyReNN, a tool for understanding and analyzing a DNN by computing its symbolic representation. The key insight is to decompose the DNN into linear functions. Our tool is designed for analyses using low-dimensional subsets of the input space, a unique design point in the space of DNN analysis tools. We describe the tool and the underlying theory, then evaluate its use and performance on three case studies: computing Integrated Gradients, visualizing a DNN’s decision boundaries, and patching a DNN.

Provable Repair of Deep Neural Networks

“…In practice, we can quickly compute 𝐿𝑖𝑛𝑅𝑒𝑔𝑖𝑜𝑛𝑠 (𝑁 , 𝑃) for either large 𝑁 with one-dimensional 𝑃 or medium-sized 𝑁 with two-dimensional 𝑃. We use the algorithm of Sotoudeh and Thakur [55] for computing 𝐿𝑖𝑛𝑅𝑒𝑔𝑖𝑜𝑛𝑠 (𝑁 , 𝑃) when 𝑃 is one-or two-dimensional. Linear programs can be solved in polynomial time [39], and many efficient, industrial-grade LP solvers such as the Gurobi solver [25] exist.…”

“…Although in the worst case there are exponentiallymany such linear regions, theoretical results indicate that, for an 𝑛-dimensional polytope 𝑃 and network 𝑁 with 𝑚 nodes, we expect |𝐿𝑖𝑛𝑅𝑒𝑔𝑖𝑜𝑛𝑠 (𝑁 , 𝑃)| = 𝑂 (𝑚 𝑛 ) [26,27]. Sotoudeh and Thakur [55] show efficient computation of one-and two-dimensional 𝐿𝑖𝑛𝑅𝑒𝑔𝑖𝑜𝑛𝑠 for real-world networks.…”

“…The main insight for solving Provable Polytope Repair is that, for piecewise-linear DDNNs, repairing polytopes (with infinitely many points) is equivalent to Provable Point Repair on finitely-many key points. These key points can be computed for DDNNs using prior work on computing symbolic representations of DNNs [54,55]. This reduction is intuitively similar to how the simplex algorithm reduces optimizing over a polytope with infinitely many points to optimizing over the finitely-many vertex points.…”

Permutation Invariance of Deep Neural Networks with ReLUs

“…Several methods can be used to construct an over-approximation that make different tradeoffs between precision and efficiency. One can construct the smallest affine region (or polytope) that includes all the reachable values possible across the Relu [17]. Computing the smallest region can be inefficient as it is an optimization problem requiring several expensive LP or convex-hull calls.…”

“…Note that the forward propagation may also be done using convex polytope propagation (which is how it is usually done, e.g. [17]), but it requires computing the convex hull each time, which is an expensive operation. In contrast, tie-class analysis helps us propagate the affine regions efficiently.…”