Certified Training: Small Boxes are All You Need

Niklas, Müller, Mark; Eckert, Franziska; Fischer, M. L.; Vechev, Martin

doi:10.48550/arxiv.2210.04871

Cited by 4 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Orthogonally, the training of certifiably robust networks remains an open problem. Despite significant progress over recent years [6,19,35,37,43,45,65], networks trained specifically to exhibit provable robustness guarantees still suffer from severely degraded standard accuracy. Therefore, most benchmarks considered in the VNN-COMP are based on networks trained without consideration for later certification.…”

Section: Remaining Challengesmentioning

confidence: 99%

First three years of the international verification of neural networks competition (VNN-COMP)

Brix

Niklas

Bak

et al. 2023

Int J Softw Tools Technol Transfer

Self Cite

View full text Add to dashboard Cite

This paper presents a summary and meta-analysis of the first three iterations of the annual International Verification of Neural Networks Competition (VNN-COMP), held in 2020, 2021, and 2022. In the VNN-COMP, participants submit software tools that analyze whether given neural networks satisfy specifications describing their input-output behavior. These neural networks and specifications cover a variety of problem classes and tasks, corresponding to safety and robustness properties in image classification, neural control, reinforcement learning, and autonomous systems. We summarize the key processes, rules, and results, present trends observed over the last three years, and provide an outlook into possible future developments.

show abstract

Section: Remaining Challengesmentioning

confidence: 99%

First three years of the international verification of neural networks competition (VNN-COMP)

Brix

Niklas

Bak

et al. 2023

Int J Softw Tools Technol Transfer

Self Cite

View full text Add to dashboard Cite

show abstract

“…Accurate bounds can significantly reduce the complexity and computational effort required during the certification process, facilitating more efficient and dependable evaluations of the network's behavior in diverse and challenging scenarios. Moreover, computing such bounds has opened the door for a new set of "certified training" algorithms (Zhang et al 2022;Lyu et al 2021;Müller et al 2022b) where these bounds are used as a regularizer that penalizes the worst-case violation of robustness or fairness, which leads to training NNs with favorable properties. While computing such lower/upper bounds is crucial, current techniques in computing lower/upper bounds on the NN outputs are either computationally efficient but result in loose lower/upper bounds or compute tight bounds but are computationally expensive.…”

Section: Introductionmentioning

confidence: 99%

DeepBern-Nets: Taming the Complexity of Certifying Neural Networks Using Bernstein Polynomial Activations and Precise Bound Propagation

Khedr,

Shoukry

2024

AAAI

View full text Add to dashboard Cite

Formal certification of Neural Networks (NNs) is crucial for ensuring their safety, fairness, and robustness. Unfortunately, on the one hand, sound and complete certification algorithms of ReLU-based NNs do not scale to large-scale NNs. On the other hand, incomplete certification algorithms are easier to compute, but they result in loose bounds that deteriorate with the depth of NN, which diminishes their effectiveness. In this paper, we ask the following question; can we replace the ReLU activation function with one that opens the door to incomplete certification algorithms that are easy to compute but can produce tight bounds on the NN's outputs? We introduce DeepBern-Nets, a class of NNs with activation functions based on Bernstein polynomials instead of the commonly used ReLU activation. Bernstein polynomials are smooth and differentiable functions with desirable properties such as the so-called range enclosure and subdivision properties. We design a novel Interval Bound Propagation (IBP) algorithm, called Bern-IBP, to efficiently compute tight bounds on DeepBern-Nets outputs. Our approach leverages the properties of Bernstein polynomials to improve the tractability of neural network certification tasks while maintaining the accuracy of the trained networks. We conduct experiments in adversarial robustness and reachability analysis settings to assess the effectiveness of the approach. Our proposed framework achieves high certified accuracy for adversarially-trained NNs, which is often a challenging task for certifiers of ReLU-based NNs. This work establishes Bernstein polynomial activation as a promising alternative for improving NN certification tasks across various NNs applications.

show abstract

“…Algorithm timeout was set to 20s per instance.QNNs and might enhance QA-IBP-based training procedures as well. Moreover, further improvements may be feasible by adapting recent advances in IBP-based training methods for non-quantized neural networks(Müller et al 2022) to our quantized IBP variant.…”

mentioning

confidence: 99%

Quantization-Aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

Lechner

Žikelić

Chatterjee

et al. 2023

AAAI

View full text Add to dashboard Cite

We study the problem of training and certifying adversarially robust quantized neural networks (QNNs). Quantization is a technique for making neural networks more efficient by running them using low-bit integer arithmetic and is therefore commonly adopted in industry. Recent work has shown that floating-point neural networks that have been verified to be robust can become vulnerable to adversarial attacks after quantization, and certification of the quantized representation is necessary to guarantee robustness. In this work, we present quantization-aware interval bound propagation (QA-IBP), a novel method for training robust QNNs. Inspired by advances in robust learning of non-quantized networks, our training algorithm computes the gradient of an abstract representation of the actual network. Unlike existing approaches, our method can handle the discrete semantics of QNNs. Based on QA-IBP, we also develop a complete verification procedure for verifying the adversarial robustness of QNNs, which is guaranteed to terminate and produce a correct answer. Compared to existing approaches, the key advantage of our verification procedure is that it runs entirely on GPU or other accelerator devices. We demonstrate experimentally that our approach significantly outperforms existing methods and establish the new state-of-the-art for training and certifying the robustness of QNNs.

show abstract

Certified Training: Small Boxes are All You Need

Cited by 4 publications

References 0 publications

First three years of the international verification of neural networks competition (VNN-COMP)

First three years of the international verification of neural networks competition (VNN-COMP)

DeepBern-Nets: Taming the Complexity of Certifying Neural Networks Using Bernstein Polynomial Activations and Precise Bound Propagation

Quantization-Aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

Contact Info

Product

Resources

About