Fastened CROWN: Tightened Neural Network Robustness Certificates

Zhaoyang, Lyu,; Ko, Ching-Yun; Kong, Zhifeng; Wong, Ngai; Lin, Dahua; Daniel, Luca

doi:10.48550/arxiv.1912.00574

Cited by 4 publications

(14 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, exact verifiers are typically based on solving NP-hard optimization problems [19] which can significantly limit their scalability. In contrast, relaxed verifiers are often based on polynomially-solvable optimization problems such as convex optimization or linear programming (LP) [2,14,22,24,28,30,32,45,48], which in turn lend themselves to faster propagation-based methods where bounds are computed by a series of variable substitutions in a backwards pass through the network [34,42,43,44,47]. Unfortunately, relaxed verifiers achieve this speed and scalability by trading off effectiveness (i.e.…”

Section: Introductionmentioning

confidence: 99%

“…Combining this with (25) gives the inequality ℓp Īq ă whp Ȗh ´Lhq. Therefore, vh P r0, 1q, and hence vh is feasible for (24). In addition, for any pI, hq P J we have that…”

mentioning

confidence: 97%

“…To prove ωpxq ě υpxq we will show that, through Φ, the greedy procedure to solve (7) described in the main text just before the statement of Proposition 2, becomes the standard greedy procedure for (24) and hence also yields an optimal basic feasible solution to (24). For simplicity, assume without loss of generality that we have re-ordered the indices in n so that…”

mentioning

confidence: 99%

“…Then the greedy procedure that incrementally grows I terminates with some pI, hq P J where I " h ´1 . Then v " Φ pp h ´1 , hqq is a basic feasible solution for (24) with the same objective value as the objective value of pI, hq for (7). To conclude that ωpxq ě υpxq, we claim that v is an optimal solution for (24) since the standard greedy procedure for ( 24) is known to generate the optimal solution for this problem.…”

mentioning

confidence: 99%

See 3 more Smart Citations

The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification

Tjandraatmadja,

Anderson,

Huchette

et al. 2020

Preprint

View full text Add to dashboard Cite

We improve the effectiveness of propagation-and linear-optimization-based neural network verification algorithms with a new tightened convex relaxation for ReLU neurons. Unlike previous single-neuron relaxations which focus only on the univariate input space of the ReLU, our method considers the multivariate input space of the affine pre-activation function preceding the ReLU. Using results from submodularity and convex geometry, we derive an explicit description of the tightest possible convex relaxation when this multivariate input is over a box domain. We show that our convex relaxation is significantly stronger than the commonly used univariate-input relaxation which has been proposed as a natural convex relaxation barrier for verification. While our description of the relaxation may require an exponential number of inequalities, we show that they can be separated in linear time and hence can be efficiently incorporated into optimization algorithms on an as-needed basis. Based on this novel relaxation, we design two polynomial-time algorithms for neural network verification: a linearprogramming-based algorithm that leverages the full power of our relaxation, and a fast propagation algorithm that generalizes existing approaches. In both cases, we show that for a modest increase in computational effort, our strengthened relaxation enables us to verify a significantly larger number of instances compared to similar algorithms.Preprint. Under review.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Combining this with (25) gives the inequality ℓp Īq ă whp Ȗh ´Lhq. Therefore, vh P r0, 1q, and hence vh is feasible for (24). In addition, for any pI, hq P J we have that…”

mentioning

confidence: 97%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification

Tjandraatmadja,

Anderson,

Huchette

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The main drawback of this strategy is that the image looks unnatural. With the rapid development of deep learning, many computer vision tasks have been significantly advanced, such as image classification [25], face recognition [1], visual question answering [4,5,6], robust design [16], and image style conversion [7]. Imaging is also irreversible and in this process of converting raw data to JPEG, there is a huge information loss, which inspired researchers to substitute data-driven-based method for traditional image signal processing(ISP) algorithms.…”

Section: Introductionmentioning

confidence: 99%

Extreme Low-Light Imaging with Multi-granulation Cooperative Networks

Wang,

Gao,

Hoi

et al. 2020

Preprint

View full text Add to dashboard Cite

Low-light imaging is challenging since images may appear to be dark and noised due to low signal-to-noise ratio, complex image content, and the variety in shooting scenes in extreme low-light condition. Many methods have been proposed to enhance the imaging quality under extreme low-light conditions, but it remains difficult to obtain satisfactory results, especially when they attempt to retain high dynamic range (HDR). In this paper, we propose a novel method of multi-granulation cooperative networks (MCN) with bidirectional information flow to enhance extreme lowlight images, and design an illumination map estimation function (IMEF) to preserve high dynamic range (HDR).To facilitate this research, we also contribute to create a new benchmark dataset of real-world Dark High Dynamic Range (DHDR) images to evaluate the performance of high dynamic preservation in low light environment. Experimental results show that the proposed method outperforms the state-of-the-art approaches in terms of both visual effects and quantitative analysis.

show abstract

DiffRNN: Differential Verification of Recurrent Neural Networks

Mohammadinejad

Paulsen

Deshmukh

et al. 2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Recurrent neural networks (RNNs) such as Long Short Term Memory (LSTM) networks have become popular in a variety of applications such as image processing, data classification, speech recognition, and as controllers in autonomous systems. In practical settings, there is often a need to deploy such RNNs on resource-constrained platforms such as mobile phones or embedded devices. As the memory footprint and energy consumption of such components become a bottleneck, there is interest in compressing and optimizing such networks using a range of heuristic techniques. However, these techniques do not guarantee the safety of the optimized network, e.g., against adversarial inputs, or equivalence of the optimized and original networks. To address this problem, we propose DIFFRNN, the first differential verification method for RNNs to certify the equivalence of two structurally similar neural networks. Existing work on differential verification for ReLU-based feed-forward neural networks does not apply to RNNs where nonlinear activation functions such as Sigmoid and Tanh cannot be avoided. RNNs also pose unique challenges such as handling sequential inputs, complex feedback structures, and interactions between the gates and states. In DIFFRNN, we overcome these challenges by bounding nonlinear activation functions with linear constraints and then solving constrained optimization problems to compute tight bounding boxes on non-linear surfaces in a high-dimensional space. The soundness of these bounding boxes is then proved using the dReal SMT solver. We demonstrate the practical efficacy of our technique on a variety of benchmarks and show that DIFFRNN outperforms state-of-the-art RNN verification tools such as POPQORN.

show abstract

Fastened CROWN: Tightened Neural Network Robustness Certificates

Cited by 4 publications

References 1 publication

The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification

The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification

Extreme Low-Light Imaging with Multi-granulation Cooperative Networks

DiffRNN: Differential Verification of Recurrent Neural Networks

Contact Info

Product

Resources

About