Toward Scalable Verification for Safety-Critical Deep Networks

Kuper, Lindsey; Katz, Guy; Gottschlich, Justin; Julian, Kyle D.; Barrett, Clark; Kochenderfer, Mykel J.

doi:10.48550/arxiv.1801.05950

Cited by 5 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hardware Intrusion: HW intrusion means that the attacker inserts malware or trojan (typically in the form of circuitry modification) in the processing HW for performing attacks such as confidence reduction and misclassification. The potential countermeasures are the typical HW security techniques, like the built-in self-test (BIST) to verify the functionality of the processing HW, the side channel analysis-based monitoring [78]- [80] to detect and identify anomalous side channel signals, the formal method analysis to quickly and comprehensively analyze the behavior of the processing HW (e.g., using property checker [78], mathematical model [81], SAT solver [82], and SMT solver [83]).…”

Section: B Secure Ml: Attacks and Defensesmentioning

confidence: 99%

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

Shafique,

Marchisio,

Putra

et al. 2021

Preprint

View full text Add to dashboard Cite

The security and privacy concerns along with the amount of data that is required to be processed on regular basis has pushed processing to the edge of the computing systems. Deploying advanced Neural Networks (NN), such as deep neural networks (DNNs) and spiking neural networks (SNNs), that offer state-of-the-art results on resourceconstrained edge devices is challenging due to the stringent memory and power/energy constraints. Moreover, these systems are required to maintain correct functionality under diverse security and reliability threats. This paper first discusses existing approaches to address energy efficiency, reliability, and security issues at different system layers, i.e., hardware (HW) and software (SW). Afterward, we discuss how to further improve the performance (latency) and the energy efficiency of Edge AI systems through HW/SW-level optimizations, such as pruning, quantization, and approximation. To address reliability threats (like permanent and transient faults), we highlight cost-effective mitigation techniques, like fault-aware training and mapping. Moreover, we briefly discuss effective detection and protection techniques to address security threats (like model and data corruption). Towards the end, we discuss how these techniques can be combined in an integrated cross-layer framework for realizing robust and energy-efficient Edge AI systems.

show abstract

Section: B Secure Ml: Attacks and Defensesmentioning

confidence: 99%

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

Shafique,

Marchisio,

Putra

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Thus, the tests generated by OGMA can explore these large input space, potentially discovering more errors when compared to limiting the test generation via the training data. In contrast to previous works, OGMA is not limited to test specific applications [29] or properties [19], [41]. OGMA works completely blackbox and can be easily adapted to test real-world classification systems for a variety of different applications.…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

“…In recent years, the software engineering research community have stepped up to develop testing methodologies for deeplearning systems [19], [29], [36], [40]. These works, however are, limited either to specific applications [29] or rely on the presence of sample inputs [36], [44]. Moreover, none of the prior works are applicable to generate grammar-based inputs in a fashion that such inputs steer the execution of machine-learning models to erroneous behaviour.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Grammar Based Directed Testing of Machine Learning Systems

Udeshi

Chattopadhyay

2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

The massive progress of machine learning has seen its application over a variety of domains in the past decade. But how do we develop a systematic, scalable and modular strategy to validate machine-learning systems? We present, to the best of our knowledge, the first approach, which provides a systematic test framework for machine-learning systems that accepts grammar-based inputs. Our OGMA approach automatically discovers erroneous behaviours in classifiers and leverages these erroneous behaviours to improve the respective models. OGMA leverages inherent robustness properties present in any well trained machine-learning model to direct test generation and thus, implementing a scalable test generation methodology. To evaluate our OGMA approach, we have tested it on three real world natural language processing (NLP) classifiers. We have found thousands of erroneous behaviours in these systems. We also compare OGMA with a random test generation approach and observe that OGMA is more effective than such random test generation by up to 489%. ! 1. God of language from Irish mythology and Scottish mythology

show abstract

“…Applications of neural networks to safety-critical domains requires ensuring that they behave as expected under all circumstances [32]. One way to achieve this is to ensure that neural networks conform with a list of specifications, i.e., relationships between the inputs and outputs of a neural network that ought to be satisfied.…”

Section: Introductionmentioning

confidence: 99%

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Dathathri¹,

Dvijotham²,

Kurakin³

et al. 2020

Preprint

View full text Add to dashboard Cite

Convex relaxations have emerged as a promising approach for verifying desirable properties of neural networks like robustness to adversarial perturbations. Widely used Linear Programming (LP) relaxations only work well when networks are trained to facilitate verification. This precludes applications that involve verificationagnostic networks, i.e., networks not specially trained for verification. On the other hand, semidefinite programming (SDP) relaxations have successfully be applied to verification-agnostic networks, but do not currently scale beyond small networks due to poor time and space asymptotics. In this work, we propose a first-order dual SDP algorithm that (1) requires memory only linear in the total number of network activations, (2) only requires a fixed number of forward/backward passes through the network per iteration. By exploiting iterative eigenvector methods, we express all solver operations in terms of forward and backward passes through the network, enabling efficient use of hardware like GPUs/TPUs. For two verification-agnostic networks on MNIST and CIFAR-10, we significantly improve 8 verified robust accuracy from 1% Ñ 88% and 6% Ñ 40% respectively. We also demonstrate tight verification of a quadratic stability specification for the decoder of a variational autoencoder. ˚Equal contribution. Alphabetical order.: Code available at https://github.com/deepmind/jax_verify.

show abstract

Toward Scalable Verification for Safety-Critical Deep Networks

Cited by 5 publications

References 0 publications

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework

Grammar Based Directed Testing of Machine Learning Systems

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Contact Info

Product

Resources

About