Concolic testing for deep neural networks

Sun, Youcheng; Wu, Min; Ruan, Wenjie; Huang, Xiaowei; Kwiatkowska, Marta; Kroening, Daniel

doi:10.1145/3238147.3238172

Cited by 314 publications

(256 citation statements)

References 73 publications

(179 reference statements)

Supporting

Mentioning

255

Contrasting

Order By: Relevance

“…Sun et al [104] presented DeepConcolic, a dynamic symbolic execution testing method for DNNs. Concrete execution is used to direct the symbolic analysis to particular MC/DC criteria' condition, through concretely evaluating given properties of the ML models.…”

Section: Symbolic Execution Based Test Input Generationmentioning

confidence: 99%

Machine Learning Testing: Survey, Landscapes and Horizons

Zhang

Harman

Ма

et al. 2022

IIEEE Trans. Software Eng.

503

318

View full text Add to dashboard Cite

This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research. It covers 128 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation). The paper also analyses trends concerning datasets, research trends, and research focus, concluding with research challenges and promising research directions in ML testing. Index Terms-machine learning, software testing, deep neural network, ! • Jie M. Zhang and Mark Harman are with CREST, University College London, United Kingdom. Mark Harman is also with Facebook London.

show abstract

Section: Symbolic Execution Based Test Input Generationmentioning

confidence: 99%

Machine Learning Testing: Survey, Landscapes and Horizons

Zhang

Harman

Ма

et al. 2022

IIEEE Trans. Software Eng.

503

318

View full text Add to dashboard Cite

show abstract

“…Robustness verification is also considered in [8,21]. Systematic testing techniques such as [43,45,48] are designed to automatically generate test cases to increase a coverage metric, i.e., explore different parts of neural network architecture by generating test inputs that maximize the number of activated neurons. These approaches ignore effects induced by the actual environment in which the network is deployed.…”

Section: Related Workmentioning

confidence: 99%

An inductive synthesis framework for verifiable reinforcement learning

Zhu

Xiong

Magill

et al. 2019

Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

Despite the tremendous advances that have been made in the last decade on developing useful machine-learning applications, their wider adoption has been hindered by the lack of strong assurance guarantees that can be made about their behavior. In this paper, we consider how formal verification techniques developed for traditional software systems can be repurposed for verification of reinforcement learningenabled ones, a particularly important class of machine learning systems. Rather than enforcing safety by examining and altering the structure of a complex neural network implementation, our technique uses blackbox methods to synthesizes deterministic programs, simpler, more interpretable, approximations of the network that can nonetheless guarantee desired safety properties are preserved, even when the network is deployed in unanticipated or previously unobserved environments. Our methodology frames the problem of neural network verification in terms of a counterexample and syntax-guided inductive synthesis procedure over these programs. The synthesis procedure searches for both a deterministic program and an inductive invariant over an infinite state transition system that represents a specification of an application's control logic. Additional specifications defining environment-based constraints can also be provided to further refine the search space. Synthesized programs deployed in conjunction with a neural network implementation dynamically enforce safety conditions by monitoring and preventing potentially unsafe actions proposed by neural policies. Experimental results over a wide range of cyberphysical applications demonstrate that software-inspired formal verification techniques can be used to realize trustworthy reinforcement learning systems with low overhead.

show abstract

“…In addition, many testing approaches for traditional software have also been adopted and applied to testing DL systems, such as statement, branch, condition and MC/DC coverage [15]. Furthermore, various forms of neuron coverage [16] have been defined, and are demonstrated as important metrics to guide test generation.…”

Section: Testing Of DL Systemsmentioning

confidence: 99%

“…Obviously it is unthinkable to exhaustively test every feasible input of the DL systems. Recently, an increasing number of researchers have contributed to testing DL systems with a variety of approaches [10], [13], [14], [15], [16]. The main idea of these approaches is to enhance input examples of test data set by different techniques.…”

Section: Introductionmentioning

confidence: 99%

Condition-Guided Adversarial Generative Testing for Deep Learning Systems

Zhang

Dai

2019

2019 IEEE International Conference on Artificial Intelligence Testing (AITest)

View full text Add to dashboard Cite

With the emerging of big data technology, Deep Learning systems (DL) based on Deep Neural Networks (DNNs) are more and more used in various aspects of our life, including unmanned vehicles, speech processing, and robotics. However, due to the limited dataset and the dependence on manual labeling data, DNNs often fail to detect their erroneous behaviors, which may lead to serious problems. Several approaches have been proposed to enhance the input examples for testing DL systems. However, they have the following limitations. First, most of them do not consider the influence of small perturbations on input examples. Some approaches take into account the perturbations, however, they design and generate adversarial examples from the perspective of model. These examples generated from a specific model may cause low generalization ability when they are applied to other models. Second, they only use surface feature constraints (e.g. pixel-level constraints) to judge the difference between the adversarial example generated and the original example. The deep feature constraints, which contain high-level semantic information, such as image object category and scene semantics are completely neglected. To address these two problems, in this paper, we propose CAGFuzz, a Coverage-guided Adversarial Generative Fuzzing testing approach, which generates adversarial examples for a targeted DNN to discover its potential defects. First, we train an adversarial case generator (AEG) from the perspective of general data set. AEG only considers the data characteristics, and avoids low generalization ability when it is testing other models. Second, we extract the depth features of the original and adversarial examples, and constrain the adversarial examples by cosine similarity to ensure that the semantic information of adversarial examples remains unchanged. Finally, we retrain effective adversarial examples to improve neuron testing coverage rate. Based on several popular data sets, we design a set of dedicated experiments to evaluate CAGFuzz. The experimental results show that CAGFuzz can improve the generalization ability, the neuron coverage rate, detect hidden errors, and also improve the accuracy of the target DNN.

show abstract

Concolic testing for deep neural networks

Cited by 314 publications

References 73 publications

Machine Learning Testing: Survey, Landscapes and Horizons

Machine Learning Testing: Survey, Landscapes and Horizons

An inductive synthesis framework for verifiable reinforcement learning

Condition-Guided Adversarial Generative Testing for Deep Learning Systems

Contact Info

Product

Resources

About