DeepConcolic: Testing and Debugging Deep Neural Networks

Sun, Youcheng; Huang, Xiaowei; Kroening, Daniel; Sharp, James; Hill, Matthew Q.; Ashmore, Rob

doi:10.1109/icse-companion.2019.00051

Cited by 53 publications

(23 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [38], the difference between test dataset and training dataset is measured by quantifying the difference between DNNs' activation patterns. Some traditional test case generation techniques such as concolic testing [18,39], symbolic execution [40] and fuzzing [24,41] have been recently extended to DNNs. Mutation testing has similarly been investigated in [42,43,44,45,46].…”

Section: Generation Of Adversarial Examples For Dnnsmentioning

confidence: 99%

Concolic testing for deep neural networks

Sun

Ruan

et al. 2018

Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

Self Cite

294

248

View full text Add to dashboard Cite

Concolic testing combines program execution and symbolic analysis to explore the execution paths of a software program. This paper presents the first concolic testing approach for Deep Neural Networks (DNNs). More specifically, we formalise coverage criteria for DNNs that have been studied in the literature, and then develop a coherent method for performing concolic testing to increase test coverage. Our experimental results show the effectiveness of the concolic testing approach in both achieving high coverage and finding adversarial examples.

show abstract

Section: Generation Of Adversarial Examples For Dnnsmentioning

confidence: 99%

Concolic testing for deep neural networks

Sun

Ruan

et al. 2018

Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

Self Cite

294

248

View full text Add to dashboard Cite

show abstract

“…For example [70] applies Satisfiability Modulo Theories (SMT) solvers to find adversarial examples for NNs used in image analysis. Further, the ideas of concolic testing, which seeks to maximise code coverage, have been applied to deep NNs [71]. This work addresses structural coverage, including neuron coverage, and other properties such as Lipschitz continuity.…”

Section: Discussionmentioning

confidence: 99%

The Role of Explainability in Assuring Safety of Machine Learning in Healthcare

Jia¹,

McDermid²,

Lawton³

et al. 2021

Preprint

View full text Add to dashboard Cite

Established approaches to assuring safety-critical systems and software are difficult to apply to systems employing machine learning (ML). In many cases, ML is used on ill-defined problems, e.g. optimising sepsis treatment, where there is no clear, pre-defined specification against which to assess validity. This problem is exacerbated by the "opaque" nature of ML where the learnt model is not amenable to human scrutiny. Explainable AI methods have been proposed to tackle this issue by producing human-interpretable representations of ML models which can help users to gain confidence and build trust in the ML system. However, there is not much work explicitly investigating the role of explainability for safety assurance in the context of ML development. This paper identifies ways in which explainable AI methods can contribute to safety assurance of ML-based systems. It then uses a concrete ML-based clinical decision support system, concerning weaning of patients from mechanical ventilation, to demonstrate how explainable AI methods can be employed to produce evidence to support safety assurance. The results are also represented in a safety argument to show where, and in what way, explainable AI methods can contribute to a safety case. Overall, we conclude that explainable AI methods have a valuable role in safety assurance of ML-based systems in healthcare but that they are not sufficient in themselves to assure safety.

show abstract

“…Similarly, DeepGauge [51] introduces a family of adequacy criteria based on a more detailed analysis of neuron activation values. DeepCT [50] proposes a combinatorial testing approach, while DeepCover [69] adapts MC/DC from traditional software testing and defines adequacy criteria that investigate the changes of successive pairs of layers. Recent research also proposes testing criteria and techniques driven by symbolic execution [31], coverage guided fuzzing [56,76] and metamorphic transformations [72], while other research explores test prioritization [16] and fault localisation [24].…”

Section: Related Workmentioning

confidence: 99%

Importance-driven deep learning system testing

Gerasimou

Enişer

Şen

et al. 2020

Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Companion Proceedings

View full text Add to dashboard Cite

Deep Learning (DL) systems are key enablers for engineering intelligent applications due to their ability to solve complex tasks such as image recognition and machine translation. Nevertheless, using DL systems in safety-and security-critical applications requires to provide testing evidence for their dependable operation. Recent research in this direction focuses on adapting testing criteria from traditional software engineering as a means of increasing confidence for their correct behaviour. However, they are inadequate in capturing the intrinsic properties exhibited by these systems. We bridge this gap by introducing DeepImportance, a systematic testing methodology accompanied by an Importance-Driven (IDC) test adequacy criterion for DL systems. Applying IDC enables to establish a layer-wise functional understanding of the importance of DL system components and use this information to guide the generation of semantically-diverse test sets. Our empirical evaluation on several DL systems, across multiple DL datasets and with state-of-the-art adversarial generation techniques demonstrates the usefulness and effectiveness of DeepImportance and its ability to guide the engineering of more robust DL systems.

show abstract

DeepConcolic: Testing and Debugging Deep Neural Networks

Cited by 53 publications

References 11 publications

Concolic testing for deep neural networks

Concolic testing for deep neural networks

The Role of Explainability in Assuring Safety of Machine Learning in Healthcare

Importance-driven deep learning system testing

Contact Info

Product

Resources

About