DeepFault: Fault Localization for Deep Neural Networks

Enişer, Hasan Ferit; Gerasimou, Simos; Şen, Alper

doi:10.1007/978-3-030-16722-6_10

Cited by 53 publications

(33 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MLSs. Six of them address the problems of studying and defining the spectrum of bugs in MLSs, and automating the debugging of MLSs (Cheng et al 2018a;Zhang et al 2018a;Ma et al 2018c;Odena et al 2019;Dwarakanath et al 2018;Eniser et al 2019). Concerning the former, two studies in our pool present an empirical study on the bugs affecting MLSs (Cheng et al 2018a;Zhang et al 2018a).…”

Section: Faults and Debugging Eight Work Considered In Our Mapping Amentioning

confidence: 99%

“…Regarding debugging automation, four studies address the problem of debugging an MLS (Ma et al 2018c;Odena et al 2019), or localising the faults within an MLS (Dwarakanath et al 2018;Eniser et al 2019). The challenge in this case is to unroll the hidden decision-making policy of the ML model, which is driven by the data it is fed with.…”

Section: Faults and Debugging Eight Work Considered In Our Mapping Amentioning

confidence: 99%

“…Five works (7%) manipulate only the input data, i.e., they perform input level testing (Bolte et al 2019;Byun et al 2019;Henriksson et al 2019;Wolschke et al 2018). The majority of the papers (64%) operate at the ML model level (model level testing) (Cheng et al 2018a;Ding et al 2017;Du et al 2019;Dwarakanath et al 2018;Eniser et al 2019;Gopinath et al 2018;Groce et al 2014;Guo et al 2018;Kim et al 2019;Li et al 2018;Ma et al 2018bMa et al , c, d, 2019Murphy et al 2007aMurphy et al , b, 2008Murphy et al , b, 2009Nakajima and Bui 2016, 2019Odena et al 2019;Patel et al 2018;Pei et al 2017;Qin et al 2018;Saha and Kanewala 2019;Sekhon and Fleming 2019;Shen et al 2018;Shi et al 2019;Spieker and Gotlieb 2019;Strickland et al 2018;Sun et al 2018a, b;Tian et al 2018;Udeshi and Chattopadhyay 2019;Udeshi et al 2018;Uesato et al 2019;Xie et al 2018Xie et al , 2019Xie et al , 2011Zhang et al 2018aZhang et al , b, 2019Zhao a...…”

Section: Cost Of Testingmentioning

confidence: 99%

“…Nearly half of the analysed papers (56%) propose and evaluate a technique which is domain-agnostic, Fig. 7 Testing Levels i.e., in principle it may be applicable to any MLS (Aniculaesei et al 2018;Byun et al 2019;Cheng et al 2018a, b;Du et al 2019;Eniser et al 2019;Guo et al 2018;Henriksson et al 2019;Kim et al 2019;Li et al 2018;Ma et al 2018bMa et al , c, d, 2019Murphy et al 2007aMurphy et al , b, 2008Murphy et al , b, 2009Nakajima and Bui 2016, 2019Odena et al 2019;Pei et al 2017;Saha and Kanewala 2019;Sekhon and Fleming 2019;Shen et al 2018;Shi et al 2019;Sun et al 2018a, b;Tian et al 2018;Udeshi and Chattopadhyay 2019;Uesato et al 2019;Xie et al 2018Xie et al , 2019Xie et al , 2011Zhang et al 2018aZhang et al , 2019Zhao and Gao 2018). Around 30% proposed approaches are designed for autonomous systems (Abeysirigoonawardena et al 2019;Beglerovic et al 2017;Bühler and Wegener 2004;Klueck et al 2018;Li et al 2016;Mullins et al 2018;de Oliveira Neves et al 2016;Patel et al 2018;Strickland et al 2018;…”

Section: Domains (Rq 13)mentioning

confidence: 99%

“…Nearly 60% (33 out of 55) of the papers present various methods to generate test inputs for the MLS Fig. 10 Test Artefacts under test (Abdessalem et al 2016(Abdessalem et al , 2018aAbeysirigoonawardena et al 2019;Beglerovic et al 2017;Bühler and Wegener 2004;Du et al 2019;Eniser et al 2019;Fremont et al 2019;Guo et al 2018;Klueck et al 2018;Li et al 2016;Ma et al 2018cMa et al , 2019Majumdar et al 2019;Murphy et al 2007a, b;de Oliveira Neves et al 2016;Odena et al 2019;Pei et al 2017;Sekhon and Fleming 2019;Sun et al 2018a, b;Tian et al 2018;Tuncali et al 2018Tuncali et al , 2019Udeshi et al 2018Udeshi et al , 2019Wolschke et al 2017Wolschke et al , 2018Xie et al 2019;Zhang et al 2018b;Zheng et al 2018b). However, what a test input represents differs across the proposed approaches and highly depends on the domain of the tested system.…”

Section: Test Artefacts (Rq 21)mentioning

confidence: 99%

See 4 more Smart Citations

Testing machine learning based systems: a systematic mapping

et al. 2020

View full text Add to dashboard Cite

Context: A Machine Learning based System (MLS) is a software system including one or more components that learn how to perform a task from a given data set. The increasing adoption of MLSs in safety critical domains such as autonomous driving, healthcare, and finance has fostered much attention towards the quality assurance of such systems. Despite the advances in software testing, MLSs bring novel and unprecedented challenges, since their behaviour is defined jointly by the code that implements them and the data used for training them. Objective: To identify the existing solutions for functional testing of MLSs, and classify them from three different perspectives: (1) the context of the problem they address, (2) their features, and (3) their empirical evaluation. To report demographic information about the ongoing research. To identify open challenges for future research. Method: We conducted a systematic mapping study about testing techniques for MLSs driven by 33 research questions. We followed existing guidelines when defining our research protocol so as to increase the repeatability and reliability of our results. Results: We identified 70 relevant primary studies, mostly published in the last years. We identified 11 problems addressed in the literature. We investigated multiple aspects of the testing approaches, such as the used/proposed adequacy criteria, the algorithms for test input generation, and the test oracles. Conclusions: The most active research areas in MLS testing address automated scenario/input generation and test oracle creation. MLS testing is a rapidly growing and developing research area, with many open challenges, such as the generation of realistic inputs and the definition of reliable evaluation metrics and benchmarks.

show abstract

Section: Faults and Debugging Eight Work Considered In Our Mapping Amentioning

confidence: 99%

Section: Faults and Debugging Eight Work Considered In Our Mapping Amentioning

confidence: 99%

Section: Cost Of Testingmentioning

confidence: 99%

Section: Domains (Rq 13)mentioning

confidence: 99%

Section: Test Artefacts (Rq 21)mentioning

confidence: 99%

See 3 more Smart Citations

Testing machine learning based systems: a systematic mapping

et al. 2020

View full text Add to dashboard Cite

show abstract

A Novel Approach of Software Fault Prediction Using Deep Learning Technique

Ghosh

Singh

2020

Learning and Analytics in Intelligent Systems

View full text Add to dashboard Cite

Understanding Local Robustness of Deep Neural Networks under Natural Variations

Zhong

Tian

Ray

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) are being deployed in a wide range of settings today, from safety-critical applications like autonomous driving to commercial applications involving image classifications. However, recent research has shown that DNNs can be brittle to even slight variations of the input data. Therefore, rigorous testing of DNNs has gained widespread attention.While DNN robustness under norm-bound perturbation got significant attention over the past few years, our knowledge is still limited when natural variants of the input images come. These natural variants, e.g., a rotated or a rainy version of the original input, are especially concerning as they can occur naturally in the field without any active adversary and may lead to undesirable consequences. Thus, it is important to identify the inputs whose small variations may lead to erroneous DNN behaviors. The very few studies that looked at DNN’s robustness under natural variants, however, focus on estimating the overall robustness of DNNs across all the test data rather than localizing such error-producing points. This work aims to bridge this gap.To this end, we study the local per-input robustness properties of the DNNs and leverage those properties to build a white-box (DeepRobust-W) and a black-box (DeepRobust-B) tool to automatically identify the non-robust points. Our evaluation of these methods on three DNN models spanning three widely used image classification datasets shows that they are effective in flagging points of poor robustness. In particular, DeepRobust-W and DeepRobust-B are able to achieve an F1 score of up to 91.4% and 99.1%, respectively. We further show that DeepRobust-W can be applied to a regression problem in a domain beyond image classification. Our evaluation on three self-driving car models demonstrates that DeepRobust-W is effective in identifying points of poor robustness with F1 score up to 78.9%.

show abstract

DeepFault: Fault Localization for Deep Neural Networks

Cited by 53 publications

References 53 publications

Testing machine learning based systems: a systematic mapping

Testing machine learning based systems: a systematic mapping

A Novel Approach of Software Fault Prediction Using Deep Learning Technique

Understanding Local Robustness of Deep Neural Networks under Natural Variations

Contact Info

Product

Resources

About