Self-Checking Deep Neural Networks for Anomalies and Adversaries in Deployment

Xiao, Yan; Beschastnikh, Ivan; Lin, Yun; Hundal, Rajdeep Singh; Xie, Xiaofei; Sun, Changsheng; Dong, Jin Song

doi:10.1109/tdsc.2022.3200421

Cited by 6 publications

(2 citation statements)

References 39 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SelfChecker operates through a layer-based approach, which necessitates white-box access and may have limited capabilities in detecting issues in shallow DNNs with a few layers. SelfChecker++ [71] has been designed to target both unintended abnormal test data and intended adversarial samples. InputReflector [73], introduced a runtime approach to identify and fix failure-inducing inputs in DL systems inspired by traditional input-debugging techniques.…”

Section: Related Workmentioning

confidence: 99%

Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment

Ahmed,

Gao,

Rajan

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

Deep learning models are trained with certain assumptions about the data during the development stage and then used for prediction in the deployment stage. It is important to reason about the trustworthiness of the model's predictions with unseen data during deployment. Existing methods for specifying and verifying traditional software are insufficient for this task, as they cannot handle the complexity of DNN model architecture and expected outcomes. In this work, we propose a novel technique that uses rules derived from neural network computations to infer data preconditions for a DNN model to determine the trustworthiness of its predictions. Our approach, DeepInfer involves introducing a novel abstraction for a trained DNN model that enables weakest precondition reasoning using Dijkstra's Predicate Transformer Semantics. By deriving rules over the inductive type of neural network abstract representation, we can overcome the matrix dimensionality issues that arise from the backward non-linear computation from the output layer to the input layer. We utilize the weakest precondition computation using rules of each kind of activation function to compute layer-wise precondition from the given postcondition on the final output of a deep neural network. We extensively evaluated DeepInfer on 29 realworld DNN models using four different datasets collected from five different sources and demonstrated the utility, effectiveness, and performance improvement over closely related work. DeepInfer efficiently detects correct and incorrect predictions of high-accuracy models with high recall (0.98) and high F-1 score (0.84) and has significantly improved over the prior technique, SelfChecker. The average runtime overhead of DeepInfer is low, 0.22 sec for all the unseen datasets. We also compared runtime overhead using the same hardware settings and found that DeepInfer is 3.27 times faster than SelfChecker, the state-of-the-art in this area. CCS CONCEPTS• Software and its engineering → Specification languages; • Computing methodologies → Machine learning.

show abstract

Section: Related Workmentioning

confidence: 99%

Inferring Data Preconditions from Deep Learning Models for Trustworthy Prediction in Deployment

Ahmed,

Gao,

Rajan

2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

show abstract

“…DNNs require that inputs in deployment come from the same distribution as the training dataset [62,63]. However, real world inputs that are semantically similar to a human observer may look different to the model.…”

Section: Introductionmentioning

confidence: 99%

Repairing Failure-inducing Inputs with Input Reflection

Xiao

Beschastnikh

Sun

et al. 2022

Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering

Self Cite

View full text Add to dashboard Cite

Trained with a sufficiently large training and testing dataset, Deep Neural Networks (DNNs) are expected to generalize. However, inputs may deviate from the training dataset distribution in real deployments. This is a fundamental issue with using a finite dataset, which may lead deployed DNNs to mis-predict in production.Inspired by input-debugging techniques for traditional software systems, we propose a runtime approach to identify and fix failureinducing inputs in deep learning systems. Specifically, our approach targets DNN mis-predictions caused by unexpected (deviating and out-of-distribution) runtime inputs. Our approach has two steps. First, it recognizes and distinguishes deviating ("unseen" semantically-preserving) and out-of-distribution inputs from indistribution inputs. Second, our approach fixes the failure-inducing inputs by transforming them into inputs from the training set that have similar semantics. We call this process input reflection and formulate it as a search problem over the embedding space on the training set.We implemented a tool called InputReflector based on the above two-step approach and evaluated it with experiments on three DNN models trained on CIFAR-10, MNIST, and FMNIST image datasets. The results show that InputReflector can effectively distinguish deviating inputs that retain semantics of the distribution (e.g., zoomed images) and out-of-distribution inputs from in-distribution inputs. InputReflector repairs deviating inputs and achieves 30.78% accuracy improvement over original models. We also illustrate how InputReflector can be used to evaluate tests generated by deep learning testing tools.

show abstract