An Integrated Process for Verifying Deep Learning Classifiers Using Dataset Dissimilarity Measures

Abstract

The specification and verification of algorithms is vital for safety-critical autonomous systems which incorporate deep learning elements. We propose an integrated process for verifying artificial neural network (ANN) classifiers. This process consists of an off-line verification and an on-line performance prediction phase. The process is intended to verify ANN classifier generalisation performance, and to this end makes use of dataset dissimilarity measures. We introduce a novel measure for quantifying the dissimilarity between the dataset used to train a classification algorithm, and the test dataset used to evaluate and verify classifier performance. A system-level requirement could specify the permitted form of the functional relationship between classifier performance and a dissimilarity measure; such a requirement could be verified by dynamic testing. Experimental results, obtained using publicly available datasets, suggest that the measures have relevance to real-world practice for both quantifying dataset dissimilarity, and specifying and verifying classifier performance.

Uncertainty Quantification of Machine Learning Model Performance via Anomaly-Based Dataset Dissimilarity Measures

“…The UQ method also returns an estimate of the uncertainty interval associated with a performance prediction. The proposed UQ method is an extension of the technique discussed in [7,47], where the authors introduce the idea of studying the relationship between ANN classifier performance and data dissimilarity in the context of ML verification. The degree of shift is to be gauged by data dissimilarity measures.…”

“…The accuracy of ML models tends to fall when used on data that are statistically different from their training data [4,7]. The term in-distribution is used to describe data which are drawn from the training data-generating distribution (i.e., the probability distribution from which training samples are drawn); out-of-distribution data are not drawn from the training data-generating distribution.…”

Contextualised Out-of-Distribution Detection Using Pattern Identification

“…We trained our detectors with RMSprop (learning rate 5 × 10 −4 , weight decay 10 −5 ), for 30 epochs (ImageNet) or 200 epochs (all other ID datasets). As a comparison, we also implemented a classbased FNRD [9], extracting neuron activation values at different layers of the classifier.…”

“…[6] proposes two related methods: MaxLogit -based on the maximum logit value -and KL-Matching which measures the KL divergence between the output of the model and the class-conditional mean softmax values. The Fractional Neuron Region Distance [9] (FNRD) computes the range of activations for each neuron over the training set in order to empirically characterise the statistical properties of these activations, then provides a score describing how many neuron outputs are outside the corresponding range boundaries for a given input. Similarly, for each layer in the model, [18] computes the range of pairwise feature correlation between channels across the training set.…”

“…Results with * are extracted from [27] -keeping only OoD-agnostic methods. We also add results of our implementation of a class-based FNRD [9]. Experiments on ImageNet using 6 CODE detectors have not yet been conducted due to limited ressources (denoted ).…”