Justification-Based Reliability in Machine Learning

Abstract

With the advent of Deep Learning, the field of machine learning (ML) has surpassed human-level performance on diverse classification tasks. At the same time, there is a stark need to characterize and quantify reliability of a model's prediction on individual samples. This is especially true in applications of such models in safety-critical domains of industrial control and healthcare. To address this need, we link the question of reliability of a model's individual prediction to the epistemic uncertainty of the model's prediction. More specifically, we extend the theory of Justified True Belief (JTB) in epistemology, created to study the validity and limits of human-acquired knowledge, towards characterizing the validity and limits of knowledge in supervised classifiers. We present an analysis of neural network classifiers linking the reliability of its prediction on a test input to characteristics of the support gathered from the input and hidden layers of the network. We hypothesize that the JTB analysis exposes the epistemic uncertainty (or ignorance) of a model with respect to its inference, thereby allowing for the inference to be only as strong as the justification permits. We explore various forms of support (for e.g., k-nearest neighbors (k-NN) and ℓp-norm based) generated for an input, using the training data to construct a justification for the prediction with that input. Through experiments conducted on simulated and real datasets, we demonstrate that our approach can provide reliability for individual predictions and characterize regions where such reliability cannot be ascertained.

Mixed Initiative Approach for Reliable Tagging of Maintenance Records with Machine Learning

“…We present outcomes for the 4 models using a Class Confusion Matrix (CM) for the 8-class problem, that is typically used to indicate prediction performance of a classifier. For the epistemic models, (Virani et al, 2020) introduced an Augmented Confusion Matrix, a variant of CM, which splits it into 3 submatrices, each of which is a confusion matrix, but separately deals with predictions that have been assigned different epistemic statuses. In other words, the I-Know predictions are grouped within a dedicated confusion matrix showing performance of the classifier for the cases where the model's epistemic uncertainty is minimal and the predictions are highly confident.…”

“…Epistemic classification is an approach, within the Humble AI initiative, inspired from the theory of Justified True Belief (JTB) in Epistemology ( (Steup, 2007) is a good exposition), which aimed to study the limits and validity of human-acquired knowledge. We extend the same concept to understanding and characterizing the validity and limits of knowledge as acquired by supervised classifiers, as detailed in (Virani et al, 2020). We showed that the JTB analysis can be leveraged to expose the uncertainty of a classification model with respect to its inference due to ambiguity or extrapolation, thereby allowing for the inference to be only as strong as the justification permits.…”

“…Through experiments conducted on simulated and real datasets, we demonstrated that our approach can provide reliability for individual predictions and characterize regions where such reliability cannot be ascertained. While specific details of the approach are in the paper (Virani et al, 2020), Figure 6 shows the primary idea of gathering epistemic evidence, along with some examples of applying epistemic classification to the digital classification problem using the MNIST dataset (Deng, 2012). The epistemic status of the classification for an individual prediction (within {I-Know, I-May-Know or I-Dont-Know}) is inferred based on the nature of support, from the training data, that the sample garners in its neighborhood, within one or more latent spaces of the neural-network based classifier.…”

“…We make use of k-NN (k-Nearest Neighbors) as the support operator; in other words, a representation of a test sample in each of the 2 embeddings is created and 'k' nearest neighbors within the training data are retrieved. The tags assigned to the retrieved neighbors are then used, in conjunction with tag prediction made by the model for the test sample, to ascertain the epistemic status of the prediction (more details of this approach are described in (Virani et al, 2020)).…”

“…Based on this insight, we target the design of a mixed initiative tagging system that can assess reliability of individual predictions of machine learning models -a goal of the Humble AI initiative at GE. Specifically, we make use of a Justification-based reliability assessment of individual predictions by DL classifiers (Virani, Iyer, & Yang, 2020), which we call Epistemic Classification. Using this approach, we ascribe reliability to tag predictions made by our BERT-based, automated tagging model; the reliability assessments further help to identify cases to assign to human experts for tagging, versus those that can be reliably tagged automatically.…”