“Boxing Clever”: Practical Techniques for Gaining Insights into Training Data and Monitoring Distribution Shift

Abstract

Training data has a significant influence on the behaviour of an artificial intelligence algorithm developed using machine learning techniques. Consequently, any argument that the trained algorithm is, in some way, fit for purpose ought to include consideration of data as an entity in its own right. We describe some simple techniques that can provide domain experts and algorithm developers with insights into training data and which can be implemented without specialist computer hardware. Specifically, we consider sampling density, test case generation and monitoring for distribution shift. The techniques are illustrated using example data sets from the University of California, Irvine, Machine Learning repository.

A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability

“…Moreover, there are different ways to define the set of hyper-rectangles. For example, the "boxing clever" method in [Ashmore and Hill, 2018], initially proposed for designing training datasets, divides the input space into a series of representative boxes. When the hyper-rectangle is sufficiently fine-grained with respect to Lipschitz constant of the DNN, the method in [Wicker et al, 2018] becomes exhaustive search and has provable guarantee on its result.…”

“…Therefore, although it is reasonable to believe that the resulting trained models can perform well on new inputs close to the training data, it is also understandable that the trained models might not perform correctly in those inputs where there is no neighbouring training data. While techniques are being requested to achieve better generalisability for DNN training algorithm including various regularisation techniques (see e.g., for a comprehensive overview), as suggested in e.g., [Amodei et al, 2016, Ashmore and Hill, 2018, Moreno-Torres et al, 2012, it is also meaningful (particularly for the certification of safety critical systems) to be able to identify those inputs on which the trained models should not have high confidence. Technically, such inputs can be formally defined as both topologically far away from training data in the input space and being classified with high probability by the trained models.…”

Concolic testing for deep neural networks

“…Whilst in [36], coverage is enforced to finite partitions of the input space, relying on predefined sets of application-specific scenario attributes. The "boxing clever" technique in [37] focuses on the distribution of training data and divides the input domain into a series of representative boxes. In [38], the difference between test dataset and training dataset is measured by quantifying the difference between DNNs' activation patterns.…”

Testing and verification of neural-network-based safety-critical control software: A systematic literature review

“…To be more specific, the instinctive features of NN-based software (e.g., NN model's architectural details and the working mechanism of NN) should be carefully considered when setting the testing criteria. That is testing criteria should be defined comprehensively and explicitly under the consideration of not only test case coverage but also the robustness of NN-based system performance (for instance, test how NN will respond when input data change slightly) and the features of training data set, such as the data density issue mentioned in [146].…”