Customizable Reference Runtime Monitoring of Neural Networks using Resolution Boxes

Abstract

We present an approach for the runtime verification of classification systems via data abstraction. Data abstraction relies on the notion of box with a resolution. Boxbased abstraction consists in representing a set of values by its minimal and maximal values in each dimension. We augment boxes with a notion of resolution; this allows to define the notion of clustering coverage, which is intuitively a quantitative metric over boxes that indicates the quality of the abstraction. This allows studying the effect of different clustering parameters on the constructed boxes and estimating an interval of sub-optimal parameters. Moreover, we show how to automatically construct monitors that make use of both the correct and incorrect behaviors of a classification system. This allows checking the size of the monitor abstractions and analysing the separability of the network. Monitors are obtained by combining the sub-monitors of each class of the system placed at some selected layers. Our experiments demonstrate the effectiveness of our clustering coverage estimation and show how to assess the effectiveness and precision of monitors according to the selected clustering parameter and the chosen monitored layers. This paper is supported by the European Union's Horizon 2020 research and innovation programme under grant agreement No. 956123 -FOCETA and the French national program "Programme Investissements d'Avenir IRT Nanoelec" (ANR-10-AIRT-05).3 AI is used in a broad sense, to include many related fields and buzzwords, such as machine learning, big data, data science, and so on.

Sensing and Machine Learning for Automotive Perception: A Review

“…However, for regression, the estimation is based on computing the variance of all nearby grids having overlapping predictions over the same object. Yet, another possibility is to consider OoD detectors built using abstraction-based approaches [182], [183], [184], where DNN-generated feature vectors from the training dataset are clustered and enclosed using hyperrectangles. Note that input outside the training data distribution may not imply that it is not in the ODD.…”