Safety Assessment: From Black-Box to White-Box

SafeSens - Uncertainty Quantification of Complex Perception Systems

“…The primary ideas to enable a holistic, uncertainty-based safety analysis and validation are: (1) A customizable, modular system architecture, (2) simulation and (3) uncertainty quantification. Subsequently, we introduced a methodology and corresponding framework [5] capable of providing all these qualities including a detailed approach for meta-models for ML [6]. Throughout this paper the methodology and framework will be referred to as SafeSens.…”

“…How to create a stochastic model for ML-based object detectors is described in detail in our previous contribution [6]. In short: Evaluated test data from a deep neural network (DNN) is used to train an introspective, outside model estimator to reliably predict the DNN behavior.…”

“…The RF model is trained on an introspective data set to reliable approximate a custom DNN regarding safety (i.e., to predict this exact ML model's behavior! ), for more details refer to [6]. All these properties or data are either part of the technical specification of the system, e.g.…”

“…In order to approximate the behavior of a specific DNN object detector an introspective data set is used to train the RF within SafeSens. On a related note, occlusion is part of the input features f for the white-box model (we used the term safety features in [6]) so this features has the potential to influence the uncertainty estimation. In our paper previous paper [6] we trained three different models to predict three different safety-relevant output variables, cf.…”

“…On a related note, occlusion is part of the input features f for the white-box model (we used the term safety features in [6]) so this features has the potential to influence the uncertainty estimation. In our paper previous paper [6] we trained three different models to predict three different safety-relevant output variables, cf. leftmost graphs for three different normal distributions (orange) in Fig.…”