Misbehaviour prediction for autonomous driving systems

Abstract

Deep Neural Networks (DNNs) are the core component of modern autonomous driving systems. To date, it is still unrealistic that a DNN will generalize correctly to all driving conditions. Current testing techniques consist of offline solutions that identify adversarial or corner cases for improving the training phase. In this paper, we address the problem of estimating the confidence of DNNs in response to unexpected execution contexts with the purpose of predicting potential safety-critical misbehaviours and enabling online healing of DNN-based vehicles. Our approach SelfOracle is based on a novel concept of self-assessment oracle, which monitors the DNN confidence at runtime, to predict unsupported driving scenarios in advance. SelfOracle uses autoencoderand time series-based anomaly detection to reconstruct the driving scenarios seen by the car, and to determine the confidence boundary between normal and unsupported conditions. In our empirical assessment, we evaluated the effectiveness of different variants of SelfOracle at predicting injected anomalous driving contexts, using DNN models and simulation environment from Udacity. Results show that, overall, SelfOracle can predict 77% misbehaviours, up to six seconds in advance, outperforming the online input validation approach of DeepRoad. CCS CONCEPTS • Software and its engineering → Software testing and debugging.

AUTOTRAINER: An Automatic DNN Training Problem Detection and Repair System

“…A great number of testing methods have been proposed to test machine learning models, such as fuzzing [68][69][70][71][72][73][74][75], symbolic execution [76][77][78][79], runtime validation [80,81], fairness testing [74,78,82], etc. DeepXplore [83] introduced the neuron coverage metric to measures the percentage of activated neurons or a given test suite and DNN model, and generates new test inputs that can maximize the metric to test DL systems.…”

Quality Metrics and Oracles for Autonomous Vehicles Testing

“…The simulator includes different track circuits and supports training and testing of AVs that performs behavioural cloning, i.e., the AV learns the lane-keeping functionality from a dataset of labeled driving scenes. We selected the Udacity simulator because it is a popular platform used by researchers to evaluate testing techniques for AVs [31,41,43,39,35].…”

“…Object. We chose DAVE-2 [7] as our AV model since it is a widely used model in DNN testing papers [31,41,43,39]. More importantly, DAVE-2 exhibits realistic behaviours in simulated platforms, as well as realistic performance degradation when not appropriately trained [39], or when the model's architecture or training data get corrupted [24].…”

“…A number of papers use offline context-unaware thresholds as test oracles [19,31,41,43] (e.g., the predicted steering angle should deviate from the ground truth angle by less than 5 0 ), which are immaterial to the specific driving task and therefore poorly indicative of the driving performance. Other works study DNN-enabled AVs within simulation platforms, but only considering violations to the safety requirements, i.e., collisions with other vehicles, pedestrians, or other objects, or episodes of off-road driving behaviours [18,39,36]. While being indeed vital requirements that must be satisfied, they represent only a coarse-grained proxy for the quality of driving.…”

Towards a Model-Integrated Runtime Monitoring Infrastructure for Cyber-Physical Systems

“…Software-intensive systems, including Cyber-Physical Systems (CPS), such as autonomous vehicles [28], self-adaptive unmanned aerial vehicles (UAVs) [12], [21], [24], and factory floor robots [18], are becoming increasingly ubiquitous in society. They introduce safety concerns, which are typically addressed through regulatory and safety requirements.…”