Logically Sound Arguments for the Effectiveness of ML Safety Measures

Abstract

We investigate the issues of achieving sufficient rigor in the arguments for the safety of machine learning functions. By considering the known weaknesses of DNN-based 2D bounding box detection algorithms, we sharpen the metric of imprecise pedestrian localization by associating it with the safety goal. The sharpening leads to introducing a conservative post-processor after the standard non-max-suppression as a counter-measure. We then propose a semi-formal assurance case for arguing the effectiveness of the post-processor, which is further translated into formal proof obligations for demonstrating the soundness of the arguments. Applying theorem proving not only discovers the need to introduce missing claims and mathematical concepts but also reveals the limitation of Dempster-Shafer's rules used in semi-formal argumentation.

Unaligned but Safe -- Formally Compensating Performance Limitations for Imprecise 2D Object Detection

“…Finally, the recent work from Cheng et al [4] initiated the concept of safety post-processing attached to the standard post-processor to address the insufficiency of imprecise prediction. In [4], one estimates the enlargement threshold based on the data.…”

“…Finally, the recent work from Cheng et al [4] initiated the concept of safety post-processing attached to the standard post-processor to address the insufficiency of imprecise prediction. In [4], one estimates the enlargement threshold based on the data. This is in contrast to the concept stated in this paper where the enlargement factor is computed using worst-case analysis.…”

“…Finally, we compare the formally derived enlargement factor with an enlargement factor directly measured from the training data, following the methodology in [4]. There can be many interpretations over the value gap.…”