Concept-based Explanations for Out-Of-Distribution Detectors

Abstract

Out-of-distribution (OOD) detection plays a crucial role in ensuring the safe deployment of deep neural network (DNN) classifiers. While a myriad of methods have focused on improving the performance of OOD detectors, a critical gap remains in interpreting their decisions. We help bridge this gap by providing explanations for OOD detectors based on learned high-level concepts. We first propose two new metrics for assessing the effectiveness of a particular set of concepts for explaining OOD detectors: 1) detection completeness, which quantifies the sufficiency of concepts for explaining an OOD-detector's decisions, and 2) concept separability, which captures the distributional separation between in-distribution and OOD data in the concept space. Based on these metrics, we propose a framework for learning a set of concepts that satisfy the desired properties of detection completeness and concept separability and demonstrate the framework's effectiveness in providing concept-based explanations for diverse OOD techniques. We also show how to identify prominent concepts that contribute to the detection results via a modified Shapley value-based importance score.

Assessing Out-of-Domain Language Model Performance from Few Examples

“…This paper relates to a long line of work on understanding explanations, including explanations' human interpretability (Miller, 2019;Jacovi and Goldberg, 2020;Alqaraawi et al, 2020;Nguyen et al, 2021), explanations' faithfulness and ability to detect shortcuts (Geirhos et al, 2020) or spurious features (Bastings et al, 2021;Madsen et al, 2021;Zhou et al, 2021), and applications to OOD data (Ye and Durrett, 2022;Choi et al, 2022), including papers in the intersection of multiple directions (Adebayo et al, 2022;.…”