A Conceptual Framework for Explainability Requirements in Software-Intensive Systems

Abstract

Software-intensive systems include enterprise systems, IoT systems, cyber-physical systems, and industrial control systems where software plays a vital role. In such systems, the software is increasingly responsible for autonomous decisionmaking. However, trust can be hindered by the black-box nature of these systems, whose autonomous decisions may be confusing or even dangerous for humans. Thus, explainability emerges as a crucial non-functional property to achieve transparency and increase the understanding of the systems' behavior, fostering their acceptance in our society.This paper introduces a conceptual framework for eliciting explainability requirements at different granularity levels. Each level is associated with a set of meta-requirements and means for instantiating the framework within a system to make it capable of producing explanations in a given application domain. We illustrate our conceptual framework using a running example from the robotics domain.