Explainable AI: A Neurally-Inspired Decision Stack Framework

Abstract

European law now requires AI to be explainable in the context of adverse decisions affecting the European Union (EU) citizens. At the same time, we expect increasing instances of AI failure as it operates on imperfect data. This paper puts forward a neurally inspired theoretical framework called “decision stacks” that can provide a way forward in research to develop Explainable Artificial Intelligence (X-AI). By leveraging findings from the finest memory systems in biological brains, the decision stack framework operationalizes the definition of explainability. It then proposes a test that can potentially reveal how a given AI decision was made.

When neuro-robots go wrong: A review

“…In the past decade, with the application of AI in several autonomous systems and robots, we have seen a tremendous amount of research interest in X-AI methods. Currently, we can choose from a suite of X-AI methods to untangle deep learning opaque models (Lipton, 2017;Došilović et al, 2018;Xu et al, 2019;Holzinger et al, 2022;Khan et al, 2022). There are various categorizations of X-AI methods based on several criteria, including structure, design transparency, agnosticness, scope, supervision, explanation type, and data type, as listed in Table 1 (Khan et al, 2022).…”

“…Overall, these excellent foundational methods [summarized in Table 1; for more details, readers may consult (Khan et al, 2022)] help produce some model understanding and present bits of human interpretable understanding. However, there is still no comprehensive understanding of how an AI implements a decision while explaining the model decision (Khan et al, 2022). These methods are far from perfect.…”

“…We would assume that robots (and neuro-robots) have become more capable than ever before because the cost of sensors has reduced significantly, and AI algorithms have matured exponentially (Silver et al, 2016). Yet, most AI algorithms, particularly the Blackbox deep neural networks, are still largely unexplainable (de Bruijn et al, 2022;Khan et al, 2022). Similarly, many brain processes are yet to be unknotted, not only in humans but also in animals with simple brains (Krichmar et al, 2019).…”

“…Modeled after biological brains, neuro-robots use a combination of neurally-inspired computing and AI. Neuro-robots operate on noisy and often uncertain data (Khan et al, 2022), to make decisions to help reduce the workload of fellow humans. When these robots work, they are of huge utility, allowing for, among other things, prosthetics (McMullen et al, 2014;Johannes et al, 2020;Handelman et al, 2022;Rejcek, 2022) and wearable systems supporting locomotion and learning processes (Colachis et al, 2018;Billard and Kragic, 2019).…”

“…The failure of robots is not startling. Intelligence is the act of decision-making based on prevailing uncertainty (Khan et al, 2022). This fact differentiates robots deploying AI from non-intelligent decision systems based on the flow-chart design, as is the case in most electronics (Friedman and Zeckhauser, 2012).…”