A nascent design theory for explainable intelligent systems

Abstract

Due to computational advances in the past decades, so-called intelligent systems can learn from increasingly complex data, analyze situations, and support users in their decision-making to address them. However, in practice, the complexity of these intelligent systems renders the user hardly able to comprehend the inherent decision logic of the underlying machine learning model. As a result, the adoption of this technology, especially for high-stake scenarios, is hampered. In this context, explainable artificial intelligence offers numerous starting points for making the inherent logic explainable to people. While research manifests the necessity for incorporating explainable artificial intelligence into intelligent systems, there is still a lack of knowledge about how to socio-technically design these systems to address acceptance barriers among different user groups. In response, we have derived and evaluated a nascent design theory for explainable intelligent systems based on a structured literature review, two qualitative expert studies, a real-world use case application, and quantitative research. Our design theory includes design requirements, design principles, and design features covering the topics of global explainability, local explainability, personalized interface design, as well as psychological/emotional factors.

How Artificial Intelligence Challenges Tailorable Technology Design

“…In the broader field of human-AI interaction, tendencies toward user tailoring are present but primarily concentrate on the initial design of AI-enabled system (Amershi et al 2019). Design theory for explainable intelligent IS partially considers the user perspective (Herm et al 2022;Langer et al 2021). Yet, it falls short in accounting for the active role of the user in tailoring the IS.…”

“…In recent years, there have been efforts to theorize the design of such systems. For example, Herm et al (2022) discussed explainable AI, and Kane et al (2021) explored emancipatory assistants. As these efforts mainly address the AI components of IS, they do not contribute to the interplay between the designer, user, and AI-enabled system.…”

Electronic Markets on AI and standardization

“…The same applies when AI models are offered as products on information marketplaces (see Alt & Zimmermann, 2022) and certifications (e.g., privacy seals) provide orientation for buyers. More insight in this direction is included in the present special issue on trust in AI (Meske et al, 2022) with suggestions for standards to define the quality of explanations (Herm et al, 2022) , for applying legal standards such as GDPR (Dickhaut et al, 2022), for establishing the transparency about data quality standards (Michalke et al, 2022) as well as for standards to collect, process, and use personal data in networked settings (Koester et al, 2022). Finally, the output side denotes data that is being generated by AI-based systems (Thiebes et al 2021, p. 458).…”

A.I. Into Fashion Processes

“…In recent years, AI has demonstrated its application potential in many areas, thus becoming more than just a domain of technical interest but involving interdisciplinary research for further progress (European Commission, 2022b). Indeed, the transversal performances of AI intelligent systems, namely systems capable of learning from increasingly complex data, analysing situations and assisting in decision-making processes (Herm et al, 2022), demonstrated their great applicability across the entire complex fashion value and supply chain over the last 30 years. Despite this, fashion industries are still sceptical about its proper integration into their processes due to a lack of knowledge and skills to manage the technology into traditional processes toward a broader paradigmatic shift (Giri et al, 2019) for a positive and responsible change in the sector.…”