AI for Explaining Decisions in Multi-Agent Environments

Kraus, Sarit; Azaria, Amos; Fiošina, Jeļena; Greve, Maike; Hazon, Noam; Kolbe, Lutz M.; Lembcke, Tim-Benjamin; Müller, Jörg P.; Schleibaum, Sören; Vollrath, Mark

doi:10.1609/aaai.v34i09.7077

Cited by 27 publications

(24 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Human Centered AI institute,(https://hai.stanford.edu/) or the UKRI Trustworthy Autonomous Systems Hub. (http://www.tas.ac.uk/) From a technical perspective, human-centered (Lepri et al, 2021;Shneiderman, 2020;Wilson and Daugherty, 2018) and machine-centered approaches (Rahwan et al, 2019;Awad et al, 2018;Kraus et al, 2020) to the development of artificial intelligence have emerged. Human-centered approaches propose to develop socially beneficial and ethical machine intelligence that also augments human capabilities and undertakes human tasks with high reliability.…”

mentioning

confidence: 99%

Trustworthy human-AI partnerships

2021

View full text Add to dashboard Cite

In this paper, we foreground some of the key research challenges that arise in the design of trustworthy human-AI partnerships. In particular, we focus on the challenges in designing human-AI partnerships that need to be addressed to help humans and organizations trust their machine counterparts individually or as a collective (e.g., as robot teams or groups of software agents). We also aim to identify the risks associated with human-AI partnerships and therefore determine the associated measures to mitigate these risks. By so doing, we will trigger new avenues of research that will address the key barriers to the adoption of AI-based systems more widely in our daily lives and in industry. INTRODUCTIONRecent advances in Artificial Intelligence (AI), Machine Learning (ML), and Robotics have significantly enhanced the capabilities of machines. Machine intelligence is now able to support human decision making, augment human capabilities, and, in some cases, take over control from humans and act fully autonomously. Real-world examples of autonomous systems including self-driving vehicles, recommender systems, facial recognition systems, and automated trading are only just beginning to demonstrate the value machine intelligence can deliver to society and the wider economy. However, unfortunately, there have also been failures in the deployment of such autonomous systems that have resulted in fatal car crashes, plane accidents, and stock market failures (Brynjolfsson and McAfee, 2014;Daugherty and Wilson, 2018). Such failures are often attributed to a poor understanding of how to weave AI systems into our societal and industrial fabric.Given this, we believe that the next big advance for AI and ML systems will involve them being significantly more tightly embedded into systems alongside humans, interacting and influencing each other in a number of ways. Such human-AI partnerships are a new form of socio-technical system in which the potential synergies between humans and machines are much more fully utilized. To achieve this, AI systems will need to leave their currently solipsistic nature behind and be able to cooperate, coordinate, and compete with one another and their human interlocutors. Such partnerships will combine their complementary skills and capabilities to make the best use of the distinctive strengths of humans and machines (Licklider, 1960), while also acknowledging their potentially diverging preferences, purpose, and objectives that may give rise to conflict or cause them to attempt to influence (intentionally or not) each other's decision-making. Likewise, humans will likely be challenged to work and live with AI systems as fully autonomous partners, rather than purely tools that they can manipulate or query. The modalities through which they engage with AI systems will also vary greatly, shifting away from typical screen-based or tactile interfaces to voice or brain-controlled, opening new opportunities and risks for interactions between humans and AI systems. These elements of the human-AI partnersh...

show abstract

mentioning

confidence: 99%

Trustworthy human-AI partnerships

2021

View full text Add to dashboard Cite

show abstract

“…In our work the explanations do not attempt to explain the output of the system to a passenger but to provide additional information that is likely to increase the user's satisfaction from the system. Therefore, our work can be seen as one of the first instances of x-MASE [29], explainable systems for multi-agent environments.…”

Section: Related Workmentioning

confidence: 99%

Explaining Ridesharing: Selection of Explanations for Increasing User Satisfaction

Zar¹,

Hazon²,

Azaria³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Transportation services play a crucial part in the development of modern smart cities. In particular, on-demand ridesharing services, which group together passengers with similar itineraries, are already operating in several metropolitan areas. These services can be of significant social and environmental benefit, by reducing travel costs, road congestion and CO2 emissions. Unfortunately, despite their advantages, not many people opt to use these ridesharing services. We believe that increasing the user satisfaction from the service will cause more people to utilize it, which, in turn, will improve the quality of the service, such as the waiting time, cost, travel time, and service availability. One possible way for increasing user satisfaction is by providing appropriate explanations comparing the alternative modes of transportation, such as a private taxi ride and public transportation. For example, a passenger may be more satisfied from a shared-ride if she is told that a private taxi ride would have cost her 50% more. Therefore, the problem is to develop an agent that provides explanations that will increase the user satisfaction. We model our environment as a signaling game and show that a rational agent, which follows the perfect Bayesian equilibrium, must reveal all of the information regarding the possible alternatives to the passenger. In addition, we develop a machine learning based agent that, when given a shared-ride along with its possible alternatives, selects the explanations that are most likely to increase user satisfaction. Using feedback from humans we show that our machine learning based agent outperforms the rational agent and an agent that randomly chooses explanations, in terms of user satisfaction.

show abstract

“…Such an interaction could be for instance conversational, formalized as dialogue between the user and the system, see e.g. [18,51,57,101,110,121,128,144,146,150,151,186,188,190,204].…”

Section: Actionable Explanationsmentioning

confidence: 99%

“…However, more generally and with respect to the user's satisfaction, Kraus et al stipulate in [121] that there has so far been little explainability in multi-agent environments. They claim explainability in MAS is more challenging than in other settings because "in addition to identifying the technical reasons that led to the decision, there is a need to convey the preferences of the agents that were involved."…”

Section: Multi-agent Systemsmentioning

confidence: 99%

Machine Reasoning Explainability

Cyras,

Badrinath,

Mohalik

et al. 2020

Preprint

View full text Add to dashboard Cite

As a field of AI, Machine Reasoning (MR) uses largely symbolic means to formalize and emulate abstract reasoning. Studies in early MR have notably started inquiries into Explainable AI (XAI) -arguably one of the biggest concerns today for the AI community. Work on explainable MR as well as on MR approaches to explainability in other areas of AI has continued ever since. It is especially potent in modern MR branches, such as argumentation, constraint and logic programming, planning. We hereby aim to provide a selective overview of MR explainability techniques and studies in hopes that insights from this long track of research will complement well the current XAI landscape. This document reports our work in-progress on MR explainability.

show abstract

AI for Explaining Decisions in Multi-Agent Environments

Cited by 27 publications

References 23 publications

Trustworthy human-AI partnerships

Trustworthy human-AI partnerships

Explaining Ridesharing: Selection of Explanations for Increasing User Satisfaction

Machine Reasoning Explainability

Contact Info

Product

Resources

About