Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy

Abstract

When AI systems interact with humans in the loop, they are often called on to provide explanations for their plans and behavior. Past work on plan explanations primarily involved the AI system explaining the correctness of its plan and the rationale for its decision in terms of its own model. Such soliloquy is wholly inadequate in most realistic scenarios where the humans have domain and task models that differ significantly from that used by the AI system. We posit that the explanations are best studied in light of these differing models. In particular, we show how explanation can be seen as a "model reconciliation problem" (MRP), where the AI system in effect suggests changes to the human's model, so as to make its plan be optimal with respect to that changed human model. We will study the properties of such explanations, present algorithms for automatically computing them, and evaluate the performance of the algorithms.

Plan Explanations as Model Reconciliation -- An Empirical Study

“…Thus, going forward, the objective function should incorporate the cost or difficulty of analyzing the plans and explanations from the point of view of the human in addition to the current costs of explicability and explanations (as shown in Table 1) modeled from the perspective of the robot model (refer to [29] for more details). Table 1 shows the statistics of the explanations / plans from 124 problem instances that required minimal explanations as per [5], and 25 and 40 instances that contained balanced and explicable plans respectively, as before. As desired, the robot gains in length of explanations but loses out in cost of plans produced as it progresses along the spectrum of optimal to explicable plans.…”

“…This seems to be a viable approach to further reduce size (c.f. selective property of explanations in [22]) of explanations in a post-hoc setting, and is out of scope of explanations developed in [5].…”

“…The authors in [22] state the three most important properties of explanations (as accounted for in the existing literature in the social sciences) as being (1) contrastive (so as to be able to compare the fact being questioned with other alternatives or foils); (2) selective (of what information among many to attribute causality of an event to); and (3) social (implying that the explainer must be able to leverage the mental model of the explainee while engaging in the explanation process). In recent work on explanation generation for planners, authors in [5] expressed similar sentiments by arguing that the explanation process towards end users "cannot be a soliloquy" but rather a process of "model reconciliation" during which the system tries to bring the mental model (social property) of the user on the same page with respect to the plan being explained. The authors in [5] addressed the contrastive property by ensuring optimality of the plan being explained in the updated human mental model, and the selectivity property by computing the minimum number of updates required to realize the above constraint.…”

Interactive Plan Explicability in Human-Robot Teaming

“…Table I shows the ratios (refer to as the explicability ratio) between the number of explicable actions and the number of actions over all plans, created for the testing problems using our approach, FF planner, and human plan, respectively. The interactive explicable plan (our approach) is created using the heuristic search method mentioned in Equation (4). Note that all the human actions will be considered explicable in our plans (although one can argue that is not the case).…”

“…The challenge is how the robot can utilize this information to synthesize a plan while avoiding conflicts or providing proactive assistance [2,5]. There are different approaches to planning with such consideration [1,4]. Another the key consideration is to be socially acceptable [8,15], where the robot must be aware of expectation of the human teammates and acts accordingly.…”