Evaluating Plan-Property Dependencies: A Web-Based Platform and User Study

Abstract

The trade-offs between different desirable plan properties -- e.g. PDDL temporal plan preferences -- are often difficult to understand. Recent work addresses this by iterative planning with explanations elucidating the dependencies between such plan properties. Users can ask questions of the form ``Why does the plan not satisfy property p?'', which are answered by ``Because then we would have to forego q''. It has been shown that such dependencies can be computed reasonably efficiently. But is this form of explanation actually useful for users? We run a large crowd-worker user study (N=100 in each of 3 domains) evaluating that question. To enable such a study in the first place, we contribute a Web-based platform for iterative planning with explanations, running in standard browsers. Comparing users with vs. without access to the explanations, we find that the explanations enable users to identify better trade-offs between the plan properties, indicating an improved understanding of the planning task.

Explaining the Space of SSP Policies via Policy-Property Dependencies: Complexity, Algorithms, and Relation to Multi-Objective Planning

“…In response, she receives a summary of the part of the solution space satisfying Φ in terms of implied exclusions, i.e., the property subsets Ψ impossible to achieve simultaneously with Φ. Such contrastive explanations (Miller 2019) have proved particularly beneficial in strengthening the user's understanding of the solution space (Eifler et al 2022).…”