Evaluating the Interpretability of the Knowledge Compilation Map: Communicating Logical Statements Effectively

Booth, Serena; Muise, Christian; Shah, Julie A.

doi:10.24963/ijcai.2019/804

Cited by 13 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Much more can be done. A particularly intriguing aspect is the experimental evaluation of the degree up to which threshold expressions are more humaninterpretable than equivalent logical formulations, along the lines of [5]. Also, with the prospect of sharing ontologies with perceptron connectives, their addition to semantic web languages will need to be carefully pursued.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Perceptron Connectives in Knowledge Representation

Galliani

Righetti

Kutz

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We discuss the role of perceptron (or threshold) connectives in the context of Description Logic, and in particular their possible use as a bridge between statistical learning of models from data and logical reasoning over knowledge bases. We prove that such connectives can be added to the language of most forms of Description Logic without increasing the complexity of the corresponding inference problem. We show, with a practical example over the Gene Ontology, how even simple instances of perceptron connectives are expressive enough to represent learned, complex concepts derived from real use cases. This opens up the possibility to import concepts learnt from data into existing ontologies.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Other, more efficient types of adder circuits are known and used in practice, and could be translated along similar lines; but ripple-carry adders have the advantage of simplicity and suffice for our purposes 5. This first sum is strictly speaking unnecessary, but we keep it for clarity of exposition.…”

mentioning

confidence: 99%

Perceptron Connectives in Knowledge Representation

Galliani

Righetti

Kutz

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…We note that the two approaches are complementary. Booth et al [15] compile logical formulas that specify when certain behaviors occur, e.g., by stating for which regime in the state space an agent will perform a particular action. However, this approach requires a state representation that is understandable to the user, which may not be the case in many complex domains, especially when DRL is used.…”

Section: Related Workmentioning

confidence: 99%

Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps

Huber

Weitz

André

et al. 2021

Artificial Intelligence

View full text Add to dashboard Cite

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the global behavior of the agent, describing the actions it takes in different states. Other approaches devised local explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summaries that extract important trajectories of states from simulations of the agent with saliency maps which show what information the agent attends to. Our results show that the choice of what states to include in the summary (global information) strongly affects people's understanding of agents: participants shown summaries that included important states significantly outperformed participants who were presented with agent behavior in a randomly set of chosen world-states. We find mixed results with respect to augmenting demonstrations with saliency maps (local information), as the addition of saliency maps did not significantly improve performance in most cases. However, we do find some evidence that saliency maps can help users better understand what information the agent relies on in its decision making, suggesting avenues for future work that can further improve explanations of RL agents.

show abstract

“…DNFs have widely been presumed to be easy to interpret (Hayes and Shah 2017;Wang et al 2015;Hauser et al 2010). A recent study suggests that while DNFs are not uniquely easy to interpret, they at least satisfy many of the necessary properties needed for interpretability, as long as the formulas are small (Booth, Muise, and Shah 2019).…”

Section: Example Application: Explaining Classifiersmentioning

confidence: 99%

More Accurate Learning of k-DNF Reference Classes

Juba

2020

AAAI

View full text Add to dashboard Cite

In machine learning, predictors trained on a given data distribution are usually guaranteed to perform well for further examples from the same distribution on average. This often may involve disregarding or diminishing the predictive power on atypical examples; or, in more extreme cases, a data distribution may be composed of a mixture of individually “atypical” heterogeneous populations, and the kind of simple predictors we can train may find it difficult to fit all of these populations simultaneously. In such cases, we may wish to make predictions for an atypical point by selecting a suitable reference class for that point: a subset of the data that is “more similar” to the given query point in an appropriate sense. Closely related tasks also arise in applications such as diagnosis or explaining the output of classifiers. We present new algorithms for computing k-DNF reference classes and establish much stronger approximation guarantees for their error rates.

show abstract

Evaluating the Interpretability of the Knowledge Compilation Map: Communicating Logical Statements Effectively

Cited by 13 publications

References 11 publications

Perceptron Connectives in Knowledge Representation

Perceptron Connectives in Knowledge Representation

Local and global explanations of agent behavior: Integrating strategy summaries with saliency maps

More Accurate Learning of k-DNF Reference Classes

Contact Info

Product

Resources

About