An English-Language Argumentation Interface for Explanation Generation with Markov Decision Processes in the Domain of Academic Advising

Abstract

A Markov Decision Process (MDP) policy presents, for each state, an action, which preferably maximizes the expected utility accrual over time. In this article, we present a novel explanation system for MDP policies. The system interactively generates conversational English-language explanations of the actions suggested by an optimal policy, and does so in real time. We rely on natural language explanations in order to build trust between the user and the explanation system, leveraging existing research in psychology in order to generate salient explanations. Our explanation system is designed for portability between domains and uses a combination of domain-specific and domain-independent techniques. The system automatically extracts implicit knowledge from an MDP model and accompanying policy. This MDP-based explanation system can be ported between applications without additional effort by knowledge engineers or model builders. Our system separates domain-specific data from the explanation logic, allowing for a robust system capable of incremental upgrades. Domain-specific explanations are generated through case-based explanation techniques specific to the domain and a knowledge base of concept mappings used to generate English-language explanations.

Lessons learned from development of a software tool to support academic advising

“…Instead, we are focused on the attitudes of students about advising and general attitudes about automated course advising tools. Additional results can be found in our other papers on this topic [1], [2].…”

“…The notion of explaining why a particular course has been recommended is unlikely to be foreign to experienced academic advisors (e.g., you should take course X to better prepare yourself for course Y). At the moment, only a few existing systems attempt to do this [1], [2], [19]. Explanation is an important part of recommendation [20].…”

“…An additional algorithm was added to later iterations of our software [2], specifically addressing student concerns raised in the Explanation System Survey. The algorithm is a probabilistic case-based calculation of the time from the current state to a user-defined goal (e.g., graduation or passing a particular "capstone" course) and presents a sentence in the following form:…”

“…Finally, we note that while the explanations presented in this paper are specific to the surveyed programs at the University of Kentucky, the explanation system itself was designed to be domain-independent [1], [2]. That is, the algorithms which generate the explanation do not depend on the specifics of the degree program, and the system was designed to permit modification of the underlying statistical model in order to support varying levels of student autonomy found at different institutions.…”

Reinforcement Learning Interpretation Methods: A Survey

“…Dodson et al [98], [99] applied a similar model on academic advising. However, their model is a mixture of domain independent and domain specific methodologies.…”