Learning Probabilistic Hierarchical Task Networks as Probabilistic Context-Free Grammars to Capture User Preferences

Abstract

We propose automatically learning probabilistic Hierarchical Task Networks (pH-TNs) in order to capture a user's preferences on plans, by observing only the user's behavior. HTNs are a common choice of representation for a variety of purposes in planning, including work on learning in planning. Our contributions are (a) learning structure and (b) representing preferences. In contrast, prior work employing HTNs considers learning method preconditions (instead of structure) and representing domain physics or search control knowledge (rather than preferences). Initially we will assume that the observed distribution of plans is an accurate representation of user preference, and then generalize to the situation where feasibility constraints frequently prevent the execution of preferred plans. In order to learn a distribution on plans we adapt an Expectation-Maximization (EM) technique from the discipline of (probabilistic) grammar induction, taking the perspective of task reductions as productions in a context-free grammar over primitive actions. To account for the difference between the distributions of possible and preferred plans we subsequently modify this core EM technique, in short, by rescaling its input.

Integrating representation learning and skill learning in a human-like intelligent agent

“…We exploit the connection between representation learning and grammar induction by extending an existing pCFG algorithm [Li et al, 2009] to support feature learning and transfer. It shares some ideas from previous work on grammar induction (e.g., [Wolff, 1982, Langley and Stromsten, 2000, Stolcke, 1994, Vanlehn, 1987), which searches for the target grammar by adding or merging non-terminal symbols 2 .…”

“…In fact, a common strategic error students make in a problem like -3x=12 is for the student to divide both sides by 3 rather than -3 [Li et al, 2011a]. Based on these observations, we built a representation learner by extending an existing probabilistic context free grammar (pCFG) learner [Li et al, 2009] to support feature learning and transfer learning. The representation learner is domain general.…”

“…Before introducing the representation learning algorithm, we first briefly review the pCFG learner [Li et al, 2009] it is based on. The pCFG learner is a variant of the inside-outside algorithm [Lari and Young, 1990].…”

“…Having reviewed Li et al's [2009] pCFG learning algorithm, we are ready to describe how it is extended to support representation learning without SimStudent. The input of the system is a set of pairs such as -3x, -3 , where the first element is the input to a feature extraction mechanism (e.g., coefficient), and the second is the extraction output (e.g., -3 is the coefficient of -3x).…”

“…The learning process contains two steps. The system first acquires the grammar using Li et al's [2009] algorithm. After that, the representation learner tries to identify a non-terminal symbol in one of the rules as the target feature.…”