Learning overhypotheses with hierarchical Bayesian models

Abstract

Inductive learning is impossible without overhypotheses, or constraints on the hypotheses considered by the learner. Some of these overhypotheses must be innate, but we suggest that hierarchical Bayesian models help explain how the rest can be acquired. To illustrate this claim, we develop models that acquire two kinds of overhypotheses -overhypotheses about feature variability (e.g. the shape bias in word learning) and overhypotheses about the grouping of categories into ontological kinds like objects and substances. Learning overhypotheses 3Learning overhypotheses with hierarchical Bayesian models Compared to our best formal models, children are remarkable for learning so much from so little. A single labelled example is enough for children to learn the meanings of some words (Carey & Bartlett, 1978), and children develop grammatical constructions that are rarely found in the sentences that they hear (Chomsky, 1980). These inductive leaps appear even more impressive when we consider the many interpretations of the data that are logically possible but apparently never entertained by children (Goodman, 1955;Quine, 1960).Learning is impossible without constraints of some sort, but the apparent ease of children's learning may rely on relatively strong inductive constraints. Researchers have suggested, for example, that the M-constraint (Keil, 1979) and the shape bias (Heibeck & Markman, 1987) help explain concept learning, that universal grammar guides the acquisition of linguistic knowledge (Chomsky, 1980), and that constraints on the properties of physical objects (Spelke, 1990) support inferences about visual scenes.Constraints like these may be called theories or schemata, but we will borrow a term of Goodman's and refer to them as overhypotheses. 1 Although overhypotheses play a prominent role in nativist approaches to development (Keil, 1979;Chomsky, 1980;Spelke, 1990), some overhypotheses are probably learned (Goldstone & Johansen, 2003). One such overhypothesis is the shape bias -the expectation that all of the objects in a given category tend to have the same shape, even if they differ along other dimensions, such as color and texture. Smith, Jones, Landau, Gershkoff-Stowe, and Samuelson (2002) provide strong evidence that the shape bias is learned by showing that laboratory training allows children to demonstrate this bias at an age before it normally emerges. Other overhypotheses that appear to be learned include constraints on the rhythmic pattern of a child's native language (Jusczyk, 2003), Learning overhypotheses 4 and constraints on the kinds of feature correlations that are worth tracking when learning about artifacts or other objects (Madole & Cohen, 1995).The acquisition of overhypotheses raises some difficult challenges for formal models.It is difficult at first to understand how something as abstract as an overhypothesis might be learned, and the threat of an infinite regress must also be confronted -what are the inductive constraints that allow inductive constraints to be learned? ...

Eliminating unpredictable variation through iterated learning

“…This information affects the tendency to generalize variants to new lexical items, in line with the predictions of Hierarchical Bayesian models which evaluate word-specific patterns in accordance with higher-level information about variability (Kemp, Perfors & Tenenbaum, 2007;Perfors, Tenenbaum & Wonnacott, in press). The current work suggests that adult learners also have some bias in favour of a lexicalized system.…”

Schema-driven facilitation of new hierarchy learning in the transitive inference paradigm

“…Although it is important to bear in mind that phases 1 and 2 involved hierarchies of differing lengths (i.e., seven vs. nine) and different amounts of training (e.g., 12 vs. 20 blocks), the significant superiority of test trial performance in phase 2 could potentially reflect a change in learning strategy. One possibility is that having completed phase 1 participants were biased toward searching for an underlying hierarchical structure in phase 2, consistent with the notion of an "overhypothesis" that can constrain the learning of new information (e.g., Kemp et al 2007). …”

Explaining prompts children to privilege inductively rich properties

“…On this view, explanation drives learners towards broad generalizations, not towards causal properties (or away from perceptual properties), per se. However, children may already have formed higher-level generalizations (Dewar & Xu, 2010;Kemp, Perfors, & Tenenbaum, 2007) suggesting that certain types of properties, such as insides and category labels, are more likely to track common causal properties than superficial perceptual ones.…”