Inductive learning is impossible without overhypotheses, or constraints on the hypotheses considered by the learner. Some of these overhypotheses must be innate, but we suggest that hierarchical Bayesian models help explain how the rest can be acquired. To illustrate this claim, we develop models that acquire two kinds of overhypotheses -overhypotheses about feature variability (e.g. the shape bias in word learning) and overhypotheses about the grouping of categories into ontological kinds like objects and substances.
Learning overhypotheses 3Learning overhypotheses with hierarchical Bayesian models Compared to our best formal models, children are remarkable for learning so much from so little. A single labelled example is enough for children to learn the meanings of some words (Carey & Bartlett, 1978), and children develop grammatical constructions that are rarely found in the sentences that they hear (Chomsky, 1980). These inductive leaps appear even more impressive when we consider the many interpretations of the data that are logically possible but apparently never entertained by children (Goodman, 1955;Quine, 1960).Learning is impossible without constraints of some sort, but the apparent ease of children's learning may rely on relatively strong inductive constraints. Researchers have suggested, for example, that the M-constraint (Keil, 1979) and the shape bias (Heibeck & Markman, 1987) help explain concept learning, that universal grammar guides the acquisition of linguistic knowledge (Chomsky, 1980), and that constraints on the properties of physical objects (Spelke, 1990) support inferences about visual scenes.Constraints like these may be called theories or schemata, but we will borrow a term of Goodman's and refer to them as overhypotheses. 1 Although overhypotheses play a prominent role in nativist approaches to development (Keil, 1979;Chomsky, 1980;Spelke, 1990), some overhypotheses are probably learned (Goldstone & Johansen, 2003). One such overhypothesis is the shape bias -the expectation that all of the objects in a given category tend to have the same shape, even if they differ along other dimensions, such as color and texture. Smith, Jones, Landau, Gershkoff-Stowe, and Samuelson (2002) provide strong evidence that the shape bias is learned by showing that laboratory training allows children to demonstrate this bias at an age before it normally emerges. Other overhypotheses that appear to be learned include constraints on the rhythmic pattern of a child's native language (Jusczyk, 2003), Learning overhypotheses 4 and constraints on the kinds of feature correlations that are worth tracking when learning about artifacts or other objects (Madole & Cohen, 1995).The acquisition of overhypotheses raises some difficult challenges for formal models.It is difficult at first to understand how something as abstract as an overhypothesis might be learned, and the threat of an infinite regress must also be confronted -what are the inductive constraints that allow inductive constraints to be learned? ...