No abstract

The question of how one should decide among competing explanations of data is at the heart of the scientific enterprise. Computational models of cognition are increasingly being advanced as explanations of behavior. The success of this line of inquiry depends on the development of robust methods to guide the evaluation and selection of these models. This article introduces a method of selecting among mathematical models of cognition known as minimum description length, which provides an intuitive and theoretically well-grounded understanding of why one model should be chosen. A central but elusive concept in model selection, complexity, can also be derived with the method. The adequacy of the method is demonstrated in 3 areas of cognitive modeling: psychophysics, information integration, and categorization.How should one choose among competing theoretical explanations of data? This question is at the heart of the scientific enterprise, regardless of whether verbal models are being tested in an experimental setting or computational models are being evaluated in simulations. A number of criteria have been proposed to assist in this endeavor, summarized nicely by Jacobs and Grainger (1994). They include (a) plausibility (are the assumptions of the model biologically and psychologically plausible?); (b) explanatory adequacy (is the theoretical explanation reasonable and consistent with what is known?); (c) interpretability (do the model and its parts-e.g., parameters-make sense? are they understandable?); (d) descriptive adequacy (does the model provide a good description of the observed data?); (e) generalizability (does the model predict well the characteristics of data that will be observed in the future?); and (f) complexity (does the model capture the phenomenon in the least complex-i.e., simplest-possible manner?).The relative importance of these criteria may vary with the types of models being compared. For example, verbal models are likely to be scrutinized on the first three criteria just as much as the last three to thoroughly evaluate the soundness of the models and their assumptions. Computational models, on the other hand, may have already satisfied the first three criteria to a certain level of acceptability earlier in their evolution, leaving the last three criteria to be the primary ones on which they are evaluated. This emphasis on the latter three can be seen in the development of quantitative methods designed to compare models on these criteria. These methods are the topic of this article.In the last two decades, interest in mathematical models of cognition and other psychological processes has increased tremendously. We view this as a positive sign for the discipline, for it suggests that this method of inquiry holds considerable promise. Among other things, a mathematical instantiation of a theory provides a test bed in which researchers can examine the detailed interactions of a model's parts with a level of precision that is not possible with verbal models. Furthermore, through systematic eval...

No abstract

No abstract

This article describes a general model of decision rule learning, the rule competition model, composed of 2 parts: an adaptive network model that describes how individuals learn to predict the payoffs produced by applying each decision rule for any given situation and a hill-climbing model that describes how individuals learn to fine tune each rule by adjusting its parameters. The model was tested and compared with other models in 3 experiments on probabilistic categorization. The first experiment was designed to test the adaptive network model using a probability learning task, the second was designed to test the parameter search process using a criterion learning task, and the third was designed to test both parts of the model simultaneously by using a task that required learning both category rules and cutoff criteria.

No abstract

A central problem in science is deciding among competing explanations of data containing random errors. We argue that assessing the ''complexity'' of explanations is essential to a theoretically wellfounded model selection procedure. We formulate model complexity in terms of the geometry of the space of probability distributions. Geometric complexity provides a clear intuitive understanding of several extant notions of model complexity. This approach allows us to reconceptualize the model selection problem as one of counting explanations that lie close to the ''truth.'' We demonstrate the usefulness of the approach by applying it to the recovery of models in psychophysics.H ow does one decide among competing explanations of data, given limited observations? This problem of model selection is at the core of progress in science. It is particularly vexing in the statistical sciences, where sources of error are diverse and hard to control. For example, in psychology experiments, the participants are themselves a serious source of uncontrolled random variation. Choosing between candidate models that purport to describe underlying regularities about human behavior given noisy data is correspondingly problematic. Over the decades, scientists have used various statistical tools to select among alternative models of data but have lacked a clear theoretical framework for understanding model selection. The purpose of this article is to alert scientists to the importance of accounting for complexity when choosing among models and to provide a geometric formulation of complexity. Not only does a geometric approach recast model selection in a more intuitive and meaningful light, but it also provides insight into the relations among conventional statistical techniques and the inherent tradeoffs between model performance and complexity.

The power law (y = ax-b) has been shown to provide a good description of data collected in a wide range of fields in psychology. R, B. Anderson and Tweney (1997) suggested that the model's data-fitting success may in part be artifactual, caused by a number of factors, one of which is the use of improper data averaging methods. The present paper follows up on their work and explains causes of the power law artifact. A method for studying the geometric relations among responses generated by mathematical models is introduced that shows the artifact is a result of the combined contributions of three factors: arithmetic averaging of data that are generated from a nonlinear model in the presence of individual differences. Schooler, 1991), or could it be due to some artifact (Estes, 1956)? Concerns of the latter type were recently rekindled in a study on the mathematical form of the forgetting function (R. B. Anderson & Tweney, 1997). These authors were concerned about the superior data-fitting ability of the power function relative to the exponential function. Simulations showed that occurrence of the artifact depended on data averaging, whether arithmetic, Specifically, they showed that data generated by either function were fit best by a power model when the data of simulated subjects were arithmetically averaged, a phenomenon we refer to throughout the paper as the power law artifact. Geometrically averaging the data sometimes eliminated the artifact, improving the fit of the exponential model relative to the power model.However, Wixted and Ebbesen (1997) and R. B. Anderson and Tweney (1997) showed that the artifact cannot be due solely to the use of an improper averaging method, because if it were, geometric averaging should always improve the fit of the exponential model. Wixted and Ebbesen (1997) reanalyzed two data sets from a previous study on the form ofthe forgetting function (Wixted & Ebbesen, 1991). The type of averaging made very little difference. Not only did the power function fit the arithmetically and geometrically averaged data better than the exponential function, but it did so by virtually the same A measure of scientific advancement in psychology is the discovery of lawful, predictable behavior. Such discoveries are particularly satisfying when the operation of the underlying mental process that mediates between stimulus and response can be described using mathematical functions. One of the functions that has enjoyed considerable popularity, especially among cognitive psychologists, is the power function (y = ax-b). The power function has been shown to provide an excellent description of human behavior in a variety offields, such as psychophysics (Stevens, 1971), memory (Wixted & Ebbesen, 1991), skill learning (Newell & Rosenbloom, 1981), and judgment and decision making (Stevenson, 1993). For example, research on human memory has explored the rate at which information is forgotten over time (Ebbinghaus, 1964;Wickens, 1998; see Rubin & Wenzel, 1996, for a review). In a typical experimental set...