Selectivity, scope, and simplicity of models: A lesson from fitting judgments of perceived depth.

Abstract

When comparing psychological models a researcher should assess their relative selectivity, scope, and simplicity. The third of these considerations can be measured by the models' parameter counts or equation length, the second by their ability to fit random data, and the first by their differential ability to fit patterned data over random data. These conclusions are based on exploration of integration models reflecting depth judgments. Replication of Massaro's (1988a) results revealed an additive model , and Massaro's fuzzy-logical model of perception (FLMP) fit data equally well, but further exploration showed that the FLMP fit random data better. The FLMP's successes may reflect not its sensitivity in capturing psychological process but its scope in fitting any data and its complexity as measured by equation length.Good scientific theories are usually thought to have several properties: They are accurate, simple, broad in scope, internally consistent, and have the ability to generate new research (Kuhn, 1977). When models can be used to instantiate theories, they might reflect these same properties. For our purposes the two key concepts in this set are simplicity and scope. Simplicity can be measured in several ways. We measure it in two: by the number of parameters in a model and, in a way not customary to experimental psychology, by the length of the equation that instantiates a model. Scope can also be measured in various ways, but here we consider how theory or model accounts for all possible data functions, where those functions are generated by a reasonably large sample of random data sets. Under this construal, broad scope is a mixed blessing. A model with greater scope than another may fit more data functions of interest to the researcher, but simultaneously it may also fit more functions of no interest. Thus, we propose a new criterion for testing and comparing models: selectivity. We define selectivity as the relative ability of a This research was supported by National Science Foundation Grant BNS-8818971 to James E. Cutting. Results without modeling or simulations were reported briefly at the 28th annual meeting of the Psychonomic Society, Seattle, Washington, November 1987.We thank Dominic W. Massaro for helping us understand and implement the fuzzy-logical model of perception; Michael S. Landy and Mark J. Young for insights into implementing other models; James L. McClelland for a general discussion about modeling; William Epstein, James A. Ferwerda, and Mary M. Hayhoe for random discussions related to the topics presented here; Carol L. Krumhansl, Michael S. Landy, Geoffrey R. Loftus, Dominic W. Massaro, and an anonymous reviewer for comments on previous versions of this article; and Nan E. Karwan for sustained interest in and discussions about the project.Correspondence concerning this article should be addressed to James E. Cutting, Department of Psychology, Uris Hall, Cornell University, Ithaca, New York 14853-7601. model to fit data functions of interest with its ability to fit random da...

The Role of Representative Design in an Ecological Approach to Cognition.

“…Among these is James Cutting (e.g., Cutting, Bruno, Brady, & Moore, 1992;Cutting, Wang, Flückiger, & Baumberger, 1999). His results suggest, however, that there is little or no difference between these two sets of stimuli.…”

Perceiving affect from the voice and the face

“…An additive model of perception (AMP) is also tested against the results (Cutting, Bruno, Brady, & Moore, 1992;Massaro, 1988;. The evaluation stage is similar to the FLMp, but the values are added at the integration stage.…”

Expectancy in melody: Tests of children and adults.

“…Following Cutting et al (1992), we evaluated and compared the I-R and two-factor models on the basis of their simplicity, scope, and selectivity. Because the two-factor model has two parameters compared with the I-R model's five, the two-factor model is simpler.…”

“…Our first goal was to examine how melodic expectancies can be best explained. We compared two existing models of melodic expectancies on the basis of their simplicity, scope, and selectivity (Cutting, Bruno, Brady, & Moore, 1992). Following one of the basic rules of science, a simpler model is preferable to a more complex model.…”

“…To this end, we tested participants who varied in age: adults and children of two different ages in Experiment 1, and children of three different ages in Experiment 2. In addition to the criteria outlined by Cutting et al (1992), a good model of melodic expectancies should be able to describe, in a systematic manner, how such expectancies change over development. A third, ancillary goal was to investigate whether cognition and cognitive development in this area are music specific or reflective of general processes relevant to perception and cognition in other domains and modalities.…”