In experimental science, it is desirable to hold all factors constant except those intentionally manipulated. In psychology, however, this ideal is often not possible. Elements such as participants and items vary, in addition to the intended factors. For example, a researcher interested in the psychology of reading might manipulate the part of speech and observe reading times. In this case, there is unintended variability from the selection of both participants and items. In his classic article, "The Languageas-Fixed-Effect Fallacy: A Critique of Language Statistics in Psychological Research," H. H. Clark (1973) discussed how unintended variability from the simultaneous selection of participants and items leads to underestimation of confidence intervals and inflation of Type I error rates in conventional analysis. Type I error rate inflation, or an increased tendency to find a significant effect when none exists, is highly undesirable.To demonstrate the problem, consider the question of whether nouns and verbs are read at the same rate. To answer this question, a researcher could randomly select suitable verbs and nouns and ask a number of participants to read them. Each participant produces a set of reading time scores for both nouns and verbs. A common approach is to tabulate for each participant one mean reading time for nouns and another for verbs. To test the hypothesis of the equality of reading rates, these pairs of mean reading times may be submitted to paired t tests. This analytic approach is often used in memory research. For example, Riefer and Rouder (1992) used this analysis to determine whether bizarre sentences are better remembered than common ones. Clark (1973), however, argued that using t tests to analyze means tabulated across different items leads to Type I error rate inflation.In the following demonstration, we show by simulation that this inflation is not only real, but also surprisingly large. We generate data for a standard ANOVA-style model (discussed below) with no part-of-speech effects. We analyze these data by first computing participant means for each part of speech and then submitting these means to a paired t test. This process is performed repeatedly, and the proportion of significant results is reported. If the test has no Type I error inflation, the proportion should be the nominal Type I error rate, which is set to the conventional value of .05.Consider the following ANOVA-style model for nouns: It is reasonable to expect that each participant has a unique effect on reading time; some participants are fast at reading, but others are slow. This effect for the ith participant 573Copyright 2005 Psychonomic Society, Inc. Although many nonlinear models of cognition have been proposed in the past 50 years, there has been little consideration of corresponding statistical techniques for their analysis. In analyses with nonlinear models, unmodeled variability from the selection of items or participants may lead to asymptotically biased estimation. This asymptotic bias, in turn, ren...
We present a statistical model for inference with response time (RT) distributions. The model has the following features. First, it provides a means of estimating the shape, scale, and location (shift) of RT distributions. Second, it is hierarchical and models between-subjects and within-subjects variability simultaneously. Third, inference with the model is Bayesian and provides a principled and efficient means of pooling information across disparate data from different individuals. Because the model efficiently pools information across individuals, it is particularly well suited for those common cases in which the researcher collects a limited number of observations from several participants. Monte Carlo simulations reveal that the hierarchical Bayesian model provides more accurate estimates than several popular competitors do. We illustrate the model by providing an analysis of the symbolic distance effect in which participants can more quickly ascertain the relationship between nonadjacent digits than that between adjacent digits. THEORETICAL AND REVIEW ARTICLES
Bayesian analysis, hierarchical models, response time, MCMC, Weibull distribution,
In fitting the process-dissociation model (L. L. Jacoby, 1991) to observed data, researchers aggregate outcomes across participant, items, or both. T. Curran and D. L. Hintzman (1995) demonstrated how biases from aggregation may lead to artifactual support for the model. The authors develop a hierarchical process-dissociation model that does not require aggregation for analysis. Most importantly, the Curran and Hintzman critique does not hold for this model. Model analysis provides for support of process dissociation--selective influence holds, and there is a dissociation in correlation patterns among participants and items. Items that are better recollected also elicit higher automatic activation. There is no correlation, however, across participants; that is, participants with higher recollection have no increased tendency toward automatic activation. The critique of aggregation is not limited to process dissociation. Aggregation distorts analysis in many nonlinear models, including signal detection, multinomial processing tree models, and strength models. Hierarchical modeling serves as a general solution for accurately fitting these psychological-processing models to data.
recognition memory, theory of signal detection, Bayesian models, hierarchical models, MCMC methods,
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.