Empirical validation of the diffusion model for recognition memory and a comparison of parameter-estimation methods

Arnold, Nina R.; Bröder, Arndt; Bayen, Ute J.

doi:10.1007/s00426-014-0608-y

Cited by 72 publications

(98 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the one hand, Ratcliff (2002) and Wagenmakers, Ratcliff, Gomez, and McKoon (2008) used brightness discrimination and lexical discrimination tasks, respectively, to demonstrate both forms of validity in accumulation rate, starting point, and boundary separation parameters. However, in a recent study, Arnold, Bröder, and Bayen (2015) found evidence for convergent, but not discriminant validity of the Full Diffusion model parameters in the domain of recognition memory. For example, they found that their manipulation of response caution had a large effect on the boundary separation parameter, but also on the accumulation rate and non-decision time parameters.…”

Section: Threats To the Validity Of Cognitive Modelsmentioning

confidence: 90%

“…For example, Arnold et al (2015) found that the EZ diffusion model, unlike the Full Diffusion model, showed both convergent and discriminant validity. Donkin, Brown, Heathcote, and Wagenmakers (2011) re-analyzed the lexical decision data set from Wagenmakers et al (2008) using both the Full Diffusion and LBA models.…”

Section: Threats To the Validity Of Cognitive Modelsmentioning

confidence: 99%

“…Arnold et al, 2015). This benefit likely comes about because the EZ2 method uses robust statistics, such as means and variances, to estimate parameters.…”

Section: Summary and Recommendationsmentioning

confidence: 99%

See 2 more Smart Citations

The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models

Dutilh¹,

Annis²,

Brown³

et al. 2016

Preprint

View full text Add to dashboard Cite

Most data analyses rely on models. To complement statistical models, psychologists have developed cognitive models, which translate observed variables into psychologically interesting constructs. Response time models, in particular, assume that response time and accuracy are the observed expression of latent variables including 1) ease of processing, 2) response caution, 3) response bias, and 4) nondecision time. Inferences about these psychological factors, hinge upon the validity of the models' parameters. Here, we use a blinded, collaborative approach to assess the validity of such model-based inferences. Seventeen teams of researchers analyzed the same 14 data sets. In each of these two-condition data sets, we manipulated properties of participants' behavior in a two-alternative forced choice task. The contributing teams were blind to the manipulations, and had to infer what aspect of behavior was changed using their method of choice. The contributors chose to employ a variety of models, estimation methods, and inference procedures. Our results show that, although conclusions were similar across different methods, these "modeler's degrees of freedom" did affect their inferences. Interestingly, many of the simpler approaches yielded as robust and accurate inferences as the more complex methods. We recommend that, in general, cognitive models become a typical analysis tool for response time data. In particular, we argue that the simpler models and procedures are sufficient for standard experimental designs. We finish by outlining situations in which more complicated models and methods may be necessary, and discuss potential pitfalls when interpreting the output from response time models.

show abstract

Section: Threats To the Validity Of Cognitive Modelsmentioning

confidence: 90%

Section: Threats To the Validity Of Cognitive Modelsmentioning

confidence: 99%

See 1 more Smart Citation

The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models

Dutilh¹,

Annis²,

Brown³

et al. 2016

Preprint

View full text Add to dashboard Cite

show abstract

“…In any diffusion model study, the probably most important issue that a researcher has to examine is validity of parameters. In recent years, several experimental validation studies (e.g., Arnold, Bröder, & Bayen, 2015;Voss et al, 2004;Wagenmakers, Ratcliff, et al, 2008) and correlational analyses (e.g., Ratcliff, Thapar, & McKoon, 2011;Schubert, Hagemann, Voss, Schankin, & Bergmann, 2015) have supplied promising results regarding parameter validity. However, for any new paradigm, the validity has to be first examined.…”

Section: Discussionmentioning

confidence: 99%

How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria

2016

View full text Add to dashboard Cite

Diffusion models (Ratcliff, 1978) make it possible to identify and separate different cognitive processes underlying responses in binary decision tasks (e.g., the speed of information accumulation vs. the degree of response conservatism). This becomes possible because of the high degree of information utilization involved. Not only mean response times or error rates are used for the parameter estimation, but also the response time distributions of both correct and error responses. In a series of simulation studies, the efficiency and robustness of parameter recovery were compared for models differing in complexity (i.e., in numbers of free parameters) and trial numbers (ranging from 24 to 5,000) using three different optimization criteria (maximum likelihood, Kolmogorov-Smirnov, and chi-square) that are all implemented in the latest version of fast-dm (Voss, Voss, & Lerche, 2015). The results revealed that maximum likelihood is superior for uncontaminated data, but in the presence of fast contaminants, Kolmogorov-Smirnov outperforms the other two methods. For most conditions, chi-square-based parameter estimations lead to less precise results than the other optimization criteria. The performance of the fast-dm methods was compared to the EZ approach (Wagenmakers, van der Maas, & Grasman, 2007) and to a Bayesian implementation (Wiecki, Sofer, & Frank, 2013). Recommendations for trial numbers are derived from the results for models of different complexities. Interestingly, under certain conditions even small numbers of trials (N < 100) are sufficient for robust parameter estimation.

show abstract

“…Experimental validation studies have been conducted supporting the validity of some or all of these four parameters using a color-discrimination task (Voss et al, 2004), a lexical-decision task (Wagenmakers, Ratcliff, Gomez, & McKoon, 2008), a motion discrimination task (Ratcliff & McKoon, 2008), brightness discrimination tasks (Ratcliff & Rouder, 1998) and-recently-also a recognition memory task (RMT; Arnold, Bröder, & Bayen, 2015). Furthermore, in correlational studies relationships between parameters and external criteria such as intelligence have been observed (e.g., Ratcliff et al, 2011).…”

Section: Psychological Interpretation Of the Diffusion Model Parametersmentioning

confidence: 99%

Retest reliability of the parameters of the Ratcliff diffusion model

Lerche

Voss

2016

Psychological Research

View full text Add to dashboard Cite

In the recent years, there is a growing interest to use the Ratcliff Diffusion Model (1978) for diagnostic purposes as the parameters of the model capture interindividual differences in specific cognitive processes. The parameters are estimated using reaction time data from binary classification tasks. For a potential diagnostic application of parameter values sufficient reliability is a necessary precondition. In two studies, each with two sessions separated by 1 week, the retest reliability of the diffusion model parameters was assessed. In Study 1, 105 participants completed a lexical decision task and a recognition memory task. In Study 2, 128 participants worked on an associative priming task. Results show that the reliability of the main parameters of the Ratcliff Diffusion Model (in particular of the speed of information accumulation and the threshold separation with rs > 0.70 for all three tasks) is satisfying. Besides, we analyzed the influence of the number of trials on the retest reliability using different estimation methods (Kolmogorov-Smirnov, Maximum Likelihood, Chi-square and EZ) and both empirical and simulated datasets.

show abstract

Empirical validation of the diffusion model for recognition memory and a comparison of parameter-estimation methods

Cited by 72 publications

References 69 publications

The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models

The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models

How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria

Retest reliability of the parameters of the Ratcliff diffusion model

Contact Info

Product

Resources

About