The Item Response Warehouse (IRW), Version 4.0

Domingue, Benjamin W.; Kanopka, Klint

doi:10.31234/osf.io/7bd54

Cited by 3 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is because the duration required to respond to a question on a psychological scale tends to be short, making it challenging to set a threshold that distinguishes careless and attentive responses since there may be relatively less of a difference between responses of the two types (Meade & Craig, 2012). For instance, using 12 personality and emotion scale (e.g., Goldberg, 1992;Lovibond and Lovibond, 1995) datasets from the Item Response Warehouse (Domingue et al, 2023), we examine the timing of 53,671,504 responses.…”

Section: Advantages Of Jointly Considering Item Responses and Respons...mentioning

confidence: 99%

“…An empirical investigation was conducted to demonstrate the real-world application of the proposed model. The data was chosen from the item response warehouse (Domingue et al, 2023), comprising 1,015,341 samples from the emotional stability dimension of the Big-Five personality scale (Goldberg, 1992). The emotional stability sub-scale is assessed through ten items, with eight being phrased negatively, and participants are asked to rate how well each item describes them on a five-point Likert scale (1 = Very Inaccurate, 5 = Very Accurate).…”

Section: Empirical Illustrationmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Factor Mixture Modeling with Response Time for Detecting Careless Respondents

Zhang,

Ulitzsch,

Domingue

2024

Preprint

View full text Add to dashboard Cite

Careless respondents inject noise into data which can distort research findings and compromise model fit. To address this, factor mixture modeling (FMM) has been widely used to identify careless respondents. Traditionally, researchers have relied on reverse-worded questions in FMM to facilitate the detection of careless responding. With the rise of online data collection platforms, response time has appeal as a means for detecting careless respondents. We introduce a Bayesian factor mixture model that utilizes response time to identify careless respondents. By jointly modeling responses and response time, this approach effectively identifies careless individuals exhibiting rapid and random response styles. Through simulation studies, we found that: (1) the proposed model achieves high estimation accuracy of key model parameters (i.e., loadings and intercepts); (2) it demonstrates high accuracy and sensitivity in correctly classifying respondents as either attentive or careless; and (3) it maintains classification error rates at an acceptable level. An additional benefit is that incorporating response time into the model enhances model convergence as well as accuracy of classification and estimation. An empirical study tests the applicability of the proposed model in real-world scenarios, comparing its performance to the traditional method based on reverse-worded questions. The results underscore the practical advantages of enriching FMM with collateral response time information in excluding careless responses and improving data quality.

show abstract

Section: Advantages Of Jointly Considering Item Responses and Respons...mentioning

confidence: 99%

Section: Empirical Illustrationmentioning

confidence: 99%

Bayesian Factor Mixture Modeling with Response Time for Detecting Careless Respondents

Zhang,

Ulitzsch,

Domingue

2024

Preprint

View full text Add to dashboard Cite

show abstract

The InterModel Vigorish as a Lens for Understanding (and Quantifying) the Value of Item Response Models for Dichotomously Coded Items

Domingue,

Kanopka,

Kapoor

et al. 2024

Psychometrika

Self Cite

View full text Add to dashboard Cite

The deployment of statistical models, such as those used in item response theory (IRT), necessitates the use of indices that are informative about the degree to which a given model is appropriate for a specific data context. We introduce the InterModel Vigorish (IMV) as an index that can be used to quantify accuracy for models of dichotomous item responses based on the improvement across two sets of predictions (i.e., predictions from two item response models or predictions from a single such model relative to prediction based on the mean). This index has a range of desirable features: it can be used for the comparison of non-nested models and its values are highly portable and generalizable. We use this fact to compare predictive performance across a variety of simulated data contexts and also demonstrate qualitative differences in behavior between the IMV and other common indices (e.g., the AIC and RMSEA). We also illustrate the utility of the IMV in empirical applications with data from 89 dichotomous item response datasets. These empirical applications help illustrate how the IMV can be used in practice and substantiate our claims regarding various aspects of model performance. These findings indicate that the IMV may be a useful indicator in psychometrics especially as it allows for easy comparison of predictions across a variety of contexts.

show abstract

Item-level heterogeneous treatment effects of selective serotonin reuptake inhibitors (SSRIs) on depression: implications for inference, generalizability, and identification

Gilbert,

Hieronymus,

Eriksson

et al. 2024

Epidemiologic Methods

View full text Add to dashboard Cite

Objectives In analysis of randomized controlled trials (RCTs) with patient-reported outcome measures (PROMs), Item Response Theory (IRT) models that allow for heterogeneity in the treatment effect at the item level merit consideration. These models for “item-level heterogeneous treatment effects” (IL-HTE) can provide more accurate statistical inference, allow researchers to better generalize their results, and resolve critical identification problems in the estimation of interaction effects. In this study, we extend the IL-HTE model to polytomous data and apply the model to determine how the effect of selective serotonin reuptake inhibitors (SSRIs) on depression varies across the items on a depression rating scale. Methods We first conduct a Monte Carlo simulation study to assess the performance of the polytomous IL-HTE model under a range of conditions. We then apply the IL-HTE model to item-level data from 24 RCTs measuring the effect of SSRIs on depression using the 17-item Hamilton Depression Rating Scale (HDRS-17) and estimate heterogeneity by subscale (HDRS-6). Results Our simulation results show that ignoring IL-HTE can yield standard errors that are as much as 50 % too small and create significant bias in treatment by covariate interaction effects when item-specific treatment effects are correlated with item location, and that the application of the IL-HTE model resolves these issues. Our empirical application shows that while the average effect of SSRIs on depression is beneficial (i.e., negative) and statistically significant, there is substantial IL-HTE, with estimates of the standard deviation of item-level effects nearly as large as the average effect. We show that this substantial IL-HTE is driven primarily by systematically larger effects on the HDRS-6 subscale items. Conclusions The IL-HTE model has the potential to provide new insights for the inference, generalizability, and identification of treatment effects in clinical trials using PROMs.

show abstract

The Item Response Warehouse (IRW), Version 4.0

Cited by 3 publications

References 15 publications

Bayesian Factor Mixture Modeling with Response Time for Detecting Careless Respondents

Bayesian Factor Mixture Modeling with Response Time for Detecting Careless Respondents

The InterModel Vigorish as a Lens for Understanding (and Quantifying) the Value of Item Response Models for Dichotomously Coded Items

Item-level heterogeneous treatment effects of selective serotonin reuptake inhibitors (SSRIs) on depression: implications for inference, generalizability, and identification

Contact Info

Product

Resources

About