Controlling Acquiescence Bias with Multidimensional IRT Modeling

Primi, Ricardo; Filho, Nelson Hauck; Valentini, Felipe; Santos, Daniel; Falk, Carl F.

doi:10.1007/978-3-030-01310-3_4

Cited by 8 publications

(10 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…By contrast, and consistent with the work of Ferrando et al (2011), the FCQ-based scores did not correlate with the ACQ, suggesting that they are more robust to this type of response bias. The adverse GSQ findings differ from the results presented by Primi et al (2017), in which a negligible correlation (i.e., 0.03) was found between Acquiescence and the domain scores after the correction with the Random Intercept model. The greater unbalanced nature of our scales might explain our pattern of results.…”

Section: Discussioncontrasting

confidence: 99%

“…The greater unbalanced nature of our scales might explain our pattern of results. Although Primi et al (2017) also used an unbalanced questionnaire in terms of polarity, the ratio between the positively and negatively keyed items in their study, i.e., 2 was lower than the one in our study which was 3. Note that in this extreme case examinees with high scores in all the traits might be less easily distinguishable from the examinees with high levels of acquiescence.…”

Section: Discussioncontrasting

confidence: 72%

See 1 more Smart Citation

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

et al. 2019

View full text Add to dashboard Cite

One important problem in the measurement of non-cognitive characteristics such as personality traits and attitudes is that it has traditionally been made through Likert scales, which are susceptible to response biases such as social desirability (SDR) and acquiescent (ACQ) responding. Given the variability of these response styles in the population, ignoring their possible effects on the scores may compromise the fairness and the validity of the assessments. Also, response-style-induced errors of measurement can affect the reliability estimates and overestimate convergent validity by correlating higher with other Likert-scale-based measures. Conversely, it can attenuate the predictive power over non-Likert-based indicators, given that the scores contain more errors. This study compares the validity of the Big Five personality scores obtained: (1) ignoring the SDR and ACQ in graded-scale items (GSQ), (2) accounting for SDR and ACQ with a compensatory IRT model, and (3) using forced-choice blocks with a multi-unidimensional pairwise preference model (MUPP) variant for dominance items. The overall results suggest that ignoring SDR and ACQ offered the worst validity evidence, with a higher correlation between personality and SDR scores. The two remaining strategies have their own advantages and disadvantages. The results from the empirical reliability and the convergent validity analysis indicate that when modeling social desirability with graded-scale items, the SDR factor apparently captures part of the variance of the Agreeableness factor. On the other hand, the correlation between the corrected GSQ-based Openness to Experience scores, and the University Access Examination grades was higher than the one with the uncorrected GSQ-based scores, and considerably higher than that using the estimates from the forced-choice data. Conversely, the criterion-related validity of the Forced Choice Questionnaire (FCQ) scores was similar to the results found in meta-analytic studies, correlating higher with Conscientiousness. Nonetheless, the FCQ-scores had considerably lower reliabilities and would demand administering more blocks. Finally, the results are discussed, and some notes are provided for the treatment of SDR and ACQ in future studies.

show abstract

Section: Discussioncontrasting

confidence: 99%

Section: Discussioncontrasting

confidence: 72%

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

et al. 2019

View full text Add to dashboard Cite

show abstract

“…However, by controlling the response bias the effect is reversed. Therefore, our study supports the literature that points out at the importance of using negative items in scales (Danner, Aichholzer, & Rammstedt, 2015;Primi, Hauck-Filho, Valentini, Santos, & Falk, 2019;Rammstedt & Farmer, 2013;Valentini, 2017). In this connection, perhaps answering Likert item is more complicated than it sounds and it is more cognitive demanding than we think in common sense.…”

Section: Discussionsupporting

confidence: 88%

Development of the Inventory of Supporting for Socio-Emotional Skills, evidence of internal structure controlling for acquiescence

Valentini

Mose

Ramos

et al. 2020

Estud. psicol. (Campinas)

Self Cite

View full text Add to dashboard Cite

This study aims to develop the Inventory of Supporting for Socio-Emotional Skills, which is an instrument for assessing the support for socio-emotional development provided by teachers, rated by students. It also aims to gather validity evidence of the internal structure of Inventory of Supporting for Socio-Emotional Skills and to control acquiescence. We elaborated the items, which content were evaluated by six experts. For the internal structure, a sample composed of elementary and high-school students (N = 443) was used to answer the instrument. The instrument presented good psychometric parameters after controlling for acquiescence using the Random Intercept Model. The final structure was set as unidimensional. We suggest controlling acquiescence to avoid biased scores. In addition, we make an inventory available that can be helpful for socio-emotional skills development.

show abstract

“…Since acquiescence varies across students, it may influence inter-item correlations confounding the correlations that we expect to be caused by the latent dimensions we are trying to measure. This systematic confounding suppresses correlations of items of opposite poles and inflates correlations between items assessing the same pole (++ and −−, Maydeu-Olivares and Steenkamp, 2018 ; Mirowsky and Ross, 1991 ; Primi et al, 2020 ; Primi et al, 2019b ).…”

Section: Discussionmentioning

confidence: 99%

“…Finally, given the anticipated heterogeneous and young respondent samples it is critically important to think about ways to reduce systematic error due to response styles that can compromise structural and predictive validity ( Primi et al, 2019a , b , d , 2020 ). Soto et al (2008) observed that psychometric and structural analyses of personality descriptive items of younger and less-educated samples rarely resemble the better structural validities found in adults and well-educated samples.…”

Section: Introductionmentioning

confidence: 99%

SENNA Inventory for the Assessment of Social and Emotional Skills in Public School Students in Brazil: Measuring Both Identity and Self-Efficacy

et al. 2021

Self Cite

View full text Add to dashboard Cite

Responding to the need for school-based, broadly applicable, low-cost, and brief assessments of socio-emotional skills, we describe the conceptual background and empirical development of the SENNA inventory and provide new psychometric information on its internal structure. Data were obtained through a computerized survey from 50,000 Brazilian students enrolled in public school grades 6 to 12, spread across the entire State of São Paulo. The SENNA inventory was designed to assess 18 particular skills (e.g., empathy, responsibility, tolerance of frustration, and social initiative), each operationalized by nine items that represent three types of items: three positively keyed trait-identity items, three negatively keyed identity items, and three (always positively keyed) self-efficacy items, totaling a set of 162 items. Results show that the 18 skill constructs empirically defined a higher-order structure that we interpret as the social-emotional Big Five, labeled as Engaging with Others, Amity, Self-Management, Emotional Regulation, and Open-Mindedness. The same five factors emerged whether we assessed the 18 skills with items representing (a) a trait-identity approach that emphasizes lived skills (what do I typically do?) or (b) a self-efficacy approach that emphasizes capability (how well can I do that?). Given that its target youth group is as young as 11 years old (grade 6), a population particularly prone to the response bias of acquiescence, SENNA is also equipped to correct for individual differences in acquiescence, which are shown to systematically bias results when not corrected.

show abstract

Controlling Acquiescence Bias with Multidimensional IRT Modeling

Cited by 8 publications

References 32 publications

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Controlling for Response Biases in Self-Report Scales: Forced-Choice vs. Psychometric Modeling of Likert Items

Development of the Inventory of Supporting for Socio-Emotional Skills, evidence of internal structure controlling for acquiescence

SENNA Inventory for the Assessment of Social and Emotional Skills in Public School Students in Brazil: Measuring Both Identity and Self-Efficacy

Contact Info

Product

Resources

About