Examining stability of personality profile solutions between Likert-type and multidimensional forced choice measure

Lee, Philseok; Joo, Seang‐Hwane; Lee, Sunhee

doi:10.1016/j.paid.2019.01.022

Cited by 34 publications

(20 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, statement factor loadings were found to differ substantially between FC and SS formats (Ackerman, Donnellan, Roberts, & Fraley, 2016; Dueber, Love, Toland, & Turner, 2019; Guenole et al, 2018; Wetzel, Roberts, Fraley, & Brown, 2016), suggesting that respondents may interpret them differently (Bürkner, Schulte, & Holling, in press). While some have reported moderate to high convergent validity (Brown & Maydeu-Olivares, 2011, 2013; Lee, Joo, & Lee, 2019; Lee, Lee, & Stark, 2018; Usami et al, 2016), others have observed very low convergent validity (Anguiano-Carrasco, MacCann, Geiger, Seybert, & Roberts, 2015; Dueber et al, 2019; Seybert, 2013). Moreover, as mentioned above, evidence has been accumulating that shows ideal point models more accurately capture the response processes underlying various psychological measures.…”

Section: Psychometric Equivalence Between Fc and Ssmentioning

confidence: 99%

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

Zhang

Sun

Drasgow

et al. 2019

Organizational Research Methods

View full text Add to dashboard Cite

Forced choice (FC) measures are gaining popularity as an alternative assessment format to single statement (SS) measures due to their potential in reducing the impact of various response styles and faking. However, a fundamental question remains to be answered: do FC and SS instruments measure the same underlying constructs? In addition, FC measures are theorized to be more cognitively challenging, so how would this feature influence respondents' reactions to FC measures compared to SS? Two studies were designed to answer these questions. Study 1 results showed that FC measures scored by the Multi-unidimensional Pairwise Preference Model (MUPP) and SS measures scored with an ideal point model yielded similar factor structures and almost identical criterion-related validity across 12 criteria. Both formats also had similar pattern of marginal reliabilities and test-retest reliabilities. Study 1 findings were replicated in Study 2.In addition, we found strong evidence for convergent validity between the two formats. Though the FC format was perceived to be more difficult, respondents showed no differential preference and expressed similar level of emotional and cognitive reactions to the two formats.

show abstract

Section: Psychometric Equivalence Between Fc and Ssmentioning

confidence: 99%

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

Zhang

Sun

Drasgow

et al. 2019

Organizational Research Methods

View full text Add to dashboard Cite

show abstract

“…Both of these methods have been criticized (e.g., Paulhus and Vazire, 2007), not least since they remove variance related to personality, which decreases the inventory's construct validity and possibly also its criterion validity (McCrae and Costa, 1983). Recent advances in ipsative assessments, such as the multidimensional forced choice approaches (Brown and Maydeu-Olivares, 2013;Lee et al, 2019) have overcome some of the previous limitations and hold promise for the future. Although reliability-related questions still remain regarding the measurement of personality (see e.g., Seybert and Becker, 2019) which in turn may have consequences for validity, ipsative assessments may be a viable alternative for reducing socially desirable responding in some contexts.…”

Section: Introductionmentioning

confidence: 99%

The Properties and Utility of Less Evaluative Personality Scales: Reduction of Social Desirability; Increase of Construct and Discriminant Validity

Bäckström

Björklund

2020

Front. Psychol.

View full text Add to dashboard Cite

Evaluative neutralization implies rephrasing items such that it is less clear to the respondent what would be a desirable response in the given population. The current research compares evaluatively neutralized scales measuring the FFM model with standard counterparts. Study 1 reveals that evaluatively neutralized scales are less influenced by social desirability. Study 2 estimates higher-order factor models for neutralized vs. standard five-factor scales. In contrast to standard inventories, there was little support for higher-order factors for neutralized scales. Study 3 demonstrates the convergent and discriminant validity for the neutralized scales, e.g., by less inflated correlations to external measures. It is argued that evaluatively neutralized inventories help researchers come to grips with social desirability in personality measurement, and are particularly useful when the factor structure is central to the research question and there is a focus on discriminant validity.

show abstract

“…For example, Hauenstein, Bradley, O'Shea, Shah, and Magill () used a think‐aloud technique to examine the cognitive processes during explicitly faking good and a simulated job application scenario; they found the response process were qualitatively distinct. Meanwhile, researchers found that despite responded under different contexts (i.e., real application or simulated application), the same three types of respondents were identified in empirical studies (Lee, Joo, & Lee, ; McLarnon, DeLongchamp, & Schneider, ; Robie, Brown, & Beaty, ; Zickar et al., ). These pieces of evidence suggest that there may be less difference between the two scenarios than previously believed.…”

Section: Discussionmentioning

confidence: 88%

Examining the item response process to personality measures in high‐stakes situations: Issues of measurement validity and predictive validity

Zhang

Cao

Tay

et al. 2019

Personnel Psychology

View full text Add to dashboard Cite

We conducted two experimental studies with between-subjects and within-subjects designs to investigate the item response process for personality measures administered in high-versus low-stakes situations. Apart from assessing measurement validity of the item response process, we examined predictive validity; that is, whether or not different response models entail differential selection outcomes. We found that ideal point response models fit slightly better than dominance response models across high-versus low-stakes situations in both studies. Additionally, fitting ideal point models to the data led to fewer items displaying differential item functioning compared to fitting dominance models. We also identified several items that functioned as intermediate items in both the faking and honest conditions when ideal point models were fitted, suggesting that ideal point model is "theoretically" more suitable across these contexts for personality inventories. However, the use of different response models (dominance vs. ideal point) did not have any substantial impact on the validity of personality measures in high-stakes situations, or the effectiveness of selection decisions such as mean performance or percent of fakers selected. These findings are significant in that although prior research supports the importance and use of ideal point models for measuring personality, we find that in the case of personality faking, though ideal point models seem to have slightly better measurement validity, the use of dominance models may be adequate with no loss to predictive validity. K E Y W O R D S dominance model, faking, ideal point model, item response process, personality assessment Personnel Psychology. 2020;73:305-332.wileyonlinelibrary.com/journal/peps

show abstract

Examining stability of personality profile solutions between Likert-type and multidimensional forced choice measure

Cited by 34 publications

References 32 publications

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

The Properties and Utility of Less Evaluative Personality Scales: Reduction of Social Desirability; Increase of Construct and Discriminant Validity

Examining the item response process to personality measures in high‐stakes situations: Issues of measurement validity and predictive validity

Contact Info

Product

Resources

About