Impact of Elaboration on Responding to Situational Judgment Test Items

Lievens, Filip; Peeters, Helga

doi:10.1111/j.1468-2389.2008.00440.x

Cited by 16 publications

(17 citation statements)

References 67 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Doing so provides estimates that underrepresent the reliability of the measure. When researchers find low internal consistency in their study, they fall back on stating that this is expected for heterogeneous content that is factorially complex and generally perform no further analysis (e.g., Lievens & Peeters, ; Peeters & Lievens, ).…”

Section: Introductionmentioning

confidence: 99%

Assessing the Reliability of Situational Judgment Tests Used in High‐Stakes Situations

Catano

Brochu

Lamerson

2012

Int J Selection Assessment

View full text Add to dashboard Cite

Assessing reliability of situational judgment tests (SJTs) in high‐stakes situations is problematic with reliability inappropriately measured by Cronbach's alpha when test items are heterogeneous. We computed the corrected, weighted mean alpha from 56 alpha coefficients, which produced a value of α = .46 and reviewed appropriate types of reliability to use with SJTs. In the current longitudinal study, SJT test–retest reliability was r = .82, compared with internal consistency, α = .46, and stratified alpha, α = .45 at Time 1 and α = .52 and stratified α = .51 at Time 2. We used a student sample (Time 1: n = 185; Time 2: n = 132) with items from a credentialing exam with ‘should do’ instructions. The SJT correlated significantly with cognitive ability, r = .30, and agreeableness, r = .24. In Study 2, we assessed test–retest reliability with Human Resource professionals (Time 1: n = 94; Time 2: n = 32) who had been recently credentialed and who participated in a pilot test of new SJT items with ‘most likely/least likely do’ response options. The SJT test–retest reliability was r = .66 compared with internal consistency, α = .43 and stratified α = .47 at Time 1 and α = .61 and stratified α = .67 at Time 2. We discuss the theoretical implications of the Study 1 results as well as the practical implications for use of SJTs in credentialing examinations.

show abstract

Section: Introductionmentioning

confidence: 99%

Assessing the Reliability of Situational Judgment Tests Used in High‐Stakes Situations

Catano

Brochu

Lamerson

2012

Int J Selection Assessment

View full text Add to dashboard Cite

show abstract

“…Several faking studies on SJTs attempted to reduce faking (Lievens & Peeters, 2008;Oostrom et al, 2017). The most common approach is asking individuals what they should do (i.e., knowledge instructions) as opposed to asking individuals what they would actually do (i.e., behavioral tendency instructions) (Nguyen et al, 2005).…”

Section: Faking On Sjtsmentioning

confidence: 99%

“…Unlike the extended research on faking on personality tests, the number of published studies on faking on SJTs is limited (Table ). As with personality tests, lab studies showed that individuals are able to obtain higher SJT scores if they are instructed to fake (Lievens & Peeters, ; Nguyen, Biderman, & McDaniel, ; Oostrom, Köbis, Ronay, & Cremers, ; Peeters & Lievens, ). The size of the faking effects seems to depend on the order in which the fake and honest conditions are presented to the respondent.…”

Section: Introductionmentioning

confidence: 99%

Faking on a situational judgment test in a medical school selection setting: Effect of different scoring methods?

Leng

Stegers‐Jager

Born

et al. 2019

Int J Selection Assessment

View full text Add to dashboard Cite

We examined the occurrence of faking on a rating situational judgment test (SJT) by comparing SJT scores and response styles of the same individuals across two naturally occurring situations. An SJT for medical school selection was administered twice to the same group of applicants (N = 317) under low‐stakes (T1) and high‐stakes (T2) circumstances. The SJT was scored using three different methods that were differentially affected by response tendencies. Applicants used significantly more extreme responding on T2 than T1. Faking (higher SJT score on T2) was only observed for scoring methods that controlled for response tendencies. Scoring methods that do not control for response tendencies introduce systematic error into the SJT score, which may lead to inaccurate conclusions about the existence of faking.

show abstract

“…Response distortion or faking can attenuate the criterion‐related validity of tests (Peeters & Lievens, 2005), negatively affect construct validity (Stark, Chernyshenko, Chan, Lee, & Drasgow, 2001), and affect who is hired (Rosse et al., 1998). As such, scholars have sought to control job applicants’ tendencies to inflate their responses through both reactive and proactive approaches (Lievens & Peeters, 2008; Vasilopoulos, Cucina, & McElreath, 2005).…”

mentioning

confidence: 99%

“…To date, four empirical studies have been conducted to test the usefulness of this technique. Three of these studies investigated its impact on biodata responses (Ramsay, Schmitt, Oswald, Kim, & Gillespie, 2006; Schmitt & Kunce, 2002; Schmitt et al., 2003), and one study investigated its impact on situational judgment test responses (Lievens & Peeters, 2008). The reported results provide initial empirical support for the usefulness of RET and show that requiring elaboration for biodata items reduces test scores.…”

mentioning

confidence: 99%

Tell Me Some More: Exploring How Verbal Ability and Item Verifiability Influence Responses to Biodata Questions in a High‐stakes Selection Context

Levashina

Morgeson

Campion

2012

Personnel Psychology

View full text Add to dashboard Cite

Although self-rated or self-scored selection measures are commonly used in selection contexts, they are potentially susceptible to applicant response distortion or faking. The response elaboration technique (RET), which requires job applicants to provide supporting information to justify their responses, has been identified as a potential way to minimize applicant response distortion. In a large-scale, high-stakes selection context (N = 16,304), we investigate the extent to which RET affects responding on a biodata test as well as the underlying reasons for any potential effect. We find that asking job applicants to elaborate their responses leads to overall lower scores on a biodata test. Item verifiability affects the extent to which RET decreases faking, which we suggest is due to increased accountability. In addition, verbal ability was more strongly related to biodata item scores when items require elaboration, although the effect of verbal ability was small. The implications of these findings for reducing faking in personnel selection are delineated.There are a variety of different methods that could be used to collect information about job applicants in order to make selection decisions. Some measures (e.g., cognitive ability measures, job knowledge tests) have objectively correct answers and ask applicants to demonstrate what is being measured. Other measures (e.g., job interviews, assessment centers) have answers judged subjectively by third parties (such as interviewers or assessors) where applicants are asked to either describe or demonstrate what is being measured. Another class of measures (e.g., personality tests, biodata measures) has answers rated subjectively by the applicants (i.e., self-ratings) where applicants are asked to self-assess, self-rate, or selfscore on what is being measured.

show abstract

Impact of Elaboration on Responding to Situational Judgment Test Items

Cited by 16 publications

References 67 publications

Assessing the Reliability of Situational Judgment Tests Used in High‐Stakes Situations

Assessing the Reliability of Situational Judgment Tests Used in High‐Stakes Situations

Faking on a situational judgment test in a medical school selection setting: Effect of different scoring methods?

Tell Me Some More: Exploring How Verbal Ability and Item Verifiability Influence Responses to Biodata Questions in a High‐stakes Selection Context

Contact Info

Product

Resources

About