Recent theorizing and empirical evidence suggesting that Situational Judgment Tests (SJTs) are more context-independent than previously thought has sparked a debate about the role of situation descriptions in SJTs. To contribute to this debate and add to our understanding of how SJTs work, this article conceptually embeds SJT performance in a situation construal model and examines the effects of situation descriptions on the construct saturation and predictive validity of SJT scores, as well as on applicant perceptions. Across two studies (N ϭ 1,092 and 578) and different SJTs, personality and cognitive ability were equally important determinants of SJT performance regardless of whether situation descriptions were presented or omitted. The effects of removing situation descriptions on the criterionrelated validity of SJT scores differed depending on the breadth of the criteria. For predicting global job performance criteria (in-role performance and organizational citizenship behavior), SJT validity was not significantly affected, whereas it decreased for predicting more specific criteria (interpersonal adaptability, efficacy for teamwork). Finally, the effects of omitting situation descriptions in SJTs on applicant perceptions were either negligible or small. Implications for SJT theory, research, and design are discussed.
Recent research challenges the importance of situation descriptions for situational judgment test (SJT) performance. This study contributes to resolving the ongoing debate on whether SJTs are situational measures, by incorporating findings on person × situation interactions into SJT research. Specifically, across three studies (N Total = 1,239), we first tested whether situation construal (i.e., the individual perception of situations in SJTs) predicts responses to SJT items. Second, we assessed whether the relevance of situation construal for SJT performance depends on test elements (i.e., situation descriptions and response options) and item features (i.e., description-dependent vs. description-independent SJT items). Lastly, we determined whether situation construal has incremental validity for job-related criteria over and above SJT performance. The results showed that, for most SJT items, situation construal significantly contributed to SJT performance, even if only response options were available. This was also true for SJT items that are significantly more difficult to solve when situation descriptions are omitted (i.e., description-dependent SJT items). Finally, situation construal explained variance in relevant criteria over and above SJT performance. Despite recent efforts to reconceptualize SJTs, our results suggest that they can still be viewed as situational measures. However, situation descriptions may be less crucial for these underlying This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Objective The symmetry principle and the frame‐of‐reference perspective have each made contributions to improving the measurement of personality. Although each perspective is valuable in its own right, we argue that even greater improvement can be achieved through the combination of both. Therefore, the goal of the current article was to show the value of a combined lens‐model and frame‐of‐reference perspective. Method We conducted a literature review to summarize relevant research findings that shed light on the interplay of both perspectives and developed an integrative model. Results Based on the literature review and on theoretical grounds, we argue that a basic premise of the frame‐of‐reference literature––that personality items are open to interpretation and allow individuals to impose their own contextual framings––should be considered from a symmetry perspective. Unintended context‐specificity in items may “spread” to personality facets and domains, and thus, impact the symmetry of personality measures with other criteria. As the individuals´ frames‐of‐reference and (a)symmetric relationships are not always apparent, we term them as “hidden.” Conclusions The proposed combination of lens‐model and frame‐of‐reference perspectives provides further insights into current issues in personality research and uncovers important avenues for future research.
Recent research has shown that many text-based situational judgment test (SJT) items can be solved even when the situational descriptions in the item stems are not presented to test takers. This finding challenges the traditional view of SJTs as low-fidelity simulations that rely on 'situational' (context-dependent) judgment. However, media richness theory and construal level theory suggest that situation descriptions presented in a richer and more concrete format (video format) will reduce uncertainty about inherent requirements and facilitate the perception that the situation is taking place in the here and now. Therefore, we hypothesized that situational judgment would be more important in video situation descriptions than in text situation descriptions. We adapted a leadership SJT to realize a 3 (situation description in the item stem: video vs. text vs. none) 9 2 (response format: video response options vs. text response options) between-subjects design (N = 279). Participants were randomly assigned to one of the six conditions. The removal of videobased situation descriptions in item stems led to an equivalent decrease in SJT scores as the removal of text-based situation descriptions in item stems (video-based version: Cohen's d = 0.535 vs. text-based version: Cohen's d = 0.531). SJT scores were also contingent on the presentation format of both situation descriptions and response options: The highest scores were observed when situation descriptions and response options were presented in the same format. Implications for SJT theory and research are discussed. Practitioner pointsThe presentation format did not moderate the effect of omitting situation descriptions in SJTsthat is, the context dependency of SJT performance did not increase when the SJT was administered in a videobased rather than a text-based format. The elimination of situation descriptions in item stems had a medium effect on overall test scores: SJT scores were significant lower without situation descriptions in comparison to SJT scores with situation descriptions (video-based version: Cohen's d = 0.535 vs. text-based version: Cohen's d = 0.531). It is important to match the stimulus and response formats in SJTs.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.