In Defense of the Situation: An Interactionist Explanation for Performance on Situational Judgment Tests

Abstract

Whereas Lievens and Motowidlo (2016) propose a model of situational judgment test (SJT) performance that removes the "situation" in favor of conceptualizing SJTs as a measure of general domain knowledge, we argue that the expression of general domain knowledge is in fact contingent on situational judgment. As we explain, the evidence cited by Lievens and Motowidlo against a situational component does not inherently exclude the importance of situations from SJTs and does overlook the strong support for a person-situation interaction explanation of behavior. Based on the interactionist literature-in particular, the trait activation theory (TAT) and situational strength literatures-we propose a model that both maintains the key pathways and definitions posited by Lievens and Motowidlo and integrates the situational component of SJTs.Interactionist explanations of work behavior have received increasing attention and support in the employee selection literature and stem from a long history of research on person-situation models of personality (e.g., Mischel, 1968). The ability to evaluate situational demands predicts performance across assessment types, including structured interviews (Melchers, Bösser, Hartstein, & Kleinmann, 2012) and assessment centers (Jansen et al., 2013). Further, the ability to identify criteria for performance evaluation (broadly conceptualized as situational cues) has been posited as a key explanation of the criterion-related validity for selection assessments (Kleinmann et al., 2011). Given the evidence supporting a person-situation interaction account of performance in assessment centers and structured

A closer look at response options: Is judgment in situational judgment tests a function of the desirability of response options?

“…SJTs (e.g., Crook, 2016;Chen et al, 2016;Fan et al, 2016;Harvey, 2016;Lievens & Motowidlo, 2016;McDaniel et al, 2016) Albeit the specific claim made by several researchers Harris et al, 2016;Melchers & Kleinmann, 2016), that is, that test takers gather situational information from response options, has not been tested explicitly, evidence that response options serve as sources of information in SJTs is indeed available. Leeds (2012) distinguished two sequential steps of processing when taking an SJT.…”

“…In light of the notion that response options in SJTs may contain enough information to even render situation descriptions superfluous (Harris et al, 2016;Krumm et al, 2015;Leeds, 2012;Melchers & Kleinmann, 2016), the social desirability of response options may be an important design feature influencing test takers' responses to SJT items. In fact, results provided by Krumm et al (2015) support this view.…”

“…Understandably, this finding has led to a controversial debate among scholars (Borneman, ; Brown, Jones, Serfass, & Sherman, ; Chen, Fan, Zeng, & Hack, ; Crook, ; Fan, Stuhlmann, Chen, & Weng, ; Harris, Siedor, Fan, Listyg, & Carter, ; Harvey, ; Lievens & Motowidlo, ; McDaniel, List, & Kepes, ; Melchers & Kleinmann, ; Naemi, Martin‐Raugh, & Kell, ; Torres & Beier, ; Whetzel & Reeder, ). Among other points, it was discussed whether response options are a valid source of information in SJTs and might enable test takers to deduce correct solutions (Harris et al, ; Melchers & Kleinmann, ). In fact, recent research suggests that response options choice in SJTs is determined by more than test takers’ perusal of the situation (Leeds, ).…”

“…The results reported by Krumm et al () have sparked a controversial debate about the relevance of situation descriptions in SJTs. Besides other arguments for and against a dismissal of situations in SJTs (e.g., Crook, ; Chen et al, ; Fan et al, ; Harvey, ; Lievens & Motowidlo, ; McDaniel et al, ), Melchers and Kleinmann () as well as Harris et al () argued that response options may provide valid and oftentimes sufficient information. Specifically, Melchers and Kleinmann stated that, in the absence of situation descriptions, “test takers try to understand the situation to which the response options are related, with the only difference that an understanding of this situation is (at least somewhat) more difficult in comparison with when an item stem with a description of the situation is included” (p. 32).…”

Removing situation descriptions from situational judgment test items: Does the impact differ for video‐based versus text‐based formats?

“…Recently, these findings have generated quite a bit of controversy (Borneman, ; Brown, Jones, Serfass, & Sherman, ; Chen, Fan, Zheng, & Hack, ; Crook, ; Fan, Stuhlman, Chen, & Weng, ; Harris, Siedor, Fan, Listyg, & Carter, ; Harvey, ; Lievens & Motowidlo, ; McDaniel et al , ; Melchers & Kleinmann, ; Naemi et al , ; Torres & Beier, ; Whetzel & Reeder, ). One group of scholars echoed Krumm et al ’s () call to reconceptualize SJTs as mainly context‐independent measures (Crook, ; Harvey, ; Lievens & Motowidlo, ).…”