Conditional Reasoning: A Review and Suggestions for Future Test Development and Validation

Abstract

Lawrence R. James spent the last 20 years of his 35-year career developing and validating a new theory of personality that he called conditional reasoning. This theory was focused on mapping and measuring core aspects of the implicit (i.e., unconscious) personality. In this article, we (a) review James’s seminal contributions to the theory and measurement of conditional reasoning, (b) discuss subsequent contributions made in the area of conditional reasoning, and (c) provide a brief “look under the hood” at James’s approach to test development and validation. This final section of our paper is designed to familiarize other researchers with the protocols that James and his colleagues have used over the past 20 years. Many of these protocols have gone unmentioned or only briefly acknowledged (e.g., in conference presentations or informal meetings); indeed, many of these validation protocols were “implicit” in the thinking of James and his approach to the study of personality. Having benefited from working closely with James, we were privy to many of these implicit assumptions and protocols that privately guided James’s early work on conditional reasoning.

Conditional Reasoning: An Integrated Approach to Item Analysis

“…After identifying items that appear to be useful predictors of one or more relevant criteria, we recommend computing item-total correlations, which reflect the extent to which an item discriminates between those examinees who “do well” and those who “do poorly” on the test as a whole (Allen & Yen, 1979, p. 120). In addition, item-total correlations are used to estimate the reliability of CRTs (James & LeBreton, 2012; LeBreton, Grimaldi, & Schoen, 2020). Just as the item-criterion correlations could be computed at the item level or the response level, it is possible to compute item-total correlations using the dichotomously or trichotomously scored items (or at the level of individual item responses).…”

“…As noted by LeBreton et al (2020), the preferred method of estimating reliability is based on test-retest coefficients rather than internal consistency estimates. This is because there is not necessarily a presumption of positive manifold among the JMs for AM and the JMs for FF.…”

Identifying, Defining, and Measuring Justification Mechanisms: The Implicit Biases Underlying Individual Differences

“…This paper aims to elucidate the importance of JMs for the broader audience of researchers interested in understanding the implicit aspects of individuals but also for researchers interested in assessing these theoretical mechanisms with CRTs. While building on the early work that described the basics of item writing and validation (e.g., James et al, 2005; James & LeBreton, 2012; LeBreton, Grimaldi, & Schoen, 2020), we emphasize the tremendous importance of JMs, which serve as the theoretical underpinnings of CR. Unlike survey methods, there are no documented guidelines or “best practices” for identifying and subsequently using JMs to construct items.…”

“…First, we describe a form of inductive theorizing useful to the processes of discovering JMs, focusing on potential sources of information that can help researchers identify JMs. In doing so, we build on existing recommendations to read broadly when uncovering JMs, encouraging researchers to draw from both academic and nonacademic literatures (LeBreton et al, 2020). To illustrate these tools and procedures, we provide examples of how these techniques led to the development of JMs for existing CR measures.…”

“…Although the development and validation of CRTs described in previous research (e.g., James et al, 2005; LeBreton et al, 2020) is important for understanding CR, existing CR work largely ignores the more theoretical focus of JMs. In this section, we detail how researchers can identify and develop the JMs that serve as the foundation of CR.…”