Asymmetric sampling of personality (ASP): A novel approach to understanding validity limits in personality assessment and possible remedies

Abstract

Assessing people's personalities using self-reports is complicated by three central problems: low predictability of behavior, discrepancies between self- and observer reports, and divergent target reports across observers. Going beyond existing research on common survey biases, we introduce an information sampling bias that can explain all three problems. In judgment and decision research, asymmetric sampling occurs when an individual can only gather a sample of information about a target object (e.g., environment, person) from his or her own experience. It follows that any personal sample is limited by the environment and, given behavioral baselines, is selective for certain experiences (e.g., based on positive affect or habits). We apply the sampling framework to personality assessment and show that, independent of motivational constraints, selective sampling alone invokes an asymmetric mental model of one's own (but also others') personality (e.g., extraversion) in which certain situations are over- or under-represented. We call this asymmetric sampling of personality (ASP). Asymmetric samples of experienced situations lack the generalizability to reliably predict behavior (as a tendency to behave consistently across situations). Moreover, differently biased situation samples can explain self-observer discrepancies in personality ratings as well as multiple observer divergences, calling for a revision of quality standards for inter-rater reliability. Understanding ASP provides a methodological solution to these three problems: it assists all respondents (i.e., self- and peer-observers) in symmetrically sampling situations that generally make it easy or difficult to express a particular personality trait. We discuss ASP in assessment contexts, as well as procedural extensions of the sampling framework, and suggest specific study paradigms for quantifying individual bias due to ASP.