Flexibility in Variable Operationalization in Social Disorganizartion Theory: A Pilot Study

Abstract

Like other social sciences, criminology is experiencing a theory crisis as part of a larger replication crisis.Niemeyer and colleagues (2022) estimate the impact of weak criminological theory on false positive findings using the Ioannidis equation and find that likely at least 90% of findings are false positives. Weak criminological theory directly contributes to a methods-based crisis of flexible operationalization and poor reproducibility of findings. To better quantify the problem, we calculate the index of qualitative variation (IQV) for theoretical constructs in a sample of social disorganization empirical tests. The IQV empirically demonstrates the extent of researcher degrees of freedom through flexibility of theoretical constructs. It accounts for both the number of categories of operationalizations used for each construct and the distribution of use across those operationalizations. We found that the IQV for poverty, racial and ethnic heterogeneity, and residential stability were extremely high in our sample with values ranging from 0.84 to 0.99. This empirically identifies a theoretical contributor to the reproducibility crisis in criminology. We discuss the troubling implications of high theoretical flexibility for social disorganization theory and for broader replication efforts.