School vs. mail surveys: Disentangling selection and measurement effects in self-reported juvenile delinquency

Abstract

Knowledge on both the prevalence of and the risk factors for juvenile delinquency rely almost exclusively on data generated by self-report research. However, several errors can occur during the process of data gathering, all of which may have an impact on the validity and reliability of the results. This has become more apparent as self-report data on (juvenile) delinquency are increasingly being used to assess the prevalence of delinquency in society. In this article we try to determine how different administration modes may result in different results with regard to the prevalence of juvenile delinquency being reported as well as the predictors of delinquent behaviour. In order to decide whether these differences can be seen as mode effects, the possible influence of selection effects-or differences in the sample population-must be controlled for. For this aim, the technique of case-control matching is used to match the samples of a mail survey and a school survey, both administered on a representative sample of Flemish youth attending secondary education. Using the technique of case-control matching, we aim to control for differences in non-response error, in order to disentangle non-response effects from effects related to administration mode.

Measurement bias in self-reports of offending: a systematic review of experiments

“…In this study, all five individual items of offending presented statistically significant higher reports in the school condition, and the overall OR showed a 25% decreased likelihood of self-reports in home settings. This result is consistent with the self-report literature and has been reported in previous quasi-experimental studies (e.g., Cops et al 2016). On the contrary, we found no evidence that supervision by teachers or supervision by research staff impacts youth self-reports of offending.…”

Data Quality of Different Modes of Supervision in Classroom Surveys

“…While there is a reasonable claim that any supervision can increase completion rates and validity of survey data when surveying adolescents (March et al 2020, Bidonde et al 2023, supervision might not be unproblematic in any case. There are two key factors speaking against teachers supervising survey research: They can make lousy research assistants (Demkowicz et al 2020;Rasberry et al 2020;March et al 2022) and they can increase social desirability or related response bias unintentionally (Strange et al 2003;Duncan and Magnuson 2013;Atkeson et al 2014;Cops et al 2016;Möhring & Schlütz 2019). The results of the analyses in this article allow the conclusion that data quality is lower when teachers are responsible for data collection with regards to item nonresponse and the prevalence of satisficing patterns.…”

“…Overall, it seemed that when there is any kind of supervision involved, response rates were at least slightly higher. The results of Cops et al (2016) suggest that the mode of administration can impact response behaviour on different levels. Survey mode influence extends to both individuals' likelihood to participate in the study, eliciting selection bias, and the potential for differential tendencies to report criminal behaviour among participating individuals, prompting measurement bias.…”

“…Adolescents may be particularly prone to social desirability bias because they are often highly influenced by their peers and social norms (Ziegel 1990;Brown & Larson 2009). Cops et al (2016) found that self-reported delinquency was significantly lower when youths were supervised by adults close to them, which implies that third parties should be avoided and surveys with sensitive questions should rather not take place at home. As Tourangeau and Smith (1996) found in an early review of studies using the computer assisted personal interview (CAPI) approach, social desirability bias can be reduced by using self-administered questionnaires or computer-assisted selfinterviewing, which can increase anonymity and decrease social pressure to conform.…”

The impact of modes of administration on self-reports of offending: evidence from a methodological experiment with university students

“…Skepticism over the utility of these methods compelled criminologists to develop a large body of research on the validity and reliability of SRO (e.g., Farrington, 1973 ; Huizinga & Elliott, 1986 ; Jolliffe et al, 2003 ; Piquero et al, 2014 ), making self-reports one of the most widely used methods in the study of offending behavior (Gomes et al, 2018 ). Current knowledge about the prevalence and causes of offending, as well as risk and protective factors for juvenile delinquency, are almost exclusively reliant on the self-report methodology (Cops et al, 2016 ; Thornberry & Krohn, 2000 ). However, little is known about the impact of measurement biases, such as the ones caused by modes of administration and questionnaire format, on the reported rates of offending and data quality.…”