Validating Earnings in the German National Educational Panel Study – Do Interviewers Have an Impact on Measurement Accuracy?

Abstract

ObjectivesWe investigate characteristics of respondents and interviewers influencing the accurateness of reported income by comparing survey data with administrative data. Questions on sensitive topics like respondents' income often produce relatively high rates of item nonresponse or measurement error. In this context several analyses have been done on item nonresponse, but little is known about accuracy of reporting. Existing evidence shows that it is unpleasant for respondents to report very low or very high income. In presence of an interviewer income questions might produce incorrect responses due to social desirability bias. On the other hand side interviewers can create a trustful atmosphere in which respondents give more accurate answers.
 ApproachUsing linked survey and administrative data we are able to measure the extent of deviation between reported and recorded incomes and explore the influence of respondent and interviewer characteristics on it. The starting point for the linkage is data from the German National Educational Panel Study (NEPS), Starting Cohort 6, which surveys adults from birth cohorts 1944 to 1986. More than 90% of the respondents consented to a linkage of their survey information with administrative data from the German Federal Employment Agency. These longitudinal earnings data are highly reliable as they are based on mandatory notifications of employers to the social security system.We include interviewer and respondent characteristics as well as their interactions into our model to estimate their respective impact on the incidence and size of any bias in reported incomes. This allows us to control for latent interviewer traits that might have influenced the respondent's answering behavior during each interview of a given interviewer.
 ResultsThe average deviation of reported from administrative earnings is relatively small (less than 10% of median earnings). Descriptive evidence shows only small variation of deviation across subgroups. Most importantly, female respondents show higher report accuracy. Multivariate results hint at a negligible influence of interviewer characteristics. The major predictors for deviation in respondents' characteristics are their sex, their absolute monthly personal income, their educational level and being born abroad.
 ConclusionAlthough the average measurement accuracy is rather high, there are some differences in deviations by subgroups. The impact of these deviations depends on the research question at hand. Research with a strong focus on the respondent’s earnings, e.g. when using them as a dependent variable, should use the linked data rather than only the NEPS survey data.

Interpersonal Conflict over Water Is Associated with Household Demographics, Domains of Water Insecurity, and Regional Conflict: Evidence from Nine Sites across Eight Sub-Saharan African Countries

“…These contradicting results may be explained by issues associated with self-reported income in survey work [69][70][71][72]. Studies note, for example, that high-income respondents report lower-than-actual earnings while lowincome respondents report higher-than-actual earnings [70,71,73]. But in the absence of publicly available income tax or pension data, survey data are the only means of collecting income information, albeit imperfect [72].…”

The attitude-behaviour gap in biosecurity: Applying social theories to understand the relationships between commercial chicken farmers' attitudes and behaviours

“…There is a possibility that some farmers tended to report desirable biosecurity-related attitudes and actions because this study did not investigate those variables onsite. However, despite the likely presence of social desirability response bias, whereby respondents tend to report better attitudes and better biosecurity status than they practise ( 87 ), this bias is likely to be reduced by the local officials' familiarity and long-term partnerships with the research participants ( 88 ). Furthermore, a group of farmers who intended not to recognise the importance of biosecurity was observed in the qualitative and the quantitative phases, and another group of farmers who had a low implementation level of biosecurity was observed in the quantitative phase, suggesting that the social desirability response bias may only have had a minor impact on this research.…”

Linking PIAAC Data to Individual Administrative Data: Insights from a German Pilot Project

“…Substantive research questions may focus, for example, on the relationship between earnings development (IEB data) and skill level (survey data). Methodological questions that exploit the potential of these two data sources may deal, for example, with the evaluation of the measurement error in the survey data by assessing it against (less error-prone) administrative data (see also Antoni et al 2019b;Gauly et al 2019).…”