Fabrication of interview data

Abstract

Purpose Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task by fabricating entire interviews via copy and paste. Design/methodology/approach Using data from the principal questionnaires in the Programme for International Student Assessment (PISA) 2012 and the Programme for the International Assessment of Adult Competencies (PIAAC) data, the author applies statistical methods to search for fraudulent methods used by interviewers and employees at survey research organizations. Findings The author provides empirical evidence for potential fraud performed by interviewers and employees of survey research organizations in several countries that participated in PISA 2012 and PIAAC. Practical implications The proposed methods can be used as early as the initial phase of fieldwork to flag potentially problematic interviewer behavior such as copying responses. Originality/value The proposed methodology may help to improve data quality in survey research by detecting fabricated data.

Perceived Corruption, Trust, and Interviewer Behavior in 26 European Countries

“…One strategy is to primarily fabricate those parts of interviews that consist of long-item batteries (Blasius and Friedrichs 2012; Waller 2013). Evidence for partially fabricated interviews exists even in surveys known to be of high quality (Blasius 2018, Blasius and Thiessen 2012, 2015; Bredl et al 2012; Thiessen and Blasius 2016). Of course, the larger the fabricated proportion of an interview, the easier it is for SROs to detect the anomalous response patterns.…”

“…Data fabrication of entire or partial respondent information is the usual evidence for malfeasance at the SRO level. Recent studies suggest that the incidence of data fabrication by employees of some SROs can be of troubling magnitude, even in well-known international surveys such as the World Value Survey 2004–2008 (Blasius and Thiessen 2012), The Programme for International Student Assessment (PISA) 2012 and 2009, principal data (Blasius 2018; Blasius and Thiessen 2015), and the International Social Survey Programme (ISSP) 2006 (Thiessen and Blasius 2016).…”

How to Get Better Survey Data More Efficiently

“…Perhaps the most efective, and most expensive, strategy for assuring data quality is the "callback," where ieldwork supervisors conduct partial re-interviews with participants to verify their participation (Biemer and Stokes 1989;Schäfer et al 2004;Stokes and Jones 1989;Swanson, Cho, and Eltinge 2003;Winker 2016). More recently, researchers have introduced checks for interview duplication and straightlining, wherein enumerators or staf at survey irms generate fraudulent interviews by illing out identical answers across a battery of questions (Blasius 2018;Blasius and Thiessen 2012Simmons et al 2016;Slomczynski, Powalko, and Krauze 2017).…”

“…Percentmatch, which detect near-duplicate interviews and lag them as likely to be fraudulent in semi-real time (Kuriakose and Robbins 2016). Abnormal participation rates and interview duration, among other patterns in the raw data and metadata, can similarly be mined for clues about data fabrication (Birnbaum et al 2012;Blasius 2018;Blasius andThiessen 2012, 2018;Bredl, Storinger, and Menold 2011;Murphy et al 2004).…”

“…. Important exceptions include recent attention to screening out duplicate and near-duplicate interviews, discussed below(Blasius 2018;Kuriakose and Robbins 2016).…”