Should we trust survey data? Assessing response simplification and data fabrication

Blasius, Jörg; Thiessen, Victor

doi:10.1016/j.ssresearch.2015.03.006

Cited by 29 publications

(14 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, fabricating large parts of the data or even entire interviews, as shown in detail for Greece and Russia, will probably not be possible without “some help” from employees of the SROs, for example, when they do not check the interviews. This is not a strong assumption, as shown for the World Value Survey 2004–2008 (Blasius and Thiessen 2012), for PISA 2012 and PISA 2009, principal data (Blasius 2018; Blasius and Thiessen 2015), and for the ISSP 2006 (Thiessen and Blasius 2016); sometimes employees of the SROs even fabricate entire interviews with simple procedures such as copy and paste.…”

Section: Resultsmentioning

confidence: 99%

“…One strategy is to primarily fabricate those parts of interviews that consist of long-item batteries (Blasius and Friedrichs 2012; Waller 2013). Evidence for partially fabricated interviews exists even in surveys known to be of high quality (Blasius 2018, Blasius and Thiessen 2012, 2015; Bredl et al 2012; Thiessen and Blasius 2016). Of course, the larger the fabricated proportion of an interview, the easier it is for SROs to detect the anomalous response patterns.…”

Section: Theoretical Frameworkmentioning

confidence: 99%

“…Data fabrication of entire or partial respondent information is the usual evidence for malfeasance at the SRO level. Recent studies suggest that the incidence of data fabrication by employees of some SROs can be of troubling magnitude, even in well-known international surveys such as the World Value Survey 2004–2008 (Blasius and Thiessen 2012), The Programme for International Student Assessment (PISA) 2012 and 2009, principal data (Blasius 2018; Blasius and Thiessen 2015), and the International Social Survey Programme (ISSP) 2006 (Thiessen and Blasius 2016).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Perceived Corruption, Trust, and Interviewer Behavior in 26 European Countries

Blasius

Thiessen

2018

Sociological Methods & Research

Self Cite

View full text Add to dashboard Cite

Identifying illicit behavior in survey research is inherently problematic, since self-reports are untrustworthy. We argue that fraudulent interviewers can, however, be identified through statistical deviance of the distributional parameters of their interviews. We document that a high proportion of the variation in the data is due to the interviewer. In addition, we show that the incidence of interviewer-induced anomalies is strongly associated with the perceived level of corruption across the countries participating in the European Social Survey 2010. The major implication of the findings is that the data from some countries cannot be used fruitfully for cross-national comparative research.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Theoretical Frameworkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Perceived Corruption, Trust, and Interviewer Behavior in 26 European Countries

Blasius

Thiessen

2018

Sociological Methods & Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…More sophisticated fabrication methods include an interviewer asking only a single question and then filling out the remaining items in a battery by inferring the respondent’s likely answers. Principal-component analysis (PCA) determines if an interviewer has a lower variance for a set of questions, which could indicate fabrication by this means (Blasius and Thiessen 2015).…”

Section: Stage 3: Assessing Data Quality After Fieldwork Endsmentioning

confidence: 99%

Improving Data Quality in Face-to-Face Survey Research

Logan¹,

Parás²,

Robbins³

et al. 2019

APSC

View full text Add to dashboard Cite

Data quality in survey research remains a paramount concern for those studying mass political behavior. Because surveys are conducted in increasingly diverse contexts around the world, ensuring that best practices are followed becomes ever more important to the field of political science. Bringing together insights from surveys conducted in more than 80 countries worldwide, this article highlights common challenges faced in survey research and outlines steps that researchers can take to improve the quality of survey data. Importantly, the article demonstrates that with the investment of the necessary time and resources, it is possible to carry out high-quality survey research even in challenging environments in which survey research is not well established.

show abstract

“…Employees of research institutes can simplify their assignments by fabricating entire interviews through copy and paste (Blasius and Thiessen, 2012, 2015). In the case of Programme for International Student Assessment (PISA), it might be that principals did not fill in the school questionnaires; in contrast to other surveys, there is no possibility of replacing the target persons, even though it is important for many studies to have information on school-level indicators.…”

Section: Introductionmentioning

confidence: 99%

Fabrication of interview data

Blasius

2018

QAE

Self Cite

View full text Add to dashboard Cite

Purpose Evidence from past surveys suggests that some interviewees simplify their responses even in very well-organized and highly respected surveys. This paper aims to demonstrate that some interviewers, too, simplify their task by at least partly fabricating their data, and that, in some survey research institutes, employees simplify their task by fabricating entire interviews via copy and paste. Design/methodology/approach Using data from the principal questionnaires in the Programme for International Student Assessment (PISA) 2012 and the Programme for the International Assessment of Adult Competencies (PIAAC) data, the author applies statistical methods to search for fraudulent methods used by interviewers and employees at survey research organizations. Findings The author provides empirical evidence for potential fraud performed by interviewers and employees of survey research organizations in several countries that participated in PISA 2012 and PIAAC. Practical implications The proposed methods can be used as early as the initial phase of fieldwork to flag potentially problematic interviewer behavior such as copying responses. Originality/value The proposed methodology may help to improve data quality in survey research by detecting fabricated data.

show abstract

Should we trust survey data? Assessing response simplification and data fabrication

Cited by 29 publications

References 44 publications

Perceived Corruption, Trust, and Interviewer Behavior in 26 European Countries

Perceived Corruption, Trust, and Interviewer Behavior in 26 European Countries

Improving Data Quality in Face-to-Face Survey Research

Fabrication of interview data

Contact Info

Product

Resources

About