Collecting Paradata for Measurement Error Evaluations

Olson, Kristen; Parkhurst, Bryan

doi:10.1002/9781118596869.ch3

Cited by 56 publications

(43 citation statements)

References 67 publications

(96 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Low item nonresponse rates in telephone surveys make it a difficult outcome to use for this purpose. Another possibility is response time, which has been previously used as a proxy measure of data quality, and is of interest for factors that contribute to the overall length of a questionnaire (Olson and Parkhurst 2013;Yan and Olson 2013).…”

Section: Introductionmentioning

confidence: 99%

The Effect of CATI Questions, Respondents, and Interviewers on Response Time

Olson¹,

Smyth²

2015

J Surv Stat Methodol

Self Cite

View full text Add to dashboard Cite

In this paper, we evaluate the joint effects of question, respondent, and interviewer characteristics on response time in a telephone survey. We include question features traditionally examined, such as the length of the question and format of response options, and features that have yet to be examined that are related to the layout and format of intervieweradministered questions. We examine how these question features affect the time to ask and answer survey questions and how different interviewers vary in their administration of these questions. This paper uses paradata from the Work and Leisure Today survey and uses cross-classified random effects models. Overall, most of the variation in response time is due to question characteristics, rather than respondent or interviewer attributes. Additionally, we find that question characteristics related to necessary survey design features and respondent confusion are the primary predictors of response time, with little effect of visual design features of the question.We also find modest differences in the effects of question characteristics by interviewer experience.

show abstract

Section: Introductionmentioning

confidence: 99%

The Effect of CATI Questions, Respondents, and Interviewers on Response Time

Olson¹,

Smyth²

2015

J Surv Stat Methodol

Self Cite

View full text Add to dashboard Cite

show abstract

“…Finally, interview length is only an indirect indicator of data quality (e.g., Olson and Parkhurst 2013;Yan and Olson 2013). Future research should also investigate the effects of variables capturing learning and response propensity on more direct indicators of data quality, such as rounding or response styles.…”

Section: Discussionmentioning

confidence: 99%

Examining Changes of Interview Length over the Course of the Field Period

Kirchner¹,

Olson²

2016

J Surv Stat Methodol

Self Cite

View full text Add to dashboard Cite

It is well established that interviewers learn behaviors both during training and on the job. How this learning occurs has received surprisingly little empirical attention: Is it driven by the interviewer herself or by the respondents she interviews? There are two competing hypotheses about what happens during field data collection: (1) interviewers learn behaviors from their previous interviews, and thus change their behavior in reaction to the behaviors previously encountered; and (2) interviewers encounter different types of and, especially, less cooperative respondents (i.e., nonresponse propensity affecting the measurement error situation), leading to changes in interview behaviors over the course of the field period. We refer to these hypotheses as the experience and response propensity hypotheses, respectively. This paper examines the relationship between proxy indicators for the experience and response propensity hypotheses on interview length using data and paradata from two telephone surveys.Our results indicate that both interviewer-driven experience and respondent-driven response propensity are associated with the length of interview. While general interviewing experience is nonsignificant, withinstudy experience decreases interview length significantly, even when accounting for changes in sample composition. Interviewers with higher cooperation rates have significantly shorter interviews in study one; however, this effect is mediated by the number of words spoken by the interviewer. We find that older respondents and male respondents have longer interviews Kirchner & Olson JSSM 2017 EXAMINING CHANGES 2 despite controlling for the number of words spoken, as do respondents who complete the survey at first contact. Not surprisingly, interviews are significantly longer the more words interviewers and respondents speak.

show abstract

“…The final sample size is 2,008 respondents after 18 (0.8 percent) cases were removed due to short completion times of less than 5 minutes. A common technique in survey research is to use completion time to identify respondents who may have sped through the survey without carefully reading or answering questions (Olson & Parkhurst, 2013). A review of the 18 cases showed that respondents either failed to answer items after the initial demographic section, or exhibited "straight-lining" (answering several sequential items with the same response).…”

Section: Data Collectionmentioning

confidence: 99%

Employees’ Perceptions of Workplace Preparedness for Climate Change-Related Natural Hazards

Sadiq¹,

Ollier²,

Tyler³

2016

Risk, Hazards &Amp; Crisis in Public Policy

View full text Add to dashboard Cite

Despite climate change being identified as a current or future risk for organizations, researchers have yet to understand employees’ perceptions of their workplaces’ preparedness for climate change‐related natural hazards (cc hazards) such as floods and wildfires. The purpose of this preliminary study is to describe the current state of workplace preparedness for cc hazards and to ascertain whether workplace preparedness for cc hazards aligns with state hazard priorities for cc hazards. Using data collected in 2014 from an online, national survey of 2,008 employees in the United States, the results indicate that workplaces are moderately prepared for cc hazards. In addition, there is a slight mismatch between workplace preparedness and state preparedness priorities for cc hazards. This study contributes to the literature on organizational preparedness for cc hazards by providing insights on the current state of workplace preparedness—from the perspectives of employees—for natural hazards associated with the global threat known as climate change. The paper concludes with a policy recommendation and provides an outline for future research on organizational preparedness for cc hazards.

show abstract

Collecting Paradata for Measurement Error Evaluations

Cited by 56 publications

References 67 publications

The Effect of CATI Questions, Respondents, and Interviewers on Response Time

The Effect of CATI Questions, Respondents, and Interviewers on Response Time

Examining Changes of Interview Length over the Course of the Field Period

Employees’ Perceptions of Workplace Preparedness for Climate Change-Related Natural Hazards

Contact Info

Product

Resources

About