Insufficient Effort Survey Responding: An Under‐Appreciated Problem in Work and Organisational Health Psychology Research

Abstract

Insufficient effort responding (IER) is problematic in that it can add a systematic source of variance for variables with average responses that depart from the scale midpoints. We present a rationale for why IER is of particular importance to Work and Organisational Health Psychology (WOHP) researchers. We also demonstrate its biasing effects using several variables of interest to WOHP researchers (perceived work ability, negative affectivity, perceived disability, work-safety tension, accident/injury frequencies, and experienced and instigated incivility) in two datasets. As expected, IER was significantly correlated with the focal study variables. We also found some evidence that hypothesised bivariate correlations between these variables were inflated when IER respondents were included. Corroborating IERs potential confounding role, we further found significant declines in the magnitude of the hypothesised bivariate correlations after partialling out IER. In addition, we found evidence for biasing (under-estimation) effects for predictors not contaminated by IER in multiple regression models where some predictors and the outcome were both contaminated by IER. We call for WOHP researchers to routinely discourage IER from occurring in their surveys, screen for IER prior to analyzing survey data, and establish a standard practice for handling IER cases.

Exploring daily affective changes in university students with a mindful positive reappraisal intervention: A daily diary randomized controlled trial

“…Participants were analysed on a conservative intent‐to‐treat basis, with the exception that at least two daily surveys were completed to allow for growth analyses (Sagarin et al, ). Data were systematically reviewed and excluded for insufficient effort responding (McGonagle, Huang, & Walsh, ; see Figure for CONSORT diagram of all attrition and exclusions). This resulted in 468 daily diary results from 106 participants (36 mindful‐reappraisal, 34 reappraisal‐only, and 36 control).…”

“…Data were systematically reviewed and excluded for insufficient effort responding (McGonagle, Huang, & Walsh, 2015; see Figure 1 for CON-SORT diagram of all attrition and exclusions). This resulted in 468 daily diary results from 106 participants (36 mindful-reappraisal, 34 reappraisal-only, and 36 control).…”

Your Attention Please! Toward a Better Understanding of Research Participant Carelessness

“…DeSimone, DeSimone, Harms, and Wood use three simulated datasets to examine the effects of two types of careless responding—random responding and non‐random responding (“straightlining”). This research is important because previous studies have not examined the differential effects of different types of careless responding (see Credé, ; Huang et al, ; McGonagle, Huang, & Walsh, ). DeSimone et al found that random responding generally decreased the inter‐item correlations, internal‐consistency reliabilities, and first component eigenvalues of substantive measures.…”

The Impact of Insufficient Effort Responding Detection Methods on Substantive Responses: Results from an Experiment Testing Parameter Invariance

“…Survey data, that is, data collected from individuals using (self‐report) questionnaires, is arguably the most widely used method of data collection in applied psychological research (e.g. Eatough & Spector, ; McGonagle, Huang, & Walsh, ). Researchers conducting surveys rely on respondents' attentiveness in completing questionnaires in order to obtain valid data representing true actual values.…”

“…Researchers conducting surveys rely on respondents' attentiveness in completing questionnaires in order to obtain valid data representing true actual values. This issue of respondent attentiveness has gained considerable attention among methodologists in recent years (Bowling, Huang, Bragg, Khazon, Liu, & Blackmore, ; Curran, ; Dunn, Heggestad, Shanock, & Theilgard, in press; Hauser & Schwarz, ; Huang, Bowling, Liu, & Li, ; Huang, Liu, & Bowling, ; Kam & Meyer, ; Liu, Bowling, Huang, & Kent, ; McGonagle et al, ; Ran, Liu, Marchiondo, & Huang, ; Ward & Pond, ). More specifically, researchers have started investigating how individuals exhibiting insufficient effort responding (IER) can be detected based on their responses (Huang, Curran, Keeney, Poposki, & DeShon, ; Meade & Craig, ).…”