Careless responding in questionnaire measures: Detection, impact, and remedies

Goldammer, Philippe; Annen, Hubert; Stöckli, Peter Lucas; Jonas, Klaus

doi:10.1016/j.leaqua.2020.101384

Cited by 117 publications

(125 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A convenience sample of four hundred nineteen U.S.-based adult employees was recruited using Amazon’s MTurk to participate in an ongoing research project entitled “Longitudinal study of work/life experiences during the COVID-19 pandemic.” Employees successively completed three waves of anonymous online surveys in May 2020 (Time 1), June 2020 (Time 2), and August 2020 (Time 3). Participation requirements included (a) being employed outside of MTurk at the time of data collection, (b) having a 90% approval rate or higher for the past 100 crowd-sourced tasks [ 14 ], and (c) not having been flagged as a careless respondent in previous waves (i.e., taking on average less than 2 s to answer the survey items, failing two or more attention checks out of four, and being a multivariate outlier; [ 15 ]). A breakdown of our sample’s demographics is available in Table 1 .…”

Section: Methodsmentioning

confidence: 99%

Economic Stressors, COVID-19 Attitudes, Worry, and Behaviors among U.S. Working Adults: A Mixture Analysis

Bazzoli

Probst

Lee

2021

IJERPH

View full text Add to dashboard Cite

Since the unfolding of the novel coronavirus global pandemic, public health research has increasingly suggested that certain groups of individuals may be more exposed to the virus. The aim of this contribution was to investigate whether workers grouped into several latent classes, based on two perceived economic stressors, would report different levels of enactment of the Centers for Disease Control (CDC) recommended behaviors to prevent the spread of such virus. We also tested propositions regarding the potential differential predictors of compliance behavior, differentiating between cognitive (i.e., attitudes toward the CDC guidelines) and affective (i.e., COVID-specific worry) predictors. Using a longitudinal dataset of 419 U.S. workers, we did not find significant differences among the levels of CDC guidelines enactment across three latent classes, representing a range of economic vulnerability. We found that cognitive attitudes were a significantly stronger predictor of compliance with CDC guidelines for workers in the most economically secure class, whereas worry was a significantly stronger predictor of compliance for the most vulnerable counterpart. We discuss these findings in light of the Conservation of Resources theory and other health behavior theories, being mindful of the need to further understand the differential impact of this health and economic crisis on employees facing economic stressors.

show abstract

Section: Methodsmentioning

confidence: 99%

Economic Stressors, COVID-19 Attitudes, Worry, and Behaviors among U.S. Working Adults: A Mixture Analysis

Bazzoli

Probst

Lee

2021

IJERPH

View full text Add to dashboard Cite

show abstract

“…Of the 377 respondents, we first excluded 58 who were not eligible and had missing values. An additional 32 RNs were excluded for careless responses that delineated unmotivated and inattentive responding patterns (Goldammer et al, 2020). In this study, careless responders were identified by Mahalanobis distance values (DeSimone et al, 2015;Goldammer et al, 2020) larger than the critical chi-square value of 21.03 at an alpha level of 0.05.…”

Section: Sample Characteristics Of the Validation Studymentioning

confidence: 99%

“…As a result, 287 respondents were included in the analysis. Response type can affect psychometric properties of constructs (Goldammer et al, 2020) and rejection of the unidimensional factor model in a CFA test (Huang et al, 2012;Woods, 2006). Table 1 summarizes…”

Section: Sample Characteristics Of the Validation Studymentioning

confidence: 99%

The development of a shortened, Korean version of the Anticipated Turnover Scale for hospital nurses

Nam

Kim

Hong

et al. 2021

Research in Nursing & Health

View full text Add to dashboard Cite

This study was a psychometric validation of a Korean version of the Anticipated Turnover Scale (ATS) involving translation, transcultural adaptation, and validation.We used data from a cross-sectional online survey of 287 Korean practicing nurses in hospitals. Confirmatory factor analysis (CFA) assessed the validity of the onefactor structure that was hypothesized in the original version. We removed any items that did not enhance the scale's reliability and validity. Test and retest reliability was performed on 206 of the nurses who completed the survey again two weeks later. The mean age of the 287 nurses was 33 years, and 278 (97%) of them were female. The final 6-item Korean version of the ATS demonstrated a satisfactory model fit in the CFA (χ 2 = 22.67; degree of freedom = 9; goodness-of-fit in-dex= 0.97; comparative fit index = 0.98; Tucker-Lewis Index = 0.97; root mean square error of approximation = 0.07; and standard root mean residual = 0.03). The intraclass correlation coefficient with the test-retest reliability for the 2-week interval was 0.81, with a good internal consistency (Cronbach's alpha = 0.85;McDonald's omega coefficient reliability = 0.92). The 6-item revised ATS is a reliable and valid scale to assess the turnover intention of Korean nurses who work in hospitals. It can be used in an acute setting and predict the anticipated turnover rate for monitoring purposes.

show abstract

“…To address this problem, we flagged careless respondents based on three criteria: average response time per item, item consistency on a semantic antonym and Mahalonobis distance (Curran 2016;Meade and Craig 2012). We adopted the cut scores set for 95% specificity from Goldammer et al (2020) for average response time per item and the Mehalonobis distance. We considered responses to be inconsistent if the absolute difference between the two reverse-item scored items was equal or larger than 2.…”

Section: Participants and Data Collectionmentioning

confidence: 99%

Unpacking the Quantifying and Qualifying Potential of Semi-Open Job Satisfaction Questions through Computer-Aided Sentiment Analysis

Wijngaards

Burger

Exel

2020

J well-being assess

View full text Add to dashboard Cite

Despite their suitability for mitigating survey biases and their potential for enhancing information richness, open and semi-open job satisfaction questions are rarely used in surveys. This is mostly due to the high costs associated with manual coding and difficulties that arise when validating text measures. Recently, advances in computer-aided text analysis have enabled researchers to rely less on manual coding to construct text measures. Yet, little is known about the validity of text measures generated by computer-aided text analysis software and only a handful of studies have attempted to demonstrate their added value. In light of this gap, drawing on a sample of 395 employees, we showed that the responses to a semi-open job satisfaction question can reliably and conveniently be converted into a text measure using two types of computer-aided sentiment analysis: SentimentR, and Linguistic Inquiry and Word Count (LIWC) 2015. Furthermore, the substantial convergence between the LIWC2015 and, in particular, SentimentR measure with a closed question measure of job satisfaction and logical associations with closed question measures of constructs that fall within and outside job satisfaction’s nomological network, suggest that a semi-open question has adequate convergent and discriminant validity. Finally, we illustrated that the responses to our semi-open question can be used to fine-tune the computer-aided sentiment analysis dictionaries and unravel antecedents of job satisfaction.

show abstract

Careless responding in questionnaire measures: Detection, impact, and remedies

Cited by 117 publications

References 44 publications

Economic Stressors, COVID-19 Attitudes, Worry, and Behaviors among U.S. Working Adults: A Mixture Analysis

Economic Stressors, COVID-19 Attitudes, Worry, and Behaviors among U.S. Working Adults: A Mixture Analysis

The development of a shortened, Korean version of the Anticipated Turnover Scale for hospital nurses

Unpacking the Quantifying and Qualifying Potential of Semi-Open Job Satisfaction Questions through Computer-Aided Sentiment Analysis

Contact Info

Product

Resources

About