Identifying Invalid Responders in a Campus Climate Survey: Types, Impact on Data, and Best Indicators

Li, Caihong R.; Follingstad, Diane R.; Campe, Margaret Irene; Chahal, Jaspreet K.

doi:10.1177/0886260520918588

Cited by 10 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using indirect careless responding indices that fall into different, previously proposed types of indices (i.e., categorized by consistency, response pattern, and RT), the present research demonstrated that an LCA approach that is adapted to the nested data structure (ML-LCA) can be used to model careless responding in AA. Our results on the latent typology of momentary careless responding were largely in line with previous research (Goldammer et al, 2020; Kam & Meyer, 2015; Li et al, 2022; Maniaci & Rogge, 2014; Meade & Craig, 2012), thereby demonstrating that different types of careless responding (long string vs. inconsistent careless responding) can also be identified in AA data. Similar to recommendations for studying careless responding in cross-sectional surveys (e.g., Curran, 2016), we recommend that researchers who want to investigate patterns of careless responding in their AA data use indices from these different types to capture different types of careless responding.…”

Section: Discussionsupporting

confidence: 90%

“…To study careful versus careless responding using LCA, researcher-defined careless responding indices (e.g., the ones summarized above) serve as observed variables in the LCA model. The LCA approach has been used successfully to identify careless responding in cross-sectional surveys (Goldammer et al, 2020; Kam & Meyer, 2015; Li et al, 2022; Maniaci & Rogge, 2014; McKibben & Silvia, 2017; Meade & Craig, 2012; Paas et al, 2018; Schneider et al, 2018). Previous research has repeatedly shown that participants can be assigned to one of three classes, with one class representing careful responders and two classes representing careless responders (Goldammer et al, 2020; Kam & Meyer, 2015; Li et al, 2022; Maniaci & Rogge, 2014; Meade & Craig, 2012).…”

Section: Careless Responding Indices In Cross-sectional Researchmentioning

confidence: 99%

“…The LCA approach has been used successfully to identify careless responding in cross-sectional surveys (Goldammer et al, 2020; Kam & Meyer, 2015; Li et al, 2022; Maniaci & Rogge, 2014; McKibben & Silvia, 2017; Meade & Craig, 2012; Paas et al, 2018; Schneider et al, 2018). Previous research has repeatedly shown that participants can be assigned to one of three classes, with one class representing careful responders and two classes representing careless responders (Goldammer et al, 2020; Kam & Meyer, 2015; Li et al, 2022; Maniaci & Rogge, 2014; Meade & Craig, 2012). Usually, one careless responding class represents long-string (i.e., invariable) responses, whereas the other careless responding class represents inconsistent responses.…”

Section: Careless Responding Indices In Cross-sectional Researchmentioning

confidence: 99%

See 2 more Smart Citations

Modeling careless responding in ambulatory assessment studies using multilevel latent class analysis: Factors influencing careless responding.

Hasselhorn¹,

Ottenstein²,

Lischetzke³

2023

Psychological Methods

View full text Add to dashboard Cite

As the number of studies using ambulatory assessment (AA) has been increasing across diverse fields of research, so has the necessity to identify potential threats to AA data quality such as careless responding. To date, careless responding has primarily been studied in cross-sectional surveys. The goal of the present research was to identify latent profiles of momentary careless responding on the occasion level and latent classes of individuals (who differ in the distribution of careless responding profiles across occasions) on the person level using multilevel latent class analysis (ML-LCA). We discuss which of the previously proposed indices seem promising for investigating careless responding in AA studies, and we show how ML-LCA can be applied to model careless responding in intensive longitudinal data. We used data from an AA study in which the sampling frequency (3 vs. 9 occasions per day, 7 days, n = 310 participants) was experimentally manipulated. We tested the effect of sampling frequency on careless responding using multigroup ML-LCA and investigated situational and respondent-level covariates. The results showed that four Level 1 profiles (“careful,” “slow,” and two types of “careless” responding) and four Level 2 classes (“careful,” “frequently careless,” and two types of “infrequently careless” respondents) could be identified. Sampling frequency did not have an effect on careless responding. On the person (but not the occasion) level, motivational variables were associated with careless responding. We hope that researchers might find the application of an ML-LCA approach useful to shed more light on factors influencing careless responding in AA studies.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: Careless Responding Indices In Cross-sectional Researchmentioning

confidence: 99%

Section: Careless Responding Indices In Cross-sectional Researchmentioning

confidence: 99%

See 1 more Smart Citation

Modeling careless responding in ambulatory assessment studies using multilevel latent class analysis: Factors influencing careless responding.

Hasselhorn¹,

Ottenstein²,

Lischetzke³

2023

Psychological Methods

View full text Add to dashboard Cite

show abstract

“…Response rate was 14.2% in the first sample and 11.2% in the second. For the current study, only students who completed the sexual violence modules were included in analyses, and 19 participants were removed for indicating that they either did not tell the truth on the survey or did not pay attention (Li et al, 2022) for a final analytic sample of 4,020 (Sample 1, n = 2,937 and Sample 2, n = 1,083). The samples were analyzed separately to see if reasons for participation were similar across different survey administrations and student populations.…”

Section: Methodsmentioning

confidence: 99%

Why Do Students Participate in Campus Sexual Assault Climate Surveys?

McMahon

Connor

Cusano

et al. 2023

J Interpers Violence

View full text Add to dashboard Cite

The purpose of the current study is to explore students’ reasons for participating in a campus sexual assault climate survey to determine if patterns exist based on various identities and experiences. More specifically, we were interested in the most frequently reported reasons for participating, and whether they differed by key demographic variables. As part of a campus climate survey administration at two campuses, 4,020 students responded to survey questions asking why they participated in the survey (Sample 1, n = 2,937 and Sample 2, n = 1,083). To examine which demographic variables predicted reasons for taking the survey while controlling for other potential correlates, we conducted logistic regression analyses. The reasons provided by students as to why they completed the campus climate survey differed by identity characteristics, suggesting that researchers must consider the demographic makeup of their institutions when designing sampling and recruitment plans and tailoring them to their student populations. For example, while gender emerged as salient for differing reasons for participation (e.g., women more likely to participate because of the issue’s importance and wanting their opinions heard more), intersectional analysis by race and gender indicates that White men (and Asian men, in Sample 1) most consistently indicated different reasons for participation, with significantly greater odds than their White and Asian women counterparts to participate for the cash incentives. This information could help improve the recruitment of representative samples for campus sexual assault surveys and therefore provide more accurate data that can be used to deliver effective intervention and prevention efforts.

show abstract

“…One of the major threats to validity in survey research comes from participants who are inattentive, respond randomly to survey questions (Kim et al, 2018; King, Kim, and McCabe, 2018; see Chandler, Paolacci and Hauser, 2020 for review), or are ‘mischievous’, providing responses that are intentionally false or misleading (Fan et al, 2006; Robinson-Cimpian, 2014; Kramer, Rubin and Coster, 2014; Fish and Russell, 2018; Kaltiala-Heino and Lindberg, 2019; Li, Follingstad. Campe, and Chahal, 2020; see Cimpian et al, 2020 for a review). Inattentive and mischievous respondents (collectively referred to in this report as ‘problematic respondents’) can bias the results of surveys by dramatically inflating point estimates and creating illusory associations.…”

Section: Introductionmentioning

confidence: 99%

Did people really drink bleach to prevent COVID-19? A tale of problematic respondents and a guide for measuring rare events in survey data

Litman

Rosen

Ronsezweig

et al. 2020

Preprint

View full text Add to dashboard Cite

Society is becoming increasingly dependent on survey research. However, surveys can be impacted by participants who are non-attentive, respond randomly to survey questions, and misrepresent who they are and their true attitudes. The impact that such respondents can have on public health research has rarely been systematically examined. In this study we examine whether Americans began to engage in dangerous cleaning practices to avoid Covid-19 infection. Prior findings reported by the CDC have suggested that people began to engage in highly dangerous cleaning practices during the Covid-19 pandemic, including ingesting household cleansers such as bleach. In a series of studies totaling close to 1400 respondents, we show that 80-90% of reports of household cleanser ingestion are made by problematic respondents. These respondents report impossible claims such as ‘recently having had a fatal heart attack’ and ‘eating concrete for its iron content’ at a similar rate to ingesting household cleaners. Additionally, respondents’ frequent misreading or misinterpreting the intent of questions accounted for the rest of such claims. Once inattentive, mischievous, and careless respondents are taken out of the analytic sample we find no evidence that people ingest cleansers to prevent Covid-19 infection. The relationship between dangerous cleaning practices and health outcomes also becomes non-significant once problematic respondents are taken out of the analytic sample. These results show that reported ingestion of household cleaners and other similar dangerous practices are an artifact of problematic respondent bias. The implications of these findings for public health and medical survey research, as well as best practices for avoiding problematic respondents in surveys are discussed.

show abstract

Identifying Invalid Responders in a Campus Climate Survey: Types, Impact on Data, and Best Indicators

Cited by 10 publications

References 30 publications

Modeling careless responding in ambulatory assessment studies using multilevel latent class analysis: Factors influencing careless responding.

Modeling careless responding in ambulatory assessment studies using multilevel latent class analysis: Factors influencing careless responding.

Why Do Students Participate in Campus Sexual Assault Climate Surveys?

Did people really drink bleach to prevent COVID-19? A tale of problematic respondents and a guide for measuring rare events in survey data

Contact Info

Product

Resources

About