Threats to Online Surveys: Recognizing, Detecting, and Preventing Survey Bots

Paths to transformation across contemporary reading practices: The role of motivations and genre preferences.

“…Nine hundred and twenty-nine people participated in the study, of which 115 were excluded due to unserious participation, by following the guidelines for automated and manual detection of survey bots (Xu et al, 2022). The final merged sample consists of 814 participants from three conditions: books ( n = 317), digital fiction ( n = 261), and Wattpad ( n = 236).…”

A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research.

“…Bots are agent-like software programs that automate computer-based tasks (Franklin & Graesser, 2015), and they are increasingly prevalent across the internet (Tsvetkova et al, 2017). Bots and other forms of fraudulent respondents (i.e., human participants who enroll in a study multiple times) are participating in online research studies at an alarming rate (Griffin et al, 2022; Simone, 2019; Xu et al, 2022). This brings cause for concern, as bots have the capacity to submit hundreds to thousands of responses to online research within a matter of hours (Burnette et al, 2022; Prince et al, 2012; Xu et al, 2022; Yarrish et al, 2019).…”

“…Bots and other forms of fraudulent respondents (i.e., human participants who enroll in a study multiple times) are participating in online research studies at an alarming rate (Griffin et al, 2022; Simone, 2019; Xu et al, 2022). This brings cause for concern, as bots have the capacity to submit hundreds to thousands of responses to online research within a matter of hours (Burnette et al, 2022; Prince et al, 2012; Xu et al, 2022; Yarrish et al, 2019). While some bots are programmed to perform benevolent actions (e.g., moderating hate speech on group forums; Tsvetkova et al, 2017), others contribute to costly losses across public domains ranging from network security to knowledge production (Geer, 2005; Orabi et al, 2020) because they are programmed to manipulate information maliciously (e.g., injecting misinformation in political elections; Orabi et al, 2020).…”

“…In recent years, social scientists have introduced and evaluated dozens of tactics to screen for bots in online survey research (Burnette et al, 2022; Griffin et al, 2022; Simone, 2019; Xu et al, 2022). Much of the work in this area has focused on question-based (i.e., screening strategies that add nonsubstantive items to surveys solely for the purpose of detecting bots) and data-based (i.e., screening strategies that leverage participant data to identify inconsistencies or patterns in participant experience) bot screening tools.…”

“…To this end, common question-based bot screening tools examined in the literature include items that are not visible to humans but detectable by bots (i.e., “honeypots”), attention checks (e.g., “please select strongly agree”), and ReCAPTCHA tools (i.e., small puzzles presented before completing a specific online activity, such as clicking pictures that correspond to text; Griffin et al, 2022; Simone, 2019; Storozuk et al, 2020). In other instances, researchers who described experiencing an overwhelming number of fraudulent respondents proposed several data-based screening tools, which may be used to flag respondents in existing data (Griffin et al, 2022; Pozzar et al, 2020; Xu et al, 2022). Common data-based screening tools described in the literature include (a) evaluating qualitative responses for duplicate phrasing or illogical answers, (b) identifying discrepancies to verifiable items, and (c) identifying identical response patterns to quantitative surveys (Griffin et al, 2022; Levi et al, 2021; Pozzar et al, 2020; Simone, 2019).…”