Conducting Quantitative Research with Hard-To-Reach-Online Populations: Using Prime Panels to Rapidly Survey Older Adults During a Pandemic

Abstract

Vulnerable populations (e.g., older adults) can be hard to reach online. During a pandemic like COVID-19 when much research data collection must be conducted online only, these populations risk being further underrepresented. This paper explores methodological strategies for rigorous, efficient survey research with a large number of older adults online, focusing on (1) the design of a survey instrument both comprehensible and usable by older adults, (2) rapid collection (within hours) of data from a large number of older adults, and (3) validation of data using attention checks, independent validation of age, and detection of careless responses to ensure data quality. These methodological strategies have important implications for the inclusion of older adults in online research.

Trust in COVID‐19 public health information

“…This study employed a cross‐sectional survey design. We used Prime Panels (Chandler et al, 2019 ; Verma et al, 2021 ) to recruit participants online. We collected data in five batches between June 26 and July 20, 2020, monitoring for data quality (see Section 3.3 for details) after each batch.…”

“…We used stratified sampling by age to ensure a sufficient number of older adult participants, given the degree to which online panel participants generally skew younger than the overall population (Chandler et al, 2019 ) and given the degree to which COVID‐19 has disproportionately impacted older adults (Xie, Charness, et al, 2020 ). Thus, we attempted to recruit roughly equal numbers of older (65+) and younger (18–64) adults (Verma et al, 2021 ). We obtained 669 responses from 397 (59.3%) adults aged 18–64 years and from 272 (40.7%) adults aged 65+ years.…”

“…Specifically, we chose to collect a stratified sample in terms of age given the disproportionate impacts of COVID‐19 on older adults (Xie, Charness, et al, 2020 ). Whereas many other studies oversample younger adults based upon limitations in online subject pools, we intentionally oversampled older adults (Verma et al, 2021 ). Further, during our validation process, we found that the data provided by older adults was of higher quality, and thus led to a larger proportion of valid responses.…”

“…Future studies could stratify on other factors such as racioethnicity or political leaning. Given biases commonly found in online datasets, it is important for some studies to intentionally oversample rather than always undersampling groups that are harder‐to‐study‐online (Verma et al, 2021 ).…”

AFFORCE: Actionable Framework for Designing Crowdsourcing Experiences for Older Adults

“…Therefore, based on our experience with using, designing and developing crowdsourcing platforms, and a keyword-driven literature review done in a form of a mind-map, which allowed us to cluster relevant findings, in this article we envision the way forward to create more engaging, accessible and inclusive experience of crowdwork, encouraging contributions from underrepresented populations [31,34]. Designing crowdsourcing experiences with older adults in mind, due to the curb-cut effect 1 , will not only improve the experience for them, but also for the general population.…”