The Importance of Measuring and Accounting for Potential Biases in Respondent-Driven Samples

Abstract

Respondent-driven sampling (RDS) is often viewed as a superior method for recruiting hard-to-reach populations disproportionately burdened with poor health outcomes. As an analytic approach, it has been praised for its ability to generate unbiased population estimates via post-stratified weights which account for non-random recruitment. However, population estimates generated with RDSAT (RDS Analysis Tool) are sensitive to variations in degree weights. Several assumptions are implicit in the degree weight and are not routinely assessed. Failure to meet these assumptions could result in inaccurate degree measures and consequently result in biased population estimates. We highlight potential biases associated with violating the assumptions implicit in degree weights for the RDSAT estimator and propose strategies to measure and possibly correct for biases in the analysis.

A Comparison Between Respondent-Driven Sampling and Time-Location Sampling Among Men Who Have Sex with Men in Shenzhen, China

“…For these features, RDS presents to be an effective sampling method with statistical rigor (Kendall et al, 2008;Wei, McFarland, Colfax, Fuqua, & Raymond, 2012). However, it is still an experimental methodology being used for surveillance of high risk groups of HIV/AIDS and debate about the underlying assumptions remains (Rudolph, Fuller, & Latkin, 2013). Through creation of a sampling frame that comprises the universe of venues, days, and time periods where and when the population congregates, TLS systematically samples potential participants at randomly selected venue-day-time periods (Ferreira, de Oliveira, Raymond, Chen, & McFarland, 2008;Karon & Wejnert, 2012;Kendall et al, 2008;Wei et al, 2012).…”

Spatial, temporal and relational patterns in respondent-driven sampling: evidence from a social network study of rural drug users

“…For example, because participants were compensated for redeemed referrals, it is possible that recruiters preferentially referred those who they perceived to be able to participate (eg, residing near or having transportation to the study office). Future studies would benefit from a follow-up survey about logistics and preferences surrounding recruitment 17. Second, use of travel distance may not have fully captured all factors affecting proximity, routes and travel time (ie, speed limits, construction, etc.).…”

“…Studies comparing demographic characteristics and risk behaviours of RDS peer recruits with those of egocentric network members reported by RDS participants typically reported differences, suggesting that peer recruitment may not be random 12–15. Two studies demonstrated the presence of non-random recruitment in respondent-driven samples,14 16 and some suggest that recruitment probability is likely driven by factors other than network size 17. Yet, few studies have used sociometric network data to evaluate the influence of demographic and behavioural similarity and relationship characteristics on RDS recruitment.…”

Collecting data from migrants in Ghana: Lessons learned using respondent-driven sampling

“…Public health researchers in particular may find these estimates and RDS's "misleadingly narrow" confidence intervals unsuitable for disease surveillance (Goel and Salganik 2010: 6746). Biases may also exist around recruitment, causing several researchers to inquire about willingness to participate or ability to recruit (CarballoDiéguez et al 2011;Liu et al 2012;Rudolph, Fuller, and Latkin 2013). A small but growing body of RDS literature addresses the need to "identify potential recruitment biases so that they can be either acknowledged as potential limitations or corrected for in the analysis" (Rudolph, Fuller, and Latkin 2013: 5).…”

“…RDS permits data capture on nonrespondents when participants return to collect their recruiting bonuses, shedding light on specific factors that affect a population's willingness to participate. Based on this information and evidence in the literature on recruitment biases (Liu et al 2012;Rudolph, Fuller, Latkin 2013), a fourth PNS question was added to the survey questionnaire in an attempt to acknowledge participants' inherent existing biases and more accurately measure PNS. When compared to PNS question 103, this fourth PNS question may help identify or minimize the potential effects of participants not recruiting individuals within their networks randomly (e.g., not recruiting individuals who plan to migrate soon, recruiting someone who needs money, not discussing the research with sceptical individuals who might decline a coupon).…”