Use of Propensity Score Matching to Create Counterfactual Group to Assess Potential HIV Prevention Interventions

Abstract

BackgroundThe design of HIV prevention trials in the context of effective HIV preventive methods is a challenge. Alternate designs, including using non-randomised ‘observational control arms’ have been proposed. We used HIV simulated vaccine efficacy trials (SiVETs) to show pitfalls that may arise from using such observational controls and suggest how to conduct the analysis in the face of the pitfalls.MethodsTwo SiVETs were nested within previously established observational cohorts of fisherfolk and female sex workers (FSW) in Uganda. SiVET participants received a licensed Hepatitis B vaccine in a schedule (0, 1 and 6 months) similar to that for a possible HIV vaccine efficacy trial. All participants received HIV counselling and testing every quarter for one year to assess HIV incidence rate ratio (IRR) between SiVET and non-SiVET (observational data). Propensity scores, conditional on baseline characteristics were calculated for SiVET participation and matched between SiVET and non-SiVET in the period before and during the SiVET study. We compared IRR before and after propensity score matching (PSM).Results In total, 3989 participants were enrolled into observational cohorts prior to SiVET, (1575 FF prior to Jul 2012 and 2414 FSW prior to Aug 2014). SiVET enrolled 572 participants (Jul 2012 to Apr 2014 in FF & Aug 2014 to Apr 2017 in FSW), with 953 non-SiVET participants observed in the SiVET concurrent period and 2928 from the pre-SiVET period (before Jul 2012 in FF or before Apr 2014 in FSW). Imbalances in baseline characteristics were observed between SiVET and non-SiVET participants in both periods before PSM. Similarly, HIV incidence was lower in SiVET than non-SiVET; SiVET-concurrent period, IRR= 0.59, 95%CI: 0.31-0.68, p=0.033 and pre-SiVET period, IRR= 0.77, 95%CI: 0.43-1.29, p=0.161. After PSM, participants baseline characteristics were comparable and there were minimal differences in HIV incidence between SiVET and non-SiVET participants. ConclusionThe process of screening for eligibility for efficacy trial selects participants with baseline characteristics different from the source population, confounding any observed differences in HIV incidence. PSM can be a useful tool to adjust the imbalance in the measured participants’ baseline characteristics creating a counterfactual group to estimate the effect of interventions on HIV incidence.

Counterfactual groups to assess treatment efficacy in HIV prevention trials in high risk populations in Uganda

“…Literature shows that participants that volunteer to join clinical trials are often different from those that do not and the HIV incidence is likely to differ [9,24,25]. Participants with high risk behaviour are less likely to volunteer as trial participants [3]. Compared to SiVETs, OBCs in both periods were dominated by participants with characteristics that have been previously associated with high risk of HIV infection such as young age, low edication level, having a new sexual partner, low condom use with new sexual partners, genital discharge and genital ulcer disease infection [12,13,[26][27][28][29].…”

“…In 2020 alone, 1.5 million people became newly infected with HIV, 58% in SSA [1]. The new infections in SSA are largely driven by the suboptimal adherence to available HIV prevention interventions [2,3]. An effective and affordable HIV vaccine could be the only hope to minimise non adherence and end HIV acquistion.…”

“…In our previous analysis [3], we showed how propensity scores (PS) and propensity score matching (PSM) statistical approaches that attempt to estimate treatment effect [10,11] can be used to create counterfactual trial arms enabling estimation of unbiased hypothetical arm HIV incidence. This paper [3] considered data collected in simulated HIV vaccine e cacy trials (SiVETs) [9] drawing controls from observational cohorts conducted in the same population prior to or during the SiVETs [9]. The HIV incidence data is available from multiple sources such as observational data in other populations and at different time points or even from placebo arms of previous trials in diverse populations.…”

“…Provision of these interventions in the trial, deminishes the background HIV incidence observed in the population from which subjects enrolled in e cacy trials are obtained hence requiring long-term follow up or extremely large sample sizes. E cacy trial investigators have proposed use of HIV incidence obtained from observational controls or earlier trials [3,6,7] or Averted Infection Ratio (AIR) (i.e. dividing the rate difference between hypothetical placebo and experimental arms by the rate difference between hypothetical placebo and active control arms) to estimate comparator HIV incidence [8].…”

“…Each approach has it challenges, for instance, participants that join clinical trials are different from those that do not, making a comparison to observational controls biased [9]. Secondly, earlier clinical trials may have been conducted in a different population or may suffer from the effect of time and improvements in healthcare standards [3]. Those proposing AIR, suggest use of HIV incidence from a run-in period cohort or that from another trial or epidemiological surveillance, all which might introduce bias because of the differences in time and/or population.…”