Assessing the validity and reliability of self-report data on contraception use in the MObile Technology for Improved Family Planning (MOTIF) randomised controlled trial

Abstract

BackgroundA variety of different approaches to measuring contraceptive use have been used or proposed, either to assess current use or adherence over time, using subjective or objective measures. This paper reports an overview of approaches to measuring adherence to the oral contraceptive, intra-uterine device, sub-dermal implant, and injectable and describes how we assessed contraception use in the MObile Technology for Improved Family Planning (MOTIF) trial in Cambodia.Main bodyWe summarise and discuss advantages and disadvantages of different subjective and objective approaches to measuring adherence to the oral contraceptive, intra-uterine device, sub-dermal implant, and injectable such as self-reports, clinic records, electronic monitoring devices, clinical examination and biomarkers.For the MOTIF trial, we did not consider it feasible to measure objective contraception use as many participants lived a long distance from the clinic and we were concerned whether it was appropriate to ask women to return to clinic for a physical examination simply to verify self-report information already provided.We aimed to assess the validity of the four-month data with 50 participants, calculating the sensitivity and specificity of self-reported data compared with objective measurement. For the 46 valid measurements obtained, the sensitivity and specificity was 100% for self-reported contraception use compared to objective measurement but this study had some limitations. To assess reliability of self-report data we compared calendar data collected on effective contraception use at months 1–4 post-abortion, collected separately at four and 12 months. Agreement ranged from 80 to 84% with a kappa statistic ranging from 0·59 to 0·67 indicating fair to good agreement.ConclusionThere is no perfect method of assessing contraception use and researchers designing future studies should give consideration of what to measure, for example current use or detailed patterns of use over time, and remain mindful of what will be feasible and acceptable to the study population. Although self-reported data on contraception use are considered less reliable, and prone to social desirability bias, it is often the standard approach for contraception research and provides data comparable to previous studies.Trial registrationClinicalTrials.gov Identifier: NCT01823861. Registered: March 30, 2013.

Free contraception and behavioural nudges in the postpartum period: evidence from a randomised control trial in Nairobi, Kenya

“…As self-reports of contraceptive use are costly and onerous to verify, there is limited rigorous evidence of the validity of self-reported contraceptive use. 36 In our study, we generally see consistency across self-report outcomes and outcomes from administrative data. We surveyed new mothers at high frequency in the postpartum period in order to avoid relying on long recall periods to reconstruct the timing of their use of postpartum family planning.…”

Non-adherence among women enrolled in a contraceptive vaginal ring use study in Kisumu, Kenya, 2014-2015

“…Moreover, such objective measures may not be entirely clear-cut in their ability to distinguish inconsistent users from consistent users. 28 Given small count, our pregnancy rate was estimated with low precision and reliability, and should be interpreted with caution. Lastly, NuvaRing® may offer a slight allowance for imperfect use given that removal of the ring for up to three hours for sexual or hygiene purposes does not compromise contraceptive efficacy.…”

“…A gold standard for assessing adherence to modern contraceptive methods is not available. 28 Our definitions of particular measures, in particular non-adherence by selfreport, non-adherence by pharmacy record, may have produced imprecisions with either observation or measurement process. Measures for assessing CVR non-adherence in the literature are largely limited to self-report and subject to under-reporting of non-adherent behavior.…”

Non-adherence among women enrolled in a contraceptive vaginal ring use study in Kisumu, Kenya, 2014-2015

“…A gold standard for assessing adherence to modern contraceptive methods is not available (29). Our definitions of particular measures, in particular non-adherence by self-report, non-adherence by pharmacy record, may have produced imprecisions with either observation or measurement process.…”

“…Performing residual hormone analysis on the returned rings from all women in our sample was not feasible. Moreover, such objective measures may not be entirely clear-cut in their ability to distinguish inconsistent users from consistent users (29). Given small count, our pregnancy rate was estimated with low precision and reliability, and should be interpreted with caution.…”