Accuracy of Self-Reported COVID-19 Vaccination Status Compared With a Public Health Vaccination Registry in Québec: Observational Diagnostic Study

Abstract

Background The accuracy of self-reported vaccination status is important to guide real-world vaccine effectiveness studies and policy making in jurisdictions where access to electronic vaccine registries is restricted. Objective This study aimed to determine the accuracy of self-reported vaccination status and reliability of the self-reported number of doses, brand, and time of vaccine administration. Methods This diagnostic accuracy study was completed by the Canadian COVID-19 Emergency Department Rapid Response Network. We enrolled consecutive patients presenting to 4 emergency departments (EDs) in Québec between March 24, 2020, and December 25, 2021. We included adult patients who were able to consent, could speak English or French, and had a proven COVID-19 infection. We compared the self-reported vaccination status of the patients with their vaccination status in the electronic Québec Vaccination Registry. Our primary outcome was the accuracy of the self-reported vaccination status (index test) ascertained during telephone follow-up compared with the Québec Vaccination Registry (reference standard). The accuracy was calculated by dividing all correctly self-reported vaccinated and unvaccinated participants by the sum of all correctly and incorrectly self-reported vaccinated and unvaccinated participants. We also reported interrater agreement with the reference standard as measured by unweighted Cohen κ for self-reported vaccination status at telephone follow-up and at the time of their index ED visit, number of vaccine doses, and brand. Results During the study period, we included 1361 participants. At the time of the follow-up interview, 932 participants reported at least 1 dose of a COVID-19 vaccine. The accuracy of the self-reported vaccination status was 96% (95% CI 95%-97%). Cohen κ for self-reported vaccination status at phone follow-up was 0.91 (95% CI 0.89-0.93) and 0.85 (95% CI 0.77-0.92) at the time of their index ED visit. Cohen κ was 0.89 (95% CI 0.87-0.91) for the number of doses, 0.80 (95% CI 0.75-0.84) for the brand of the first dose, 0.76 (95% CI 0.70-0.83) for the brand of the second dose, and 0.59 (95% CI 0.34-0.83) for the brand of the third dose. Conclusions We reported a high accuracy of self-reported vaccination status for adult patients without cognitive disorders who can express themselves in English or French. Researchers can use self-reported COVID-19 vaccination data on the number of doses received, vaccine brand name, and timing of vaccination to guide future research with patients who are capable of self-reporting their vaccination data. However, access to official electronic vaccine registries is still needed to determine the vaccination status in certain susceptible populations where self-reported vaccination data remain missing or impossible to obtain. Trial Registration Clinicaltrials.gov NCT04702945; https://clinicaltrials.gov/ct2/show/NCT04702945

Determinants of SARS-CoV-2 IgG response and decay in Canadian healthcare workers: a prospective cohort study

“…Although the information on vaccinations and infection was self-reported, the participants were all HCWs who may be more accurate in their reporting than the general population. Previous studies have shown that, in patients, self-report of vaccination status was very largely concordant with vaccination records but that self-reported dates and type of vaccine were less accurate [24] [25]. In the current study public health records were accessed for a majority of the serology sub-cohort and the congruence in estimates using purely self-report episodes and episodes corrected from external records gives some confidence to the findings for the serology cohort overall.…”

Factors Associated with COVID‐19 Vaccine Uptake Among High School Students in a Large, Diverse Metropolitan Area

“…Self-reported COVID-19 vaccination status appears to have good accuracy. [15][16][17][18] Participants rated 15 items regarding their attitudes toward the COVID-19 pandemic, the COVID-19 vaccine, and a possible vaccine mandate; 11 of the questions were from national surveys 12,13 and 4 were added to investigate concern about COVID-19, perceived quality of public health messaging, and support from family and friends. They also rated the trustworthiness of 11 sources of information on the COVID-19 vaccine (eg, family, friends, medical professionals, social media, US President); 9 questions were from a national survey 10 and 2 additional questions assessed the possible impact of celebrities and the US President on COVID-19 vaccine opinions.…”

Perceived Risk for Severe COVID-19 and Oral Antiviral Use Among Antiviral-Eligible US Adults

“…Crude (perceived risk only) agreement between the two [47][48][49]. Further, selfreported health conditions [50] and use of medication [51] have been found to have high specificity and moderate sensitivity; thus, we may have underestimated the prevalence of health conditions or oral antiviral use.…”