Accuracy of self-reported COVID-19 vaccination status compared with a public health vaccination registry in Québec : observational diagnostic study

Abstract

Background: The accuracy of self-reported vaccination status is important to guide real-world vaccine effectiveness studies and policy making in jurisdictions where access to electronic vaccine registries is restricted.

Objective: This study aimed to determine the accuracy of self-reported vaccination status and reliability of the self-reported number of doses, brand, and time of vaccine administration.

Methods: This diagnostic accuracy study was completed by the Canadian COVID-19 Emergency Department Rapid Response Network. We enrolled consecutive patients presenting to 4 emergency departments (EDs) in Québec between March 24, 2020, and December 25, 2021. We included adult patients who were able to consent, could speak English or French, and had a proven COVID-19 infection. We compared the self-reported vaccination status of the patients with their vaccination status in the electronic Québec Vaccination Registry. Our primary outcome was the accuracy of the self-reported vaccination status (index test) ascertained during telephone follow-up compared with the Québec Vaccination Registry (reference standard). The accuracy was calculated by dividing all correctly self-reported vaccinated and unvaccinated participants by the sum of all correctly and incorrectly self-reported vaccinated and unvaccinated participants. We also reported interrater agreement with the reference standard as measured by unweighted Cohen κ for self-reported vaccination status at telephone follow-up and at the time of their index ED visit, number of vaccine doses, and brand.

Results: During the study period, we included 1361 participants. At the time of the follow-up interview, 932 participants reported at least 1 dose of a COVID-19 vaccine. The accuracy of the self-reported vaccination status was 96% (95% CI 95%-97%). Cohen κ for self-reported vaccination status at phone follow-up was 0.91 (95% CI 0.89-0.93) and 0.85 (95% CI 0.77-0.92) at the time of their index ED visit. Cohen κ was 0.89 (95% CI 0.87-0.91) for the number of doses, 0.80 (95% CI 0.75-0.84) for the brand of the first dose, 0.76 (95% CI 0.70-0.83) for the brand of the second dose, and 0.59 (95% CI 0.34-0.83) for the brand of the third dose.

Conclusions: We reported a high accuracy of self-reported vaccination status for adult patients without cognitive disorders who can express themselves in English or French. Researchers can use self-reported COVID-19 vaccination data on the number of doses received, vaccine brand name, and timing of vaccination to guide future research with patients who are capable of self-reporting their vaccination data. However, access to official electronic vaccine registries is still needed to determine the vaccination status in certain susceptible populations where self-reported vaccination data remain missing or impossible to obtain.

Trial Registration: Clinicaltrials.gov NCT04702945; https://clinicaltrials.gov/ct2/show/NCT04702945