Evaluation of the performance of 25 SARS-CoV-2 serological rapid diagnostic tests using a reference panel of plasma specimens at the Uganda Virus Research Institute
Abstract:Introduction
Serological testing is needed to better understand the epidemiology of the SARS-CoV-2 virus. Rapid Diagnostic Tests (RDTs) have been developed to detect specific antibodies, IgM and IgG, to the virus. We evaluated the performance of 25 of these RDTs.
Methods
A serological reference panel of 50 positive and 100 negative plasma specimens was developed from SARS-CoV-2 PCR and antibody positive patients and pre-pandemic SARS-CoV-2-negative specimens collected i… Show more
“…Parallelism was determined to assess relationships between optical densities derived from serum and plasma. WHO verification antibody panels 20/B770-02 and 20/136, monoclonal antibodies CR3022 and CR3009, and local plasmas with known reactivity ( 7 , 9 ) were used in Levy-Jennings curves to monitor detection consistency over time. Inter-operator, intra- and inter-assay precision were monitored to ensure that OD450 values from repetitive tests conformed within the expected ≤ 25% coefficient of variation (CV).…”
Section: Methodsmentioning
confidence: 99%
“…While several commercial kits and in-house assays developed elsewhere have become available ( 7 – 9 ), a validated antibody immunoassay that considers different levels of prior cross-reactivity in different populations is needed ( 10 ). Several existing commercial assays have shown high levels of inter-assay discordance ( 11 ).…”
There is an urgent need for better immunoassays to measure antibody responses as part of immune-surveillance activities and to profile immunological responses to emerging SARS-CoV-2 variants. We optimised and validated an in-house conventional ELISA to identify and quantify SARS-CoV-2 spike- (S-), receptor binding domain- (RBD-), and nucleoprotein- (N-) directed IgG, IgM, and IgA binding antibodies in the Ugandan population and similar settings. Pre- and post-pandemic specimens were used to compare the utility of mean ± 2SD, mean ± 3SD, 4-fold above blanks, bootstrapping, and receiver operating characteristic (ROC) analyses in determining optimal cut-off optical densities at 450 nm (OD) for discriminating between antibody positives and negatives. “Limits of detection” (LOD) and “limits of quantitation” (LOQ) were validated alongside the assay’s uniformity, accuracy, inter-assay and inter-operator precision, and parallelism. With spike-directed sensitivity and specificity of 95.33 and 94.15%, respectively, and nucleoprotein sensitivity and specificity of 82.69 and 79.71%, ROC was chosen as the best method for determining cutoffs. Accuracy measurements were within the expected CV range of 25%. Serum and plasma OD values were highly correlated (r = 0.93, p=0.0001). ROC-derived cut-offs for S-, RBD-, and N-directed IgG, IgM, and IgA were 0.432, 0.356, 0.201 (S), 0.214, 0.350, 0.303 (RBD), and 0.395, 0.229, 0.188 (N). The sensitivity and specificity of the S-IgG cut-off were equivalent to the WHO 20/B770-02 S-IgG reference standard at 100% level. Spike negative IgG, IgM, and IgA ODs corresponded to median antibody concentrations of 1.49, 3.16, and 0 BAU/mL, respectively, consistent with WHO low titre estimates. Anti-spike IgG, IgM, and IgA cut-offs were equivalent to 18.94, 20.06, and 55.08 BAU/mL. For the first time, we provide validated parameters and cut-off criteria for the in-house detection of subclinical SARS-CoV-2 infection and vaccine-elicited binding antibodies in the context of Sub-Saharan Africa and populations with comparable risk factors.
“…Parallelism was determined to assess relationships between optical densities derived from serum and plasma. WHO verification antibody panels 20/B770-02 and 20/136, monoclonal antibodies CR3022 and CR3009, and local plasmas with known reactivity ( 7 , 9 ) were used in Levy-Jennings curves to monitor detection consistency over time. Inter-operator, intra- and inter-assay precision were monitored to ensure that OD450 values from repetitive tests conformed within the expected ≤ 25% coefficient of variation (CV).…”
Section: Methodsmentioning
confidence: 99%
“…While several commercial kits and in-house assays developed elsewhere have become available ( 7 – 9 ), a validated antibody immunoassay that considers different levels of prior cross-reactivity in different populations is needed ( 10 ). Several existing commercial assays have shown high levels of inter-assay discordance ( 11 ).…”
There is an urgent need for better immunoassays to measure antibody responses as part of immune-surveillance activities and to profile immunological responses to emerging SARS-CoV-2 variants. We optimised and validated an in-house conventional ELISA to identify and quantify SARS-CoV-2 spike- (S-), receptor binding domain- (RBD-), and nucleoprotein- (N-) directed IgG, IgM, and IgA binding antibodies in the Ugandan population and similar settings. Pre- and post-pandemic specimens were used to compare the utility of mean ± 2SD, mean ± 3SD, 4-fold above blanks, bootstrapping, and receiver operating characteristic (ROC) analyses in determining optimal cut-off optical densities at 450 nm (OD) for discriminating between antibody positives and negatives. “Limits of detection” (LOD) and “limits of quantitation” (LOQ) were validated alongside the assay’s uniformity, accuracy, inter-assay and inter-operator precision, and parallelism. With spike-directed sensitivity and specificity of 95.33 and 94.15%, respectively, and nucleoprotein sensitivity and specificity of 82.69 and 79.71%, ROC was chosen as the best method for determining cutoffs. Accuracy measurements were within the expected CV range of 25%. Serum and plasma OD values were highly correlated (r = 0.93, p=0.0001). ROC-derived cut-offs for S-, RBD-, and N-directed IgG, IgM, and IgA were 0.432, 0.356, 0.201 (S), 0.214, 0.350, 0.303 (RBD), and 0.395, 0.229, 0.188 (N). The sensitivity and specificity of the S-IgG cut-off were equivalent to the WHO 20/B770-02 S-IgG reference standard at 100% level. Spike negative IgG, IgM, and IgA ODs corresponded to median antibody concentrations of 1.49, 3.16, and 0 BAU/mL, respectively, consistent with WHO low titre estimates. Anti-spike IgG, IgM, and IgA cut-offs were equivalent to 18.94, 20.06, and 55.08 BAU/mL. For the first time, we provide validated parameters and cut-off criteria for the in-house detection of subclinical SARS-CoV-2 infection and vaccine-elicited binding antibodies in the context of Sub-Saharan Africa and populations with comparable risk factors.
“…We used two tests in parallel to increase on performance, especially to ensure we also capture early infection since singly, these rapid diagnostic tests performed less well on IgM alone. 13 Our sensitivity, specificity, and accuracy were determined for combined IgM/IgG (either or both IgM and igG). Figure 1.…”
We report the screening and enrollment process for a phase I vaccine trial in Masaka, Uganda that investigated the safety and immunogenicity of a self-amplifying SARS-CoV-2 RNA vaccine amongst individuals with and without antibodies to SARS-CoV-2. Participant screening and enrollment were conducted between December 2021 and April 2022. Individuals were eligible if they were aged between 18 and 45 years, healthy, and never vaccinated against COVID-19. SARS-CoV-2 antibody status was determined using two point-of-care rapid tests, i.e. Multi G (MGFT3) and Standard Q (Standard Q COVID-19 IgM/IgG Plus). Data were entered and managed in OpenClinica. Analyses were performed and presented descriptively. A total of 212 individuals were screened and 43(20.3%) enrolled. The most common reasons for exclusion were ≥ grade 1 laboratory abnormalities (39, 18.4%), followed by discordant SARS-CoV-2 antibody results (23, 10.9%). While the first 38 participants were quickly enrolled over a period of 9 weeks, it took another 9 weeks to enroll the remaining five, as antibody negative participants became scarce during the surge of the Omicron variant. The SARS-CoV-2 antibody positivity rate was determined to be 60.8% and 84.4% in each half of the 18 months of screening respectively. The mean age (±Standard Deviation, SD) of screened and enrolled participants was 27.7 (±8.1) and 30.2 (±8.3) years respectively. We demonstrated that it is feasible to successfully screen and enroll participants for COVID-19 vaccine trials in Uganda in the time of a pandemic. Our experiences may be useful for investigators planning to undertake similar work in Africa.
“…In Uganda, the antigen based Rapid Diagnostics test kits (AgRDTs) and Polymerase Chain Reaction (mainly Real time PCR) testing platforms have been validated and approved for use [ 5 ]. Currently, there are 67 Ministry of Health approved laboratories across Uganda conducting COVID19 PCR testing.…”
Background
Significant milestones have been made in the development of COVID19 diagnostics Technologies. Government of the republic of Uganda and the line Ministry of Health mandated Uganda Virus Research Institute to ensure quality of COVID19 diagnostics. Re-testing was one of the methods initiated by the UVRI to implement External Quality assessment of COVID19 molecular diagnostics.
Method
participating laboratories were required by UVRI to submit their already tested and archived nasopharyngeal samples and corresponding meta data. These were then re-tested at UVRI using the WHO Berlin protocol, the UVRI results were compared to those of the primary testing laboratories in order to ascertain performance agreement for the qualitative & quantitative results obtained. Ms Excel window 12 and GraphPad prism ver 15 was used in the analysis. Bar graphs, pie charts and line graphs were used to compare performance agreement between the reference Laboratory and primary testing Laboratories.
Results
Eleven (11) Ministry of Health/Uganda Virus Research Institute COVID19 accredited laboratories participated in the re-testing of quality control samples. 5/11 (45%) of the primary testing laboratories had 100% performance agreement with that of the National Reference Laboratory for the final test result. Even where there was concordance in the final test outcome (negative or positive) between UVRI and primary testing laboratories, there were still differences in CT values. The differences in the Cycle Threshold (CT) values were insignificant except for Tenna & Pharma Laboratory and the UVRI(p = 0.0296). The difference in the CT values were not skewed to either the National reference Laboratory(UVRI) or the primary testing laboratory but varied from one laboratory to another. In the remaining 6/11 (55%) laboratories where there were discrepancies in the aggregate test results, only samples initially tested and reported as positive by the primary laboratories were tested and found to be false positives by the UVRI COVID19 National Reference Laboratory.
Conclusion
False positives were detected from public, private not for profit and private testing laboratories in almost equal proportion. There is need for standardization of molecular testing platforms in Uganda. There is also urgent need to improve on the Laboratory quality management systems of the molecular testing laboratories in order to minimize such discrepancies.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
