Side-by-Side Comparison of Three Fully Automated SARS-CoV-2 Antibody Assays with a Focus on Specificity

Abstract

Background In the context of the COVID-19 pandemic, numerous new serological test systems for the detection of anti-SARS-CoV-2 antibodies rapidly have become available. However, the clinical performance of many of these is still insufficiently described. Therefore, we compared three commercial, CE-marked, SARS-CoV-2 antibody assays side by side. Methods We included a total of 1,154 specimens from pre-COVID-19 times and 65 samples from COVID-19 patients (≥14 days after symptom onset) to evaluate the test performance of SARS-CoV-2 serological assays by Abbott, Roche, and DiaSorin. Results All three assays presented with high specificities: 99.2% (98.6-99.7) for Abbott, 99.7% (99.2-100.0) for Roche, and 98.3% (97.3-98.9) for DiaSorin. In contrast to the manufacturers’ specifications, sensitivities only ranged from 83.1% to 89.2%. Although the three methods were in good agreement (Cohen’s Kappa 0.71-0.87), McNemar tests revealed significant differences between results obtained from Roche and DiaSorin. However, at low seroprevalences, the minor differences in specificity resulted in profound discrepancies of positive predictive values at 1% seroprevalence: 52.3% (36.2-67.9), 77.6% (52.8-91.5), and 32.6% (23.6-43.1) for Abbott, Roche, and DiaSorin, respectively. Conclusion We found diagnostically relevant differences in specificities for the anti-SARS-CoV-2 antibody assays by Abbott, Roche, and DiaSorin that have a significant impact on the positive predictive values of these tests.

Anti-SARS-CoV-2 IgM improves clinical sensitivity early in disease course

“…The higher overall pooled sensitivity reported in this systematic review and meta -analysis compared to our study is likely due to its higher combined sample size of over 29,000 tests and the fact that 60% of our samples were analyzed within three weeks of PCR positivity. Perkmann, et al also evaluated Abbott anti-SARS-CoV-2 IgG, DiaSorin anti-SARS-CoV-2 IgG, and Roche anti-SARS-CoV-2 total and reported sensitivities more similar to those observed in our study, ranging from 83.1% to 89.2% in 65 samples from COVID-19 patients evaluated ≥ 14 days after symptom onset [22] . However, as is apparent in our study and others [10] , [23] , sensitivity of the different immunoglobulin classes differ depending on days post symptom onset or PCR positivity.…”

“…A meta -analysis reported overall specificity for anti-SARS-CoV-2 IgG and IgM assays of 99.1% (44 studies included) and 98.7% (41 studies included), respectively [9] . Perkmann, et al previously reported specificities of 99.2% for Abbott anti-SARS-CoV-2 IgG, 98.3% for DiaSorin anti-SARS-CoV-2 IgG, and 99.7% for Roche anti-SARS-CoV-2 total using over 1,154 pre-COVID-19 specimens [22] . Other studies reported specificities of 99.9% using 1,020 serum specimens collected prior to SARS-CoV-2 circulation [25] and 99.3% using 107 serum specimens collected either before the emergence of SARS-CoV-2 or from PCR negative patients [26] for Abbott anti-SARS-CoV-2 IgG.…”

Population-based seroprevalence of SARS-CoV-2 is more than halfway through the herd immunity threshold in the State of Maranhão, Brazil

“…6 In this study, we used the Elecsys® Anti-SARS-CoV-2 electrochemiluminescence immunoassay, which presented a high specificity rate of 99•7% (95%CI 99•2-100•0) and a positive predictive value (PPV) of 97•4% with a 10% seroprevalence rate. 31 We have likely achieved a higher PPV because the prevalence in our survey was four times that seroprevalence rate. It has been shown that electrochemiluminescence immunoassays present higher sensitivity than lateral flow immunoassays.…”

Evaluation of Three Commercial SARS-CoV-2 Serologic Assays and Their Performance in Two-Test Algorithms

“…In the two-test approach, the overall result was considered positive if the sample tested positive by both assays and negative if the sample was negative using one or both assays. When calculating the positive predictive value (PPV) or negative predictive value (NPV) at various prevalence rates, assay sensitivity was defined as the observed sensitivity using samples collected >14 days after symptom onset (when sensitivity is reported to be its highest) ( 8 – 10 ). Specificity was defined (for PPV and NPV calculations) as the value obtained when all control samples were combined.…”

“…Considering the relatively low prevalence of COVID-19 infection in many tested populations and the implications of false-positive results for patient care and public health measures, the Centers for Disease Control and Prevention (CDC) has determined that highly specific (≥99.5%) serologic tests are required to provide adequate positive predictive value (PPV) ( 7 ). Although high specificity is reported for many commercial SARS-CoV-2 serologic assays, not all of them consistently meet this specificity threshold (see Table S1 in the supplemental material) ( 8 – 10 ). A potential approach to ensure consistently high specificity involves the application of a two-test algorithm in which reactivity using one assay is confirmed using a different (orthogonal) assay.…”