Bayesian analysis of tests with unknown specificity and sensitivity

Gelman, Andrew; Carpenter, Bob

doi:10.1101/2020.05.22.20108944

Cited by 60 publications

(101 citation statements)

References 15 publications

(33 reference statements)

Supporting

Mentioning

101

Contrasting

Order By: Relevance

“…We estimated seroprevalence and 95% credible intervals (CI) using a hierarchical Bayesian framework to account for diagnostic sensitivity and specificity. [5][6][7][8][9][10][11] Stratified models were used to estimate seroprevalence by age, sex, ZIP Code, ethnicity, employment status, and a priori participant-reported COVID-19 status.…”

Section: Analytic Methodsmentioning

confidence: 99%

High Community SARS-CoV-2 Antibody Seroprevalence in a Ski Resort Community, Blaine County, Idaho, US. Preliminary Results

McLaughlin

Doll

Morrison

et al. 2020

Preprint

View full text Add to dashboard Cite

Community-level seroprevalence surveys are needed to determine the proportion of the population with previous SARS-CoV-2 infection, a necessary component of COVID-19 disease surveillance. In May, 2020, we conducted a cross-sectional seroprevalence study of IgG antibodies for nucleocapsid of SARS-CoV-2 among the residents of Blaine County, Idaho, a ski resort community with high COVID-19 attack rates in late March and Early April (2.9% for ages 18 and older). Participants were selected from volunteers who registered via a secure web link, using prestratification weighting to the population distribution by age and gender within each ZIP Code. Participants completed a survey reporting their demographics and symptoms; 88% of volunteers who were invited to participate completed data collection survey and had 10 ml of blood drawn. Serology was completed via the Abbott Architect SARS-CoV-2 IgG immunoassay. Primary analyses estimated seroprevalence and 95% credible intervals (CI) using a hierarchical Bayesian framework to account for diagnostic uncertainty. Stratified models were run by age, sex, ZIP Code, ethnicity, employment status, and a priori participant-reported COVID-19 status. Sensitivity analyses to estimate seroprevalence included base models with post-stratification for ethnicity, age, and sex, with or without adjustment for multi-participant households. IgG antibodies to the virus that causes COVID-19 were found among 22.7% (95% CI: 20.1%, 25.5%) of residents of Blaine County. Higher levels of antibodies were found among residents of the City of Ketchum 34.8% (95% CI 29.3%, 40.5%), compared to Hailey 16.8% (95%CI 13.7%, 20.3%) and Sun Valley 19.4% (95% 11.8%, 28.4%). People who self-identified as not believing they had COVID-19 had the lowest prevalence 4.8% (95% CI 2.3%, 8.2%). The range of seroprevalence after correction for potential selection bias was 21.9% to 24.2%. This study suggests more than 80% of SARS-CoV-2 infections were not reported. Although Blaine County had high levels of SARS-CoV-2 infection, the community is not yet near the herd immunity threshold.

show abstract

Section: Analytic Methodsmentioning

confidence: 99%

High Community SARS-CoV-2 Antibody Seroprevalence in a Ski Resort Community, Blaine County, Idaho, US. Preliminary Results

McLaughlin

Doll

Morrison

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…We began by mentioning that standard serological diagnostics use a single threshold to decide between positive and negative cases, and then estimate how frequently this is correct in two directions: sensitivity and specificity. It is possible to apply principled adjustments to these hard-threshold models when estimating population prevalence, correcting for the expected proportion of false positives and false negatives [6]. But the underlying hard-threshold model treats a sample whose measurement is just over the threshold as having the same uncertainty as a strong positive signal.…”

Section: Discussionmentioning

confidence: 99%

“…Finally, future work will require comparison to emerging approaches that either correct for uncertain sensitivity and specificity when estimating seroprevalence [6], or that use cutoff-free approaches, but in a generative modeling framework [7]. Figure 2: Posterior prevalence trajectories with increasing population data, due to more samples at each time (top to bottom), and sampling at more times (left to right).…”

Section: Discussionmentioning

confidence: 99%

Discriminative Bayesian Serology: Counting Without Cutoffs

Christian

Murrell

2020

Preprint

View full text Add to dashboard Cite

During the emergence of a pandemic, we need to estimate the prevalence of a disease using serological assays whose characterization is incomplete, relying on limited validation data. This introduces uncertainty for which we need to account. In our treatment, the data take the form of continuous assay measurements of antibody response to antigens (eg. ELISA), and fall into two groups. The training data includes the confirmed positive or negative infection status for each sample. The population data includes only the assay measurements, and is assumed to be a random sample from the population from which we estimate the seroprevalence. We use the training data to model the relationship between assay values and infection status, capturing both individual-level uncertainty in infection status, as well as uncertainty due to limited training data. We then estimate the posterior distribution over population prevalence, additionally capturing uncertainty due to finite samples. Finally, we introduce a means to pool information over successive time points, using a Gaussian process, which dramatically reduces the variance of our estimates. The methodological approach we here describe was developed to support the longitudinal characterization of the seroprevalence of COVID-19 in Stockholm, Sweden.

show abstract

“…We used a Bayesian modeling approach to estimate population seroprevalence based on the results of the two antibody testing platforms, accounting for test performance characteristics. [ 24 ] For validation data, we used the package insert data for the Abbott test[ 17 ] (1066 of 1070 negative controls tested negative; 88 of 88 PCR+ positive controls tested positive), and in-house validation of the ELISA test (95 of 95 negative controls tested negative; 42 of 44 positive controls tested positive). We note that test sensitivities were based on different patient populations (Abbott test based on hospitalized patients, ELISA based predominantly on ambulatory patients) and therefore not directly comparable.…”

Section: Methodsmentioning

confidence: 99%

Universal PCR and antibody testing demonstrate little to no transmission of SARS-CoV-2 in a rural community

Appa¹,

Takahashi²,

Rodríguez-Barraquer³

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: The absence of systematic surveillance for SARS-CoV-2 has curtailed accurate appraisal of transmission intensity. Our objective was to perform case detection of an entire rural community to quantify SARS-CoV-2 transmission using PCR and antibody testing. Methods: We conducted a cross-sectional survey of the prevalence and cumulative incidence of SARS-CoV-2 infection in the rural town of Bolinas, California (population 1,620), four weeks following shelter-in-place orders. Residents and county essential workers were tested between April 20th-24th, 2020. Prevalence by PCR and seroprevalence combining data from two forms of antibody testing were performed in parallel (Abbott ARCHITECT IgG to nucleocapsid protein and in-house IgG ELISA to the receptor binding domain). Results: Of 1,891 participants, 1,312 were confirmed Bolinas residents (>80% community ascertainment). Zero participants were PCR positive. Assuming 80% sensitivity, it would have been unlikely to observe these results (p<0.05) if there were >3 active infections in the community. Based on antibody results, estimated prevalence of prior infection was 0.16% (95% CrI: 0.02%, 0.46%). Seroprevalence estimates using only one of the two tests would have been higher, with greater uncertainty. The positive predictive value (PPV) of a positive result on both tests was 99.11% (95% CrI: 95.75%, 99.94%), compared to PPV 44.19%-63.32% (95% CrI range 3.25%-98.64%) if only one test was utilized. Conclusions: Four weeks following shelter-in-place, active and prior SARS-CoV-2 infection in a rural Northern California community was extremely rare. In this low prevalence setting, use of two antibody tests increased the PPV and precision of seroprevalence estimates.

show abstract

Bayesian analysis of tests with unknown specificity and sensitivity

Cited by 60 publications

References 15 publications

High Community SARS-CoV-2 Antibody Seroprevalence in a Ski Resort Community, Blaine County, Idaho, US. Preliminary Results

High Community SARS-CoV-2 Antibody Seroprevalence in a Ski Resort Community, Blaine County, Idaho, US. Preliminary Results

Discriminative Bayesian Serology: Counting Without Cutoffs

Universal PCR and antibody testing demonstrate little to no transmission of SARS-CoV-2 in a rural community

Contact Info

Product

Resources

About