Setting Bias Specifications Based on Qualitative Assays With a Quantitative Cutoff Using COVID-19 as a Disease Model

Abstract

Objectives Automated qualitative serology assays often measure quantitative signals that are compared against a manufacturer-defined cutoff for qualitative (positive/negative) interpretation. The current general practice of assessing serology assay performance by overall concordance in a qualitative manner may not detect the presence of analytical shift/drift that could affect disease classifications. Methods We describe an approach to defining bias specifications for qualitative serology assays that considers minimum positive predictive values (PPVs) and negative predictive values (NPVs). Desirable minimum PPVs and NPVs for a given disease prevalence are projected as equi-PPV and equi-NPV lines into the receiver operator characteristic curve space of coronavirus disease 2019 serology assays, and the boundaries define the allowable area of performance (AAP). Results More stringent predictive values produce smaller AAPs. When higher NPVs are required, there is lower tolerance for negative biases. Conversely, when higher PPVs are required, there is less tolerance for positive biases. As prevalence increases, so too does the allowable positive bias, although the allowable negative bias decreases. The bias specification may be asymmetric for positive and negative direction and should be method specific. Conclusions The described approach allows setting bias specifications in a way that considers clinical requirements for qualitative assays that measure signal intensity (eg, serology and polymerase chain reaction).