Recommendations for Systematic Statistical Computation of Immunogenicity Cut Points

Abstract

Today, the assessment of immunogenicity is integral in nonclinical and clinical testing of new biotherapeutics and biosimilars. A key component in the risk-based evaluation of immunogenicity involves the detection and characterization of anti-drug antibodies (ADA). Over the past couple of decades, much progress has been made in standardizing the generalized approach for ADA testing with a three-tiered testing paradigm involving screening, confirmation, and quasi-quantitative titer assessment representing the typical harmonized scheme. Depending on a biotherapeutic's structural attributes, more characterization and testing may be appropriate. Unlike bioanalytical assays used to support the evaluation of pharmacokinetics or toxicokinetics, an important component in immunogenicity testing is the calculation of cut points for the identification (screening), confirmation (specificity), and titer assessment responses in animals and humans. Several key publications have laid an excellent foundation for statistical design and data analysis to determine immunogenicity cut points. Yet, the process for statistical determination of cut points remains a topic of active discussion by investigators who conduct immunogenicity assessments to support biotherapeutic drug development. In recent years, we have refined our statistical approach to address the challenges that have arisen due to the evolution in biotherapeutics and the analytical technologies used for quasi-quantitative detection. Based on this collective experience, we offer a simplified statistical analysis process and flow-scheme for cut point evaluations that should work in a large majority of projects to provide reliable estimates for the screening, confirmatory, and titering cut points.

An ELISA for detection of complement-bound circulating immune complexes in mice

“…Data were analyzed using log-transformed S/N data and statistical information about distribution, means and variances were obtained using SAS.JMP version 13.0.0 (SAS, Cary, NC). A parametric cut-point approach allowing a false positive rate of 5% was based on the formula [Mean of S/N] þ [(t 0.05,df ¼ 1.645 for df:1) Â [SD of S/N], followed by logarithmic back-transformation, as described by Devanarayan et al (2017).…”

Development and validation of cell-based luciferase reporter gene assays for measuring neutralizing anti-drug antibodies against interferon beta

“…Important factors to consider are serum matrix effects and the use of a statistically based cut-point approach. These recommendations was recently updated by Devanarayan et al (288). In conclusion, even though the use of validated immunoassays that comply with regulatory authority requirements result in increased assay sensitivity the clinical relevance of these modifications needs to be determined.…”

Are Lessons Learned in Setting Cut Points for Detection of Anti-Drug Antibodies Also Useful in Serology Assays for Robust Detection of SARS-CoV-2 Reactive Antibodies?

“…Due to the relatively greater importance of specificity over sensitivity and PPV over NPV, and due to the minor impact on NPV when targeting a very low FPER, we believe it is appropriate to set the FPER for a diagnostic cut point threshold at as low as 0.1% or even 0.01% for reliable detection of SARS-CoV-2-reactive antibodies. Given the need for setting the cut point at such a low FPER, we propose testing at least 100 presumptive antibody-negative serum samples from COVID-19-negative individuals in duplicate across six or more assay runs using a balanced block design framework during pre-study assay validation (21,24). Depending on the assay design, it would be appropriate to also evaluate potential sources of both fixed and random variation, such as analyst and instrument.…”

“…For specific detection of anti-SARS-CoV-2 serum antibodies, our intended normalization strategy will be outlined in detail in another manuscript that is currently undergoing preparation. After the evaluation and removal of analytical and biological outliers (24), the cut point on normalized signal responses can be determined based on the calculation of mean + 3.09 × SD (standard deviation), which represents the 99.9th percentile of the population under normal distribution, and therefore corresponds to 0.1% FPER (similarly, the mean + 3.719 × SD can be used to target 0.01% FPER). To help ensure approximate data normality, this calculation can be performed after logarithmic or other suitable data transformation.…”

“…Other robust statistical alternatives to mean and SD, such as median and 1.4826 × MAD (median absolute deviation), may also be employed (24), if there is an appreciable number of borderline outliers even after eliminating more extreme outliers. Upon establishment of a screening cut point during method validation, samples from the population that have response values exceeding the screening cut point are classified as having SARS-CoV-2-reactive antibodies.…”