Clinical Validation of a Multi-protein, Serum-based Assay for Disease Activity Assessments in Multiple Sclerosis

Abstract

Background and objectives: An unmet need exists for validated quantitative tools to measure multiple sclerosis (MS) disease activity and progression. We developed a custom immunoassay-based MS disease activity (MSDA) Test incorporating 18 protein concentrations into an algorithm to calculate four Disease Pathway scores (Immunomodulation, Neuroinflammation, Myelin Biology, and Neuroaxonal Integrity) and an overall Disease Activity score. The objective was to clinically validate the MSDA Test based on associations between scores and clinical/radiographic assessments. Methods: Serum samples (N=614) from patients with MS at multiple sites were split into Train (n=426; algorithm development) and Test (n=188; evaluation) subsets. Subsets were stratified by demographics, sample counts per site, and gadolinium-positive (Gd+) lesion counts; age and sex were used to demographically adjust protein concentrations. MSDA Test results were evaluated for potential association with Gd+ lesion presence/absence, new and enlarging (N/E) T2 lesion presence, and active versus stable disease status (composite endpoint combining radiographic and clinical evidence of disease activity). Results: A multi-protein model was developed (trained and cross-validated) using the Train subset. When applied to the Test subset, the model classified the Gd+ lesion presence/absence, N/E T2 lesion presence, and active versus stable disease status assessments with an area under the receiver operating characteristic (AUROC) of 0.781, 0.750, and 0.768, respectively. In each case, the multi-protein model had significantly (bootstrapped, one-sided p<0.05) greater AUROC performance when compared with the top-performing, demographically adjusted (by age and sex) single-protein model based on neurofilament light polypeptide chain. Algorithmic score thresholds corresponded to low, moderate, or high levels of disease activity. Based on the Test subset, the diagnostic odds ratios determined that the odds of having ≥1 Gd+ lesions among samples with a moderate/high Disease Activity score were 4.49 times that of a low Disease Activity score. The odds of having ≥2 Gd+ lesions among samples with a high Disease Activity score were 20.99 times that of a low/moderate Disease Activity score. Discussion: The MSDA Test was clinically validated; the multi-protein model had greater performance compared with the top-performing single-protein model. The MSDA Test may serve as a quantitative and objective tool to enhance care for MS.

Analytical validation of a multi‐protein, serum‐based assay for disease activity assessments in multiple sclerosis

“…A two‐layer, L2‐penalized logistic regression stacked classifier model was developed and clinically validated in a separate study that optimized the model's performance to classify serum samples based on the presence of gadolinium‐enhancing lesions (0 lesions or ≥1 lesions) on an MRI administered within 60 days of blood draw [26]. In the first layer of the model, individual protein concentrations in log 10 which were demographically corrected for age and sex and LOQ‐imputed (referred to as adjusted concentrations) were used as inputs into the four Disease Pathway models (Immunomodulation, Neuroinflammation, Myelin Biology, and Neuroaxonal Integrity).…”

Analytical validation of a multi-protein, serum-based assay for disease activity assessments in multiple sclerosis

“…A two-layer, L2-penalized logistic regression stacked classifier model was developed and clinically validated in a separate study that optimized the model’s performance to classify serum samples based on the presence of gadolinium-enhancing lesions (0 lesions or ≥1 lesions) on an MRI administered within 60 days of blood draw. [26] In the first layer of the model, individual protein concentrations in log 10 which were demographically corrected for age and sex and LOQ-imputed (referred to as adjusted concentrations) were used as inputs into the four Disease Pathway models (Immunomodulation, Neuroinflammation, Myelin Biology, and Neuroaxonal Integrity). The second layer of the model used the adjusted protein concentrations and the output (eg, the probability) of the Disease Pathway models as meta features to calculate an overall Disease Activity score ( File S1, Supporting Information ).…”

Proteomic signatures of physical, cognitive, and imaging outcomes in multiple sclerosis

“…MRI-based brain volumes, EDSS, and neuropsychological test outcomes were used as dependent variables, and age, sex, BMI, and all proteomic measures as independent predictors (outcome score = age + sex + body mass index (BMI) + biomarker concentration) and for the linear mixedeffects model, subject ID was set as random effect (outcome score = age + sex + BMI + time point + biomarker concentration + (1|patient ID). For entry into the regression models, the proteomic data, MRI-based brain volumes, EDSS, and neuropsychological test scores were transformed using log (10) and all the statistical tests were applied to the log-transformed data. Logistic regression models were similarly used if the dependent variable was of categorical nature.…”

“…The multiple sclerosis disease activity (MSDA) is a recently developed and analytically validated 9 panel of 18 biomarkers that represent changes within four main pathophysiological pathways of neuroinflammation, immunomodulation, myelin biology, and neuroaxonal integrity 9 . In the first clinical validation study, the MSDA platform was trained and tested as a predictor for presence of gadolinium‐enhancing lesions or new/newly enlarging T2 lesions in a cohort of 614 samples 10 . The multi‐protein scores outperformed the best individual protein (NfL) with area under curve change from 0.726 to 0.781 10 .…”

“…In the first clinical validation study, the MSDA platform was trained and tested as a predictor for presence of gadolinium‐enhancing lesions or new/newly enlarging T2 lesions in a cohort of 614 samples 10 . The multi‐protein scores outperformed the best individual protein (NfL) with area under curve change from 0.726 to 0.781 10 . Determining the relation of the MSDA panel to long‐term disability outcomes and examining the longitudinal predictive properties of such an assay are essential for clinical adoption and wide‐spread clinical utility.…”