Prospective Validation of Prediction Model for Kidney Discard

Zhou, Sheng; Massie, Allan B.; Holscher, Courtenay M.; Waldram, Madeleine M.; Ishaque, Tanveen; Thomas, Alvin G.; Segev, Dorry L.

doi:10.1097/tp.0000000000002362

Cited by 19 publications

(25 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the NLP model highlights specific types of PHS risk (ie, intravenous drug use) associated with discard. This finding is aligned with research on kidney offer declines in addition to the similar factors, hypertension, diabetes, and insulin use used in previous models . Having a directional understanding of these nuanced associations, in relation to PHS risk, is important given that the Centers for Disease Control and Prevention (CDC) is reevaluating PHS risk definitions…”

Section: Discussionsupporting

confidence: 54%

“…We sought to understand the concordance between our donor text model and other indices in order to understand if we are measuring the same thing in just a different way also known as “old wine in new bottles.” For the purpose of this study, we chose to compare our model with the reduced PODD model, known as r‐PODD, and KDPI. Our rationale for choosing r‐PODD and KDPI was that these models only leverage variables likely to be available at time of the match run . To compare the indices to each other, we examined model statistics at the donor level.…”

Section: Methodsmentioning

confidence: 99%

“…[9][10][11] Although these models have exhibited strong performance rank ordering, their sensitivity is limited. [9][10][11] The potential to improve these models is limited if further attempts are based solely on the same traditional OPTN data sources. Often the only way to improve model performance is by incorporating new data sources or finding new insights within the data.…”

Section: Motivationmentioning

confidence: 99%

“…Our rationale for choosing r-PODD and KDPI was that these models only leverage variables likely to be available at time of the match run. 10 To compare the indices to each other, we examined model statistics at the donor level. We employed correlation analyses (Spearman) and examined ROC curves to understand the overlap and performance of each index.…”

Section: Correlation With Other Indicesmentioning

confidence: 99%

See 3 more Smart Citations

Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation

Placona

Martinez

McGehee

et al. 2020

American Journal of Transplantation

View full text Add to dashboard Cite

Although expedited placement could ameliorate stagnant kidney utilization, precisely identifying difficult‐to‐place organs is crucial to mitigate potential harms associated with this policy. Existing algorithms have only leveraged structured data from the Organ Procurement and Transplantation Network (OPTN); however, detailed, free text case information about a donor exists. No known research exists about the utility of these data. We developed a model to predict the probability of delay or discard for adult deceased kidney donors between 2010 and 2018, leveraging donor free text data. The resultant model had a c‐statistic of 0.75 compared to 0.80 ( Reduced Probability of Delay or Discard [model], r‐PODD) and 0.77 ( Kidney Donor Profile Index, KDPI) on the test dataset. Analysis of the top predictive words suggest both known and potentially novel clinical factors (ie, a known factor such as hypertension vs a novel factor such as stents), and nuanced social factors (intravenous drug use) could negatively affect kidney utilization. These findings suggest that donor narratives have utility; the natural language processing (NLP) model is only moderately correlated with existing indices and provides directional evidence about additional cardiovascular risk factors that may affect kidney utilization. More research is needed to understand the potential to enhance existing indices of kidney utilization to better enable and mitigate the effects of policy interventions such as expedited placement.

show abstract

Section: Discussionsupporting

confidence: 54%

Section: Methodsmentioning

confidence: 99%

Section: Motivationmentioning

confidence: 99%

Section: Correlation With Other Indicesmentioning

confidence: 99%

See 2 more Smart Citations

Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation

Placona

Martinez

McGehee

et al. 2020

American Journal of Transplantation

View full text Add to dashboard Cite

show abstract

“…It is likely that many centers are willing to accept at least one high risk attribute in a deceased-donor kidney but the summation of multiple concerning attributes in a single kidney up for offer leads to decline and discard. Another recent analysis created a model which scored individual kidneys based on a composite of high-risk features, reliably predicting the risk of organ discard or cold ischemia time .36 hours (6). Only 15% of centers transplanted the majority of kidneys with very high scores and approximately 60% of centers never accepted any such offers.…”

mentioning

confidence: 99%

No Time to Wait

Davis

Cooper

2019

CJASN

View full text Add to dashboard Cite

Prediction of High-Risk Donors for Kidney Discard and Nonrecovery Using Structured Donor Characteristics and Unstructured Donor Narratives

Sageshima,

Than,

Goussous

et al. 2024

JAMA Surg

View full text Add to dashboard Cite

ImportanceDespite the unmet need, many deceased-donor kidneys are discarded or not recovered. Inefficient allocation and prolonged ischemia time are contributing factors, and early detection of high-risk donors may reduce organ loss.ObjectiveTo evaluate the feasibility of machine learning (ML) and natural language processing (NLP) classification of donors with kidneys that are used vs not used for organ transplant.Design, Setting, and ParticipantsThis retrospective cohort study used donor information (structured donor characteristics and unstructured donor narratives) from the United Network for Organ Sharing (UNOS). All donor offers to a single transplant center between January 2015 and December 2020 were used to train and validate ML models to predict donors who had at least 1 kidney transplanted (at our center or another center). The donor data from 2021 were used to test each model.ExposuresDonor information was provided by UNOS to the transplant centers with potential transplant candidates. Each center evaluated the donor and decided within an allotted time whether to accept the kidney for organ transplant.Main Outcomes and MeasuresOutcome metrics of the test cohort included area under the receiver operating characteristic curve (AUROC), F1 score, accuracy, precision, and recall of each ML classifier. Feature importance and Shapley additive explanation (SHAP) summaries were assessed for model explainability.ResultsThe training/validation cohort included 9555 donors (median [IQR] age, 50 [36-58] years; 5571 male [58.3%]), and the test cohort included 2481 donors (median [IQR] age, 52 [40-59] years; 1496 male [60.3%]). Only 20% to 30% of potential donors had at least 1 kidney transplanted. The ML model with a single variable (Kidney Donor Profile Index) showed an AUROC of 0.69, F1 score of 0.42, and accuracy of 0.64. Multivariable ML models based on basic a priori structured donor data showed similar metrics (logistic regression: AUROC = 0.70; F1 score = 0.42; accuracy = 0.62; random forest classifier: AUROC = 0.69; F1 score = 0.42; accuracy = 0.64). The classic NLP model (bag-of-words model) showed its best metrics (AUROC = 0.60; F1 score = 0.35; accuracy = 0.59) by the logistic regression classifier. The advanced Bidirectional Encoder Representations From Transformers model showed comparable metrics (AUROC = 0.62; F1 score = 0.39; accuracy = 0.69) only after appending basic donor information. Feature importance and SHAP detected the variables (and words) that affected the models most.Conclusions and RelevanceResults of this cohort study suggest that models using ML can be applied to predict donors with high-risk kidneys not used for organ transplant, but the models still need further elaboration. The use of unstructured data is likely to expand the possibilities; further exploration of new approaches will be necessary to develop models with better predictive metrics.

show abstract

Prospective Validation of Prediction Model for Kidney Discard

Abstract: Prioritizing allocation of high-PODD kidneys to centers that are more likely to transplant them might help reduce kidney discard.

Cited by 19 publications

References 46 publications

Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation

Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation

No Time to Wait

Prediction of High-Risk Donors for Kidney Discard and Nonrecovery Using Structured Donor Characteristics and Unstructured Donor Narratives

Contact Info

Product

Resources

About