Operational Ontology for Oncology (O3): A Professional Society-Based, Multistakeholder, Consensus-Driven Informatics Standard Supporting Clinical and Research Use of Real-World Data From Patients Treated for Cancer

Mayo, Charles S.; Feng, Mary U.; Brock, Kristy K.; Kudner, Randi; Balter, Peter; Buchsbaum, Jeffrey C.; Caissie, Amanda; Covington, Elizabeth; Daugherty, Emily C.; Dekker, Andre L.; Fuller, Clifton D.; Hallstrom, Anneka L.; Hong, David S.; Hong, Julian C.; Kamran, Sophia C.; Katsoulakis, Eva; Kildea, John; Krauze, Andra V.; Kruse, Jon J.; McNutt, Tod; Mierzwa, Michelle; Moreno, Amy; Palta, Jatinder R.; Popple, Richard; Purdie, Thomas G.; Richardson, Susan; Sharp, Gregory C.; Satomi, Shiraishi; Tarbox, Lawrence R.; Venkatesan, Aradhana M.; Witztum, Alon; Woods, Kelly E.; Yao, Yuan; Farahani, Keyvan; Aneja, Sanjay; Gabriel, Peter E.; Hadjiiski, Lubomire; Ruan, Dan; Siewerdsen, Jeffrey H.; Bratt, Steven; Casagni, Michelle; Chen, Su; Christodouleas, John C.; DiDonato, Anthony; Hayman, James; Kapoor, Rishhab; Kravitz, Saul; Sebastian, Sharon; Von Siebenthal, Martin; Bosch, Walter; Hurkmans, Coen; Yom, Sue S.; Xiao, Ying

doi:10.1016/j.ijrobp.2023.05.033

Cited by 12 publications

(4 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…20 29 30 32 Additional efforts to develop AI-ready data infrastructures rely on ontology approaches, such as the Operational Ontology for Oncology, to standardise real-world data for use in training or testing of novel algorithms. 39 Two examples of large scale, high-quality prospective evaluation of AI tools in real-world setting were published by Dembrower et al 40 and Lång et al 41 to validate AI-supported mammogram screening for breast cancer in Sweden. 40 41 Gap between AI efficacy and clinical outcome First, the measured metrics of the AI tool used in research might not directly translate into clinical benefits.…”

Section: Discussionmentioning

confidence: 99%

Prospective evaluation of artificial intelligence (AI) applications for use in cancer pathways following diagnosis: a systematic review

Macheka,

Ng,

Ginsburg

et al. 2024

bmjonc

View full text Add to dashboard Cite

The role of artificial intelligence (AI) in cancer care has evolved in the face of ageing population, workforce shortages and technological advancement. Despite recent uptake in AI research and adoption, the extent to which it improves quality, efficiency and equity of care beyond cancer diagnostics is uncertain to date. Henceforth, the objective of our systematic review is to assess the clinical readiness and deployability of AI through evaluation of prospective studies of AI in cancer care following diagnosis.We undertook a systematic review to determine the types of AI involved and their respective outcomes. A PubMed and Web of Science search between 1 January 2013 and 1 May 2023 identified 15 articles detailing prospective evaluation of AI in postdiagnostic cancer pathway. We appraised all studies using Risk of Bias Assessment of Randomised Controlled Trials and Risk of Bias In Non-randomised Studies-of Interventions quality assessment tools, as well as implementational analysis concerning time, cost and resource, to ascertain the quality of clinical evidence and real-world feasibility of AI.The results revealed that the majority of AI oncological research remained experimental without prospective clinical validation or deployment. Most studies failed to establish clinical validity and to translate measured AI efficacy into beneficial clinical outcomes. AI research are limited by lack of research standardisation and health system interoperability. Furthermore, implementational analysis and equity considerations of AI were largely missing.To overcome the triad of low-level clinical evidence, efficacy-outcome gap and incompatible research ecosystem for AI, future work should focus on multicollaborative AI implementation research designed and conducted in accordance with up-to-date research standards and local health systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Prospective evaluation of artificial intelligence (AI) applications for use in cancer pathways following diagnosis: a systematic review

Macheka,

Ng,

Ginsburg

et al. 2024

bmjonc

View full text Add to dashboard Cite

show abstract

“…D0%[Gy] corresponds to Max[Gy] and D100%[Gy] corresponds to Min[Gy]. 5 Bilateral structures (parotid and submandibular glands) were categorized as _High vs. _Low according to their relative median dose. Following TG-263 nomenclature, for plans with multiple GTV and PTV volumes treated to different dose levels, the volumes receiving the highest and lowest dose were categorized as xxx_High and _Low and if three volumes, then xxx_Mid were also used.…”

Section: Methodsmentioning

confidence: 99%

Data Farming to Table: Combined Use of a Learning Health System Infrastructure, Statistical Profiling, and Artificial Intelligence for Automating Toxicity and 3-year Survival for Quantified Predictive Feature Discovery from Real-World Data for Patients Having Head and Neck Cancers

Mayo,

Su,

Rosen

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Introduction: Clinicians iteratively adjust treatment approaches to improve outcomes but to date, automatable approaches for continuous learning of risk factors as these adjustments are made are lacking. We combined a large-scale comprehensive real-world Learning Health System infrastructure (LHSI), with automated statistical profiling, visualization, and artificial intelligence (AI) approach to test evidence-based discovery of clinical factors for three use cases: dysphagia, xerostomia, and 3-year survival for head and neck cancer patients. Our hypothesis was that the combination would enable automated discovery of prognostic features generating testable insights. Methods: Records for 964 patients treated at a single instiution for head and neck cancers with conventional fractionation between 2017 and 2022 were used. Combined information on demographics, diagnosis and staging, social determinants of health measures, chemotherapy, radiation therapy dose volume histogram curves, and treatment details, laboratory values, and outcomes from the LHSI to winnow evidence for 485 candidate prognostic features. Univariate statistical profiling using benchmark resampling to detail confidence intervals for thresholds and metrics: area under the curve (AUC), sensitivity (SN), specificity (SP), F1, diagnostic odds ratio (DOR), p values for Wilcoxon Rank Sum (WRS), Kolmogorov-Smirnov (KS), and logistic fits of distributions detailed predictive evidence of individual features. Statistical profiling was used to benchmark, parsimonious XGBoost models were constructed with 10-fold cross validation using training (70%), validation (10%), and test (20%) sets. Probabilistic models utilizing statistical profiling logistic fits of distributions were used to benchmark XGBoost models. Results: Automated standardized analysis identified novel features and clinical thresholds. Validity of automated findings were affirmed with supporting literature benchmarks. Average incidence of dysphagia ≥grade 3 within 1 year of treatment was low (11%). Xerostomia ≥ grade 2 (39% to 16%) and survival ≤ 3 years decreased (25% to 15%) over the time range. Standard planning constraints used limited contribution of those features: : Musc_Constrict_S: Mean[Gy] < 50, Glnd_Submand_High: Mean[Gy] ≤ 30, Glnd_Submand_Low: Mean[Gy] ≤ 10, Parotid_High: Mean[Gy] ≤ 24, Parotid_Low: Mean[Gy] ≤ 10 Additional prognostic features identified for dysphagia included Glnd_Submand_High:D1%[Gy] ≥ 71.1, Glnd_Submand_Low:D4%[Gy] ≥ 55.1 , Musc_Constric_S:D10%[Gy] ≥ 56.5 , GTV_Low:Mean[Gy] ≥ 71.3. Strongest grade 2 xerostomia feature was Glnd_Submand_Low: D15%[Gy] ≥ 45.2 with a logistic model quantifying a gradual rather than an abrupt increase in probability 13.5 + 0.18 (x-41.0 Gy). Strongest prognostic factors for lower likelihood of death by 3 years were GTV_High: Volume[cc] ≤ 21.1, GTV_Low: Volume[cc] ≤ 57.5, Baseline Neutrophil-Lymphocyte Ratio (NLR) ≤ 5.6, Monocyte-Lymphocyte Ratio (MLR) ≤0.56, Platelet-Lymphocyte ratio (PLR) ≤ 202.5. All predictors had WRS and KS p values < 0.02. Statistical profiling enabled detailing gains of XGBoost models with respect to individual features. Time period reductions in distribution of GTV volumes correlated with reductions in death by 3 years. Discussion: Confirming our hypothesis, automated, standardized statistical profiling of a set of statistical metrics and visualizations supported detailing predictive strength and confidence intervals of individual features, benchmarking of subsequent AI models, and clinical assessment. Association of high dose values to submandibular gland volumes, highlighted relevance as surrogate measures for proximal un-contoured muscles including digastric muscles. Higher values of PLR, NLR, and MLR were associated with lower survival rates. Combined use of Learning Health System Infrastructure, Statistical Profiling and Artificial Intelligence provided a basis for faster, more efficient evidence-based continuous learning of risk factors and development of clinical trial testable hypothesis. Benchmarking AI models with simple probabilistic models provided a means of understanding when results are driven by general areas of overall risk vs. more complex interactions.

show abstract

“…Recent advancements in language models have led to the most current generation of large language models (LLMs), which may be able to make 39 including the adoption of consensus data standards, [40][41][42][43] will also be imperative to overcoming these obstacles. Some data models are beginning to include AE elements, 41 but more work is needed to expand them to comprehensively capture AE information about severity, causality, and timing.…”

Section: Future Directions For Ai-enabled Pharmacovigilancementioning

confidence: 99%

Section: Future Directions For Ai-enabled Pharmacovigilancementioning

confidence: 99%

Navigating the Complexities of Artificial Intelligence–Enabled Real-World Data Collection for Oncology Pharmacovigilance

Gallifant,

Celi,

Sharon

et al. 2024

JCO Clin Cancer Inform

View full text Add to dashboard Cite

show abstract

Operational Ontology for Oncology (O3): A Professional Society-Based, Multistakeholder, Consensus-Driven Informatics Standard Supporting Clinical and Research Use of Real-World Data From Patients Treated for Cancer

Cited by 12 publications

References 31 publications

Prospective evaluation of artificial intelligence (AI) applications for use in cancer pathways following diagnosis: a systematic review

Prospective evaluation of artificial intelligence (AI) applications for use in cancer pathways following diagnosis: a systematic review

Data Farming to Table: Combined Use of a Learning Health System Infrastructure, Statistical Profiling, and Artificial Intelligence for Automating Toxicity and 3-year Survival for Quantified Predictive Feature Discovery from Real-World Data for Patients Having Head and Neck Cancers

Navigating the Complexities of Artificial Intelligence–Enabled Real-World Data Collection for Oncology Pharmacovigilance

Contact Info

Product

Resources

About