Clinical application of machine learning and computer vision to indocyanine green quantification for dynamic intraoperative tissue characterisation: how to do it

Abstract

Introduction Indocyanine green (ICG) quantification and assessment by machine learning (ML) could discriminate tissue types through perfusion characterisation, including delineation of malignancy. Here, we detail the important challenges overcome before effective clinical validation of such capability in a prospective patient series of quantitative fluorescence angiograms regarding primary and secondary colorectal neoplasia. Methods ICG perfusion videos from 50 patients (37 with benign (13) and malignant (24) rectal tumours and 13 with colorectal liver metastases) of between 2- and 15-min duration following intravenously administered ICG were formally studied (clinicaltrials.gov: NCT04220242). Video quality with respect to interpretative ML reliability was studied observing practical, technical and technological aspects of fluorescence signal acquisition. Investigated parameters included ICG dosing and administration, distance–intensity fluorescent signal variation, tissue and camera movement (including real-time camera tracking) as well as sampling issues with user-selected digital tissue biopsy. Attenuating strategies for the identified problems were developed, applied and evaluated. ML methods to classify extracted data, including datasets with interrupted time-series lengths with inference simulated data were also evaluated. Results Definable, remediable challenges arose across both rectal and liver cohorts. Varying ICG dose by tissue type was identified as an important feature of real-time fluorescence quantification. Multi-region sampling within a lesion mitigated representation issues whilst distance–intensity relationships, as well as movement-instability issues, were demonstrated and ameliorated with post-processing techniques including normalisation and smoothing of extracted time–fluorescence curves. ML methods (automated feature extraction and classification) enabled ML algorithms glean excellent pathological categorisation results (AUC-ROC > 0.9, 37 rectal lesions) with imputation proving a robust method of compensation for interrupted time-series data with duration discrepancies. Conclusion Purposeful clinical and data-processing protocols enable powerful pathological characterisation with existing clinical systems. Video analysis as shown can inform iterative and definitive clinical validation studies on how to close the translation gap between research applications and real-world, real-time clinical utility.

Indocyanine Green (ICG) and Colorectal Surgery: A Literature Review on Qualitative and Quantitative Methods of Usage

“…There is little question that each of these advances is theoretically realisable; nevertheless, the extent to which they can be used and how well they can be utilised in clinical contexts are the primary factors that will determine their therapeutic utility. Because these are screen-based techniques and call for an increased level of comprehension to visualise the surgical area, there is a significant demand for the sharing and growth of expertise, such as through the recording of surgical procedures on film, as suggested by Hardy [ 111 ]. Validating innovations in this way allows them to be paired with new computational methodologies such as AI in order to obtain better degrees of interpretation accuracy.…”

Fluorescent Probes for Imaging in Humans: Where Are We Now?

“…Fluorescence image guidance also holds great potential to accelerate the tissue identification and delineation of malignancy. In the past few years, several groups have attempted to incorporate artificial intelligence (AI) in fluorescence-guided surgeries to interpret fluorescence signals more objectively, − and computer algorithms have enabled discrimination between tissue types thanks to the differential perfusion patterns of ICG in cancerous, benign, and normal tissues …”

Geotemporal Fluorophore Biodistribution Mapping of Colorectal Cancer: Micro and Macroscopic Insights

“…New near infrared (NIR) fluorophores are in clinical trials for this purpose and potentially also for targeted cancer immunotherapy [7][8][9]. Even now, the sole approved fluorophore, indocyanine green (ICG), is being assessed for its usefulness in cancer marking in surgery for different subtypes (ICG trapping within malignant tissue has been attributed to enhanced vascular leakage, involving inflammatory mediators, increased tumour vascularity and clathrin-mediated endocytosis) and is showing promise as a means by which to characterize endoscopic cancer using computer vision and artificial intelligence methods to exploit dynamic inflow/outflow comparative perfusion/diffusion differentials between areas of neoplasia and adjacent normal tissues [10][11][12][13]. However, underlying molecular mechanisms of tumour-dye interactions (as opposed to cancer cell-dye pharmacokinetics) in humans, are poorly described.…”