Preparing Data for Artificial Intelligence in Pathology with Clinical-Grade Performance

Abstract

The pathology is decisive for disease diagnosis but relies heavily on experienced pathologists. In recent years, there has been growing interest in the use of artificial intelligence in pathology (AIP) to enhance diagnostic accuracy and efficiency. However, the impressive performance of deep learning-based AIP in laboratory settings often proves challenging to replicate in clinical practice. As the data preparation is important for AIP, the paper has reviewed AIP-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. An in-depth analysis of data preparation methods is conducted, encompassing the acquisition of pathological tissue slides, data cleaning, screening, and subsequent digitization. Expert review, image annotation, dataset division for model training and validation are also discussed. Furthermore, we delve into the reasons behind the challenges in reproducing the high performance of AIP in clinical settings and present effective strategies to enhance AIP’s clinical performance. The robustness of AIP depends on a randomized collection of representative disease slides, incorporating rigorous quality control and screening, correction of digital discrepancies, reasonable annotation, and sufficient data volume. Digital pathology is fundamental in clinical-grade AIP, and the techniques of data standardization and weakly supervised learning methods based on whole slide image (WSI) are effective ways to overcome obstacles of performance reproduction. The key to performance reproducibility lies in having representative data, an adequate amount of labeling, and ensuring consistency across multiple centers. Digital pathology for clinical diagnosis, data standardization and the technique of WSI-based weakly supervised learning will hopefully build clinical-grade AIP.

Biodistribution Assessment of a Novel 68Ga-Labeled Radiopharmaceutical in a Cancer Overexpressing CCK2R Mouse Model: Conventional and Radiomics Methods for Analysis

“…Histopathological assessments play a crucial role in evaluating the percentage of necrosis in tumor tissues, a significant factor influencing the uptake level and contributing to inhomogeneous or poor uptake [ 28 ]. Additionally, histological assessments serve as the gold standard for diagnostic cancer procedure in clinical practice [ 29 , 30 , 31 , 32 ]. Conversely, the pathophysiology of targeted organs can be studied with minimal invasiveness through the radiomics approach [ 33 ].…”

Detailed Image Data Quality and Cleaning Practices for Artificial Intelligence Tools for Breast Cancer

“…Both issues often stem from a lack of uniform practices among different stakeholders: developers, regulatory agencies, medical facilities (hospitals), and data registries/aggregators. 5,6,[8][9][10][11][12][13][14] A critical phase of CAD development or usage or image collection is image data preprocessing that is not standardized presently, which may lead to different biases among different data sets to optimize and validate the algorithms. To clear medical AI products, regulatory agencies require data quality assurance and accurate labels (ie, diagnosis) to optimize CAD, but do not define nor quantify the steps to satisfy the requirements, 15 nor do they benchmark the CADs on gold-standard databases.…”

“…7,20 This image preparation phase is practiced among DL developers to gain a competitive accuracy advantage. 13,[16][17][18][19]21 The unresolved problem is that the techniques are proprietary, sometimes subjective, and seldomly disclosed even in medical journals so that image sets prepared differently can lead to poor or inconsistent clinical accuracy versus the manufacturer's advertised accuracy. 5,8,14 Although different algorithms can achieve comparable results, their fundamental limitation is data quality and cleanliness, which should arise from objective, predictable selection criteria, include timely and accurate labels (regulatory approval requires static products so that DL algorithms are trained by supervised learning using labels), and removal of substandard images, duplicates, inconsistent information, or extraneous objects in the image (eg, text and irrelevant tissue; Data Supplement, Figs S1 and S2).…”

“…5,8,14 Although different algorithms can achieve comparable results, their fundamental limitation is data quality and cleanliness, which should arise from objective, predictable selection criteria, include timely and accurate labels (regulatory approval requires static products so that DL algorithms are trained by supervised learning using labels), and removal of substandard images, duplicates, inconsistent information, or extraneous objects in the image (eg, text and irrelevant tissue; Data Supplement, Figs S1 and S2). 13,18 The efficiency of the selection criteria in preserving the largest number of cases is also critical because there is a general lack of accessible data to optimize medical DL algorithms. 6,7 Clinicians learn to analyze poorly taken images or images containing irrelevant objects.…”