Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists

Abstract

The Gleason score is the most important prognostic marker for prostate cancer patients, but it suffers from significant observer variability. Artificial intelligence (AI) systems based on deep learning can achieve pathologist-level performance at Gleason grading. However, the performance of such systems can degrade in the presence of artifacts, foreign tissue, or other anomalies. Pathologists integrating their expertise with feedback from an AI system could result in a synergy that outperforms both the individual pathologist and the system. Despite the hype around AI assistance, existing literature on this topic within the pathology domain is limited. We investigated the value of AI assistance for grading prostate biopsies. A panel of 14 observers graded 160 biopsies with and without AI assistance. Using AI, the agreement of the panel with an expert reference standard increased significantly (quadratically weighted Cohen's kappa, 0.799 vs. 0.872; p = 0.019). On an external validation set of 87 cases, the panel showed a significant increase in agreement with a panel of international experts in prostate pathology (quadratically weighted Cohen's kappa, 0.733 vs. 0.786; p = 0.003). In both experiments, on a grouplevel, AI-assisted pathologists outperformed the unassisted pathologists and the standalone AI system. Our results show the potential of AI systems for Gleason grading, but more importantly, show the benefits of pathologist-AI synergy. Members of the ISUP Pathology Imagebase Expert Panel are listed below Acknowledgements.

Artificial Intelligence and Machine Learning in Prostate Cancer Patient Management—Current Trends and Future Perspectives

“…There are also limitations to the system because pathologists can assess the volume more qualitatively and the system counts the exact area of the individual glands and the findings of Egevad et al [ 40 ] should be used to improve calibration of AI systems. Therefore, Bulten et al [ 41 ] published yet other results showing that the limitations can be addressed if AI is assisting pathologists rather than having a competition in assessing the performance of either pathologists or AI systems. The list of studies discussing different types of algorithms used to train the models, which were looking into Gleason grading of biopsies, TMAs, whole section, the amount of data used and their results are listed in Table 2 .…”

“…The time to train an AI system is high and it is not possible without human intervention. Some of the tasks of AI can surprisingly match, in some perspectives, the performance of experts but still limitations and challenges remain [ 41 ]. Regulation is mandatory for every test that will be introduced in clinical practice.…”

Rapid Assisted Visual Search

“…The assistive system consisted of a zoomable user interface visualising regions of interest using bounding boxes, color-coded for prediction certainty. In a system for Gleason grading, AI assistance increased concordance for most observers [4]. In a study of a content-based retrieval system for pathologists, the system demonstrated potential to resolve some difficult decisions by allowing users to refine results interactively [6].…”

Computerized diagnosis of prostate cancer based on whole slide histology images and deep learning methods

“…К настоящему времени уже был выполнен ряд работ по проблеме компьютеризированной диагностики и оценки степени злокачественности рака простаты [5][6][7][8]. Так, в работах [5,6] с помощью методов глубокого обучения были получены хорошие результаты при решении задач бинарной классификации ткани простаты (рак против нормы, ранняя стадия рака против поздней).…”

“…Так, в работах [5,6] с помощью методов глубокого обучения были получены хорошие результаты при решении задач бинарной классификации ткани простаты (рак против нормы, ранняя стадия рака против поздней). В то же время уровень определения конкретного показателя Глисона был значительно хуже при применении компьютеризированных систем не только в качестве самостоятельного предсказателя ( [5,8], каппа 0,62 и 0,70 соответственно), но и в качестве вспомогательного модуля в процессе поддержки принятия решений гистопатологами ( [7], каппа 0,733). При этом во всех рассмотренных работах количество полнослайдовых изображений (ПСИ), использованных для обучения нейронных сетей, было существенно меньше, чем в настоящем исследовании.…”