An integrated iterative annotation technique for easing neural network training in medical image analysis

Abstract

Neural networks promise to bring robust, quantitative analysis to medical fields, but adoption is limited by the technicalities of training these networks. To address this translation gap between medical researchers and neural networks in the field of pathology, we have created an intuitive interface which utilizes the commonly used whole slide image (WSI) viewer, Aperio ImageScope (Leica Biosystems Imaging, Inc.), for the annotation and display of neural network predictions on WSIs. Leveraging this, we propose the use of a human-in-the-loop strategy to reduce the burden of WSI annotation. We track network performance improvements as a function of iteration and quantify the use of this pipeline for the segmentation of renal histologic findings on WSIs. More specifically, we present network performance when applied to segmentation of renal micro compartments, and demonstrate multi-class segmentation in human and mouse renal tissue slides. Finally, to show the adaptability of this technique to other medical imaging fields, we demonstrate its ability to iteratively segment human prostate glands from radiology imaging data.

Digital Pathology: Advantages, Limitations and Emerging Perspectives

“…AI applications have been shown to benefit from human input for increased classification performance and reduced numbers of digital slides required for training, a concept termed “Human-in-the loop” [ 31 ]. Lutnick et al [ 32 ] have recently demonstrated the use of a routine WSI viewer (Aperio ImageScope viewer) for a human-in-the-loop interaction with a convolutional neural network (DeepLab v2). A small number of WSIs were initially annotated by a human and subsequently classified by the algorithm.…”

Development and evaluation of deep learning–based segmentation of histologic structures in the kidney cortex with multiple histologic stains

“…These false-positive annotations were removed by the pathologist upon review of the initial classifier output and corrected images were returned to the network for retraining without changing the experimental setup or the network parameters to eliminate false positives and negative errors of the DL algorithm. 45 In line with current sharing guidelines, with this report, we are making all of our data and accompanying ground truth annotations publicly available for the community. Online supplemental material released as part of this work is anticipated to advance the field of computational renal pathology 46 and provide best practices for generating annotations, 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 augmentations, 47 magnifications and recommended stains to perform segmentation tasks optimally.…”

“…Different DL approaches have been used for the segmentation of histologic primitives, such as Gadermayr et al's application of generative adversarial deep networks for stainindependent glomerular segmentation. 45 Bel et al employed cycle-consistent generative adversarial networks (cycleGANs) in DL applications for multicenter stain transformation. 40 Hermsen et al has demonstrated U-Net based segmentation of 7 tissue classes using 40 transplant biopsies on PAS stain.…”

Orbit Image Analysis: An open-source whole slide image analysis tool

“…Only with that approach was it possible to detect glomeruli across tile borders. Lutnick et al [42] published a similar workflow for glomeruli detection based on the combination of the tool ImageScope from Aperio and a DeepLab V2 network. In contrast to our approach, the authors provide an interactive workflow which allows experts to correct annotations during multiple iterations.…”