Evaluation of an open-source machine-learning tool to quantify bone marrow plasma cells

Abstract

AimsThe objective of this study was to develop and validate an open-source digital pathology tool, QuPath, to automatically quantify CD138-positive bone marrow plasma cells (BMPCs).MethodsWe analysed CD138-scanned slides in QuPath. In the initial training phase, manual positive and negative cell counts were performed in representative areas of 10 bone marrow biopsies. Values from the manual counts were used to fine-tune parameters to detect BMPCs, using the positive cell detection and neural network (NN) classifier functions. In the testing phase, whole-slide images in an additional 40 cases were analysed. Output from the NN classifier was compared with two pathologist’s estimates of BMPC percentage.ResultsThe training set included manual counts ranging from 2403 to 17 287 cells per slide, with a median BMPC percentage of 13% (range: 3.1%–80.7%). In the testing phase, the quantification of plasma cells by image analysis correlated well with manual counting, particularly when restricted to BMPC percentages of <30% (Pearson’s r=0.96, p<0.001). Concordance between the NN classifier and the pathologist whole-slide estimates was similarly good, with an intraclass correlation of 0.83 and a weighted kappa for the NN classifier of 0.80 with the first rater and 0.90 with the second rater. This was similar to the weighted kappa between the two human raters (0.81).ConclusionsThis represents a validated digital pathology tool to assist in automatically and reliably counting BMPC percentage on CD138-stained slides with an acceptable error rate.

Deep Learning Accurately Quantifies Plasma Cell Percentages on CD138-Stained Bone Marrow Samples

“…Our approach is therefore distinct from previous studies, which generally used less accurate area- or segmentation-based methods to identify/quantify plasma cells, and relied on low magnification visual estimates from pathologists as gold-standard references. 3 , 4 , 5 , 6 , 7 , 8 One exception is a recent publication from Baranova et al 10 that also started with small image patches, before scaling up to WSIs. The major technical difference between our study and Baranova et al 10 is that they relied on about 40 features from QuPath's cell segmentation algorithm to train a downstream neural network, whereas ours is a single-stage CNN trained on raw image pixels and pathologist labels.…”

“… 3 , 4 , 5 , 6 , 7 , 8 One exception is a recent publication from Baranova et al 10 that also started with small image patches, before scaling up to WSIs. The major technical difference between our study and Baranova et al 10 is that they relied on about 40 features from QuPath's cell segmentation algorithm to train a downstream neural network, whereas ours is a single-stage CNN trained on raw image pixels and pathologist labels. We speculate that our CNN's independence from prior segmentation explains why our reported measures of concordance appear to be higher than that reported by Baranova et al, although a fair comparison would involve a head-to-head evaluation using validation images from both studies.…”

“…As a result of the focus on whole slide estimates, most studies also did not report on concordance between algorithms and humans at a cell-by-cell level. Finally, most studies relied on older area-based and cell- or nuclear-segmentation-based methods, and until very recently, 10 did not employ deep learning.…”

Deep learning accurately quantifies plasma cell percentages on CD138-stained bone marrow samples

“…Our approach is therefore distinct from previous studies, which generally used less accurate area- or segmentation-based methods to identify/quantify plasma cells, and relied on low magnification visual estimates from pathologists as gold-standard references [3–8]. One exception is a recent publication from Baranova et al [10] that also started with small image patches, before scaling up to WSIs. The major technical difference between our study and Baranova et al [10] is that they relied on about 40 features from QuPath’s cell segmentation algorithm to train a downstream neural network, while ours is a single-stage CNN trained on raw image pixels and pathologist labels.…”

“…One exception is a recent publication from Baranova et al [10] that also started with small image patches, before scaling up to WSIs. The major technical difference between our study and Baranova et al [10] is that they relied on about 40 features from QuPath’s cell segmentation algorithm to train a downstream neural network, while ours is a single-stage CNN trained on raw image pixels and pathologist labels. We speculate that our CNN’s independence from prior segmentation explains why our reported measures of concordance appear to be higher than that reported by Baranova et al, although a fair comparison would involve a head-to-head evaluation using validation images from both studies.…”