Assessment of individual tumor buds using keratin immunohistochemistry: moderate interobserver agreement suggests a role for machine learning

Abstract

Tumor budding is a promising and cost-effective biomarker with strong prognostic value in colorectal cancer. However, challenges related to interobserver variability persist. Such variability may be reduced by immunohistochemistry and computer-aided tumor bud selection. Development of computer algorithms for this purpose requires unequivocal examples of individual tumor buds. As such, we undertook a large-scale, international, and digital observer study on individual tumor bud assessment. From a pool of 46 colorectal cancer cases with tumor budding, 3000 tumor bud candidates were selected, largely based on digital image analysis algorithms. For each candidate bud, an image patch (size 256 × 256 µm) was extracted from a pan cytokeratin-stained whole-slide image. Members of an International Tumor Budding Consortium (n = 7) were asked to categorize each candidate as either (1) tumor bud, (2) poorly differentiated cluster, or (3) neither, based on current definitions. Agreement was assessed with Cohen's and Fleiss Kappa statistics. Fleiss Kappa showed moderate overall agreement between observers (0.42 and 0.51), while Cohen's Kappas ranged from 0.25 to 0.63. Complete agreement by all seven observers was present for only 34% of the 3000 tumor bud candidates, while 59% of the candidates were agreed on by at least five of the seven observers. Despite reports of moderate-to-substantial agreement with respect to tumor budding grade, agreement with respect to individual pan cytokeratin-stained tumor buds is moderate at most. A machine learning approach may prove especially useful for a more robust assessment of individual tumor buds.

Interobserver Variability in the Assessment of Tumor Budding in pT 3/4 Colon Cancer: Improvement by Supporting Immunohistochemistry?

“…Other studies comparing cytokeratin(only)-based assessments to H & E-based assessments demonstrated non superiority according to the prognostic value as well [12,13]. Small epithelial structures are straightforward to recognize on cytokeratin, but an elaborate study of experts has shown that the assessment of individual tumor buds using cytokeratin immunohistochemistry has just a moderate level of interobserver agreement, which indicates the issue with cytokeratin staining [14]. Using a supporting cytokeratin staining might be helpful in special situations, but a limitation of this study is that there was no standardized recording of qualitative factors (for example, inflammation).…”

Taking tumour budding to the next frontier — a post International Tumour Budding Consensus Conference (ITBCC) 2016 review

“…Scoring tumour budding according to the ITBCC method requires a bud count in a certain area, and this process, by its nature, requires time and training, and is subject to interobserver and intraobserver variation 11,12 . Interobserver variation in tumour budding assessment can be suboptimal, and depends on multiple factors, such as training and experience 13,14 .…”

“…Most semi‐automatic tumour scoring methods have been developed with immunohistochemically stained slides, 11,12,15–21 in some cases with immunofluorescent staining 20,22,23 . Only one detection system has been proposed for H&E‐stained slides 24 …”

“…As tumour budding is best described in CRC, is it not surprising that most of the methods have also been developed with CRC tissue 11,12,15,17,18,20,21,23,24 . However, automation attempts have also been evaluated for early‐stage oral squamous cell carcinoma, 16 muscle‐invasive bladder cancer, 22 and breast tumours 19 …”

“…Most approaches use a combination of classic image analysis operations, such as binarisation and colour deconvolution, with a machine learning algorithm 12,20–24 . However, automated tumour bud detection can be successful without the use of machine learning 11,15,18,19 . For approaches that use commercially available software, the type of machine learning classifier that is used is not specified 20,23 .…”