Joint reconstruction and classification of tumor cells and cell interactions in melanoma tissue sections with synthesized training data

Abstract

PurposeCancers are almost always diagnosed by morphologic features in tissue sections. In this context, machine learning tools provide new opportunities to describe tumor immune cell interactions within the tumor microenvironment and thus provide phenotypic information that might be predictive for the response to immunotherapy.MethodsWe develop a machine learning approach using variational networks for joint image denoising and classification of tissue sections for melanoma, which is an established model tumor for immuno-oncology research. The manual annotation of real training data would require substantial user interaction of experienced pathologists for each single training image, and the training of larger networks would rely on a very large number of such data sets with ground truth annotation. To overcome this bottleneck, we synthesize training data together with a proper tissue structure classification. To this end, a stochastic data generation process is used to mimic cell morphology, cell distribution and tissue architecture in the tumor microenvironment. Particular components of this tool are random placement and rotation of a large number of patches for presegmented cell nuclei, a stochastic fast marching approach to mimic the geometry of cells and texture generation based on a color covariance analysis of real data. Here, the generated training data reflect a large range of interaction patterns.ResultsIn several applications to histological tissue sections, we analyze the efficiency and accuracy of the proposed approach. As a result, depending on the scenario considered, almost all cells and nuclei which ought to be detected are actually marked as classified and hardly any misclassifications occur.ConclusionsThe proposed method allows for a computer-aided screening of histological tissue sections utilizing variational networks with a particular emphasis on tumor immune cell interactions and on the robust cell nuclei classification.

Artificial intelligence and digital pathology: Opportunities and implications for immuno-oncology

“…DL-based image analysis has been used extensively to study the spatial organisation of the immune infiltrate across cancer types, revealing rich and diverse patterns from routine clinical H&E [ 43 ]. Effland et al [ 41 ] demonstrate the use of an ML algorithm which can detect immune cells in the immediate neighbourhood of tumour cells. The model could also be used to identify immune cells proximate to other immune cells, and thereby define immune-rich zones.…”

Applying artificial intelligence for cancer immunotherapy

“…In addition, Effland et al. 28 developed a DL approach utilizing variational networks to explore complex phenotype interactions in melanoma histopathology that could be predictive of response to immunotherapy. Furthermore, co-development of purpose-built AI in parallel with computational pathology might benefit harmonizing immunotherapy companion diagnostics by facilitating easy sharing and standardizing of image analysis algorithms.…”

“…Effland et al. 28 used variational networks for joint image reconstruction and segmentation based on a DL approach to illustrate the direct interactions between immune cells and melanoma cells. AI also has the potential to predict the response to ICB through focusing on the antigen presenting pathway 108 .…”