NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images

Abstract

Object Segmentation is an important step in the work-flow of computational pathology. Deep learning based models as the best forming models require huge amount of labeled data for precise and reliable prediction. However, collecting labeled data is expensive, because it necessarily involves expert knowledge. This is perhaps best illustrated by medical tasks where measurements call for expensive machinery and labels are the fruit of a time-consuming analysis that draws from multiple human experts. As nuclei, cells and glands are fundamental objects for downstream analysis in histology, in this paper we propose a simple CNN-based approach to speed up collecting segmentation annotation for these objects by utilizing minimum input from an annotator. We show for nuclei and cells as small objects, one click inside objects is enough to have precise annotation. For glands as large objects, providing a squiggle to show the extend of gland can guide the model to outline the exact boundaries. This supervisory signals are fed to network as an auxiliary channels along with RGB channels. With detailed experiments, we show that our approach is generalizable, robust against variations in the user input and that it can be used to obtain annotations for completely different domains. Practically, a model trained on the masks generated by NuClick could achieve first rank in LYON19 challenge. Furthermore, as the output of our framework, we release two data-sets: 1) a dataset of lymphocyte annotations within IHC images and 2) a dataset of WBCs annotated in blood sample images.