Spatially-resolved molecular profiling by immunostaining tissue sections is a key feature in cancer diagnosis, subtyping, and treatment, where it complements routine histopathological evaluation by clarifying tumor phenotypes. In this work, we present a deep learning-based method called speedy histological-to-immunofluorescent translation (SHIFT) which takes histologic images of hematoxylin and eosin (H&E)-stained tissue as input, then in near-real time returns inferred virtual immunofluorescence (IF) images that estimate the underlying distribution of the tumor cell marker pan-cytokeratin (panCK). To build a dataset suitable for learning this task, we developed a serial staining protocol which allows IF and H&E images from the same tissue to be spatially registered. We show that deep learning-extracted morphological feature representations of histological images can guide representative sample selection, which improved SHIFT generalizability in a small but heterogenous set of human pancreatic cancer samples. With validation in larger cohorts, SHIFT could serve as an efficient preliminary, auxiliary, or substitute for panCK IF by delivering virtual panCK IF images for a fraction of the cost and in a fraction of the time required by traditional IF.
KEY POINTS• Spatially-resolved molecular profiling is an essential complement to histopathological evaluation of cancer tissues.• Information obtained by immunofluorescence imaging is encoded by features in histological images.• SHIFT leverages previously unappreciated features in histological images to facilitate virtual immunofluorescence staining.• Feature representations of images guide sample selection, improving model generalizability.
KEY WORDSDigital pathology, deep learning, generative adversarial networks, hematoxylin and eosin, immunofluorescence ABSTRACT Spatially-resolved molecular profiling by immunostaining tissue sections is a key feature in cancer diagnosis, subtyping, and treatment, where it complements routine histopathological evaluation by clarifying tumor phenotypes. In this work, we present a deep learning-based method called speedy histological-toimmunofluorescent translation (SHIFT) which takes histologic images of hematoxylin and eosin-stained tissue as input, then in near-real time returns inferred virtual immunofluorescence (IF) images that accurately depict the underlying distribution of phenotypes without requiring immunostaining of the tissue being tested. We show that deep learning-extracted feature representations of histological images can guide representative sample selection, which improves SHIFT generalizability. SHIFT could serve as an efficient preliminary, auxiliary, or substitute for IF by delivering multiplexed virtual IF images for a fraction of the cost and in a fraction of the time required by nascent multiplexed imaging technologies.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.