Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging

Abstract

. Significance: Hyperspectral imaging (HSI) provides rich spectral information for improved histopathological cancer detection. However, acquiring high-resolution HSI data for whole-slide imaging (WSI) can be time-consuming and requires a huge amount of storage space. Aim: WSI using a color camera can be achieved with fast speed, high image resolution, and excellent image quality due to the established techniques. We aim to develop an RGB-guided unsupervised hyperspectral super-resolution reconstruction method that is hypothesized to improve image quality while maintaining the spectral characteristics. Approach: High-resolution hyperspectral images of 32 histologic slides were obtained via automated WSI. High-resolution RGB histology images were registered to the hyperspectral images for RGB guidance. An unsupervised super-resolution network was trained to take the downsampled low-resolution hyperspectral patches (LR-HSI) and high-resolution RGB patches (HR-RGB) as inputs to reconstruct high-resolution hyperspectral patches (HR-HSI). Then, an Inception-based network was trained with the HR-RGB, original HR-HSI, and generated HR-HSI, respectively, for whole-slide histopathological cancer detection. Results: Our super-resolution reconstruction network generated high-resolution hyperspectral images with well-maintained spectral characteristics and improved image quality. Image classification using the original hyperspectral data outperformed RGB because of the extra spectral information. The generated hyperspectral image patches further improved the results. Conclusions: The proposed method potentially reduces image acquisition time, saves storage space without compromising image quality, and improves the image classification performance.

An automatic whole-slide hyperspectral imaging microscope

“…In the bottom row, we show the same whole slides captured using our own HWSI system. The images in the bottom row are shown in synthesized-RGB using a method previously reported by us [15] to produce colored images from hyperspectral images. These HWSI are composited by using 80 -120 hyperspectral images, each individual hyperspectral image is of size 84 × 2000 × 2000 pixel.…”

“…Preliminary research shows that training histopathology classifiers using hyperspectral images yield better cancer margin compared to that trained on RGB images [9][10][11][12][13][14]. Ma et al [15,16] showed that using hyperspectral images of head and neck squamous cell carcinoma to train neural networks can result in improvements of patch-wise classification over using regular RGB images. Ortega et al [17,18] used hyperspectral imaging and deep learning techniques to detect breast cancer and brain cancer (glioblastoma) in histological slides.…”

“…Ortega et al [23] developed a HSI database of brain cancer by manually capturing many different tissues. Our previous research [13,15] relied on automatic methods to create in-focused hyperspectral images of the tissues. One of the biggest tasks in automating HSI reliably is the problem of focusing.…”

Conditional generative adversarial network (cGAN) for synthesis of digital histologic images from hyperspectral images

“…We utilized hematoxylin and eosin (H&E)-stained histological slides as well as their corresponding digital histologic images obtained from four different head and neck squamous cell carcinoma (SCC) patients who underwent routine surgery [14]. Each patient has one slide of normal tissue (N) and one slide of tumor tissue (T), which were imaged using an automated hyperspectral microscopic imaging system [11]. Hyperspectral images were acquired with a 40× objective magnification.…”

“…In the past, several studies tried to synthesize hyperspectral images from RGB images, since RGB images were relatively easier to acquire [9,10]. However, as the significance of HSI being recognized in recent years, some automated or semi-automated hyperspectral microscopic imaging systems have been developed [6,[11][12][13], which can implement the acquisition of high-quality digital hyperspectral histologic images for computational pathology purposes. Although two imaging modalities can be integrated into one system, i.e., both the hyperspectral and color cameras can be mounted onto a microscope and acquire both HSI and RGB images simultaneously, it can be more cost effective if only one camera is needed.…”