Conditional generative adversarial network (cGAN) for synthesis of digital histologic images from hyperspectral images

Abstract

Hyperspectral imaging (HSI) has been demonstrated in various digital pathology applications. However, the intrinsic high dimensionality of hyperspectral images makes it difficult for pathologists to visualize the information. The aim of this study is to develop a method to transform hyperspectral images of hemoxylin & eosin (H&E)-stained slides to naturalcolor RGB histologic images for easy visualization. Hyperspectral images were obtained at 40× magnification with an automated microscopic imaging system and downsampled by various factors to generate data equivalent to different magnifications. High-resolution digital histologic RGB images were cropped and registered to the corresponding hyperspectral images as the ground truth. A conditional generative adversarial network (cGAN) was trained to output natural color RGB images of the histological tissue samples. The generated synthetic RGBs have similar color and sharpness to real RGBs. Image classification was implemented using the real and synthetic RGBs, respectively, with a pretrained network. The classification of tumor and normal tissue using the HSI-synthesized RGBs yielded a comparable but slightly higher accuracy and AUC than the real RGBs. The proposed method can reduce the acquisition time of two imaging modalities while giving pathologists access to the high information density of HSI and the quality visualization of RGBs. This study demonstrated that HSI may provide a potentially better alternative to current RGB-based pathologic imaging and thus make HSI a viable tool for histopathological diagnosis.

Dual Regression-Enhanced Gaze Target Detection in the Wild

“…The segment anything model (SAM) [2], which has been trained on more than 1 billion masks and has significant skills for creating accurate object masks based on prompts (e.g., bounding boxes, points, sentences), is the original and best-known segmentation foundation model. Several works have revealed that SAM can fail on common medical image segmentation tasks [3], [4], [5], [6]. This is reasonable and predictable given that SAM's training set consists primarily of natural image datasets with substantial edge information.…”

Hyperspectral dark-field microscopy of human breast lumpectomy samples for tumor margin detection in breast-conserving surgery

“…HSI produces images to form a three-dimensional “data cube” consisting of stacked two-dimensional (2D) images of the same scene at multiple contiguous wave bands. More recently, the technologies have increasingly been applied to enable histologic evaluation of tumors in biopsy or post-lumpectomy tissues 7 , 24 – 27 to enable intraoperative assessment of tissues. A variety of applications of the HSI for tumor margin imaging have successfully been demonstrated for image-guided clinical treatments involving diabetic wounds, 28 , 29 oral cancer, 30 head and neck cancer, 25 , 27 , 31 and breast cancer.…”

“…More recently, the technologies have increasingly been applied to enable histologic evaluation of tumors in biopsy or post-lumpectomy tissues 7 , 24 – 27 to enable intraoperative assessment of tissues. A variety of applications of the HSI for tumor margin imaging have successfully been demonstrated for image-guided clinical treatments involving diabetic wounds, 28 , 29 oral cancer, 30 head and neck cancer, 25 , 27 , 31 and breast cancer. 32 – 34 Extensive discussions on the progress in medical applications of various hyperspectral imaging techniques are available in reviews elsewhere.…”