Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS)

Abstract

Histopathological visualizations are a pillar of modern medicine and biological research. Surgical oncology relies exclusively on post-operative histology to determine definitive surgical success and guide adjuvant treatments. The current histology workflow is based on bright-field microscopic assessment of histochemical stained tissues and has some major limitations. For example, the preparation of stained specimens for brightfield assessment requires lengthy sample processing, delaying interventions for days or even weeks. Therefore, there is a pressing need for improved histopathology methods. In this paper, we present a deep-learning-based approach for virtual label-free histochemical staining of total-absorption photoacoustic remote sensing (TA-PARS) images of unstained tissue. TA-PARS provides an array of directly measured label-free contrasts such as scattering and total absorption (radiative and non-radiative), ideal for developing H&E colorizations without the need to infer arbitrary tissue structures. We use a Pix2Pix generative adversarial network to develop visualizations analogous to H&E staining from label-free TA-PARS images. Thin sections of human skin tissue were first virtually stained with the TA-PARS, then were chemically stained with H&E producing a one-to-one comparison between the virtual and chemical staining. The one-to-one matched virtually- and chemically- stained images exhibit high concordance validating the digital colorization of the TA-PARS images against the gold standard H&E. TA-PARS images were reviewed by four dermatologic pathologists who confirmed they are of diagnostic quality, and that resolution, contrast, and color permitted interpretation as if they were H&E. The presented approach paves the way for the development of TA-PARS slide-free histological imaging, which promises to dramatically reduce the time from specimen resection to histological imaging.

Deep learning-enabled virtual histological staining of biological samples

“…4f) 36,37 ; multichannel total absorption photoacoustic remote sensing (TA-PARS) was utilized to create virtual H&E staining of human skin tissue (Fig. 4g) 38 ; images acquired with UV microscopy were computationally transformed into Giemsa staining of whole blood Fig. 2 Training and inference of label-free virtual staining networks.…”

“…f Virtual H&E staining using bright-field images 36 . g Virtual H&E staining using TA-PARS images 38 . h Virtual acetic acid and H&E staining using in vivo RCM images 38 .…”

“…g Virtual H&E staining using TA-PARS images 38 . h Virtual acetic acid and H&E staining using in vivo RCM images 38 . All the scale bars represent 100 μm…”

“…For instance, to avoid intensity variations caused by potential photobleaching in autofluorescence imaging, Rivenson et al normalized the input autofluorescence images by subtracting the mean value across the entire tissue slide and dividing it by the standard deviation of the pixel values 18 . Alternatively, to mitigate these variations and enhance the image contrast, some works saturated the top 1% and the bottom 1% of the pixels 38,39 . Pradhan et al also reported that normalizing label-free nonlinear multi-modal (NLM) images in a pixel value range from -1 to 1 could avoid large number multiplications during the training process, helping with better network convergence 35 .…”

“…Various network structures have been reported for virtual staining, among which the generative adversarial network 77 (GAN) is one of the most commonly and widely used frameworks due to its strong representation capability 18,[20][21][22][23]31,32,[36][37][38]48,51,56,59,78,79 . Compared to non-GAN-based inference models, GANs can generate relatively higher resolution and perceptually more realistic images 13,14,59 .…”