Learned end-to-end high-resolution lensless fiber imaging towards real-time cancer diagnosis

Abstract

Recent advances in label-free histology promise a new era for real-time diagnosis in neurosurgery. Deep learning using autofluorescence is promising for tumor classification without histochemical staining process. The high image resolution and minimally invasive diagnostics with negligible tissue damage is of great importance. The state of the art is raster scanning endoscopes, but the distal lens optics limits the size. Lensless fiber bundle endoscopy offers both small diameters of a few 100 microns and the suitability as single-use probes, which is beneficial in sterilization. The problem is the inherent honeycomb artifacts of coherent fiber bundles (CFB). For the first time, we demonstrate an end-to-end lensless fiber imaging with exploiting the near-field. The framework includes resolution enhancement and classification networks that use single-shot CFB images to provide both high-resolution imaging and tumor diagnosis. The well-trained resolution enhancement network not only recovers high-resolution features beyond the physical limitations of CFB, but also helps improving tumor recognition rate. Especially for glioblastoma, the resolution enhancement network helps increasing the classification accuracy from 90.8 to 95.6%. The novel technique enables histological real-time imaging with lensless fiber endoscopy and is promising for a quick and minimally invasive intraoperative treatment and cancer diagnosis in neurosurgery.

In vivo brain tumor classification using fiber endoscopy

“…We found that a cascade of U-Net in combination with an EDSR achieves the best image enhancement [9]. Classification is performed using a VGG-19 network.…”

Step-by-step transfer function reversal for single-shot 3D fiber endoscopy using a diffuser

“…For the reconstruction through the CFB, an additional U-Net is used. 32,33 A more sophisticated network architecture could yield even better results. The reconstruction quality is measured by calculating the peak signal-to-noise ratio (PSNR) between the ground truth and the reconstructed image.…”

Object Function Retrieval by Model-Based Optimization in Fourier Holographic Endoscopy

“…One approach that has gained attention is the utilization of deep learning technology in holographic endoscopy. − In this method, the TMs of the fibers are trained by using different conformations, allowing the inference of the TM for arbitrary conformations at a later stage. While promising, this approach has shown effectiveness primarily for simple artificial objects. − Another approach involves the placement of a partial reflector at the distal end of a fiber bundle and using the reflection from this reflector as a reference wave to compensate for core-to-core phase retardations …”