Learning to image and compute with multimode optical fibers

Abstract

Multimode fibers (MMF) were initially developed to transmit digital information encoded in the time domain. There were few attempts in the late 60s and 70s to transmit analog images through MMF. With the availability of digital spatial modulators, practical image transfer through MMFs has the potential to revolutionize medical endoscopy. Because of the fiber’s ability to transmit multiple spatial modes of light simultaneously, MMFs could, in principle, replace the millimeters-thick bundles of fibers currently used in endoscopes with a single fiber, only a few hundred microns thick. That, in turn, could potentially open up new, less invasive forms of endoscopy to perform high-resolution imaging of tissues out of reach of current conventional endoscopes. Taking endoscopy by its general meaning as looking into, we review in this paper novel ways of imaging and transmitting images using a machine learning approach. Additionally, we review recent work on using MMF to perform machine learning tasks. The advantages and disadvantages of using machine learning instead of conventional methods is also discussed. Methods of imaging in scattering media and particularly MMFs involves measuring the phase and amplitude of the electromagnetic wave, coming out of the MMF and using these measurements to infer the relationship between the input and the output of the MMF. Most notable techniques include analog phase conjugation [A. Yariv, “On transmission and recovery of three-dimensional image information in optical waveguides,” J. Opt. Soc. Am., vol. 66, no. 4, pp. 301–306, 1976; A. Gover, C. Lee, and A. Yariv, “Direct transmission of pictorial information in multimode optical fibers,” J. Opt. Soc. Am., vol. 66, no. 4, pp. 306–311, 1976; G. J. Dunning and R. Lind, “Demonstration of image transmission through fibers by optical phase conjugation,” Opt. Lett., vol. 7, no. 11, pp. 558–560, 1982; A. Friesem, U. Levy, and Y. Silberberg, “Parallel transmission of images through single optical fibers,” Proc. IEEE, vol. 71, no. 2, pp. 208–221, 1983], digital phase conjugation [I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “Focusing and scanning light through a multimode optical fiber using digital phase conjugation,” Opt. Express, vol. 20, no. 10, pp. 10583–10590, 2012; I. N. Papadopoulos, S. Farahi, C. Moser, and D. Psaltis, “High-resolution, lensless endoscope based on digital scanning through a multimode optical fiber,” Biomed. Opt. Express, vol. 4, no. 2, pp. 260–270, 2013] or the full-wave holographic transmission matrix method. The latter technique, which is the current gold standard, measures both the amplitude and phase of the output patterns corresponding to multiple input patterns to construct a matrix of complex numbers relaying the input to the output [Y. Choi, et al., “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett., vol. 109, no. 20, p. 203901, 2012; A. M. Caravaca-Aguirre, E. Niv, D. B. Conkey, and R. Piestun, “Real-time resilient focusing through a bending multimode fiber,” Opt. Express, vol. 21, no. 10, pp. 12881–12887; R. Y. Gu, R. N. Mahalati, and J. M. Kahn, “Design of flexible multi-mode fiber endoscope,” Opt. Express, vol. 23, no. 21, pp. 26905–26918, 2015; D. Loterie, S. Farahi, I. Papadopoulos, A. Goy, D. Psaltis, and C. Moser, “Digital confocal microscopy through a multimode fiber,” Opt. Express, vol. 23, no. 18, pp. 23845–23858, 2015]. This matrix is then used for imaging of the inputs or projection of desired patterns. Other techniques rely on iteratively optimizing the pixel value of the input image to perform a particular task (such as focusing or displaying an image) [R. Di Leonardo and S. Bianchi, “Hologram transmission through multi-mode optical fibers,” Opt. Express, vol. 19, no. 1, pp. 247–254, 2011; T. Čižmár and K. Dholakia, “Shaping the light transmission through a multimode optical fibre: complex transformation analysis and applications in biophotonics,” Opt. Express, vol. 19, no. 20, pp. 18871–18884, 2011; T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun., vol. 3, no. 1, pp. 1–9, 2012; S. Bianchi and R. Di Leonardo, “A multi-mode fiber probe for holographic micromanipulation and microscopy,” Lab Chip, vol. 12, no. 3, pp. 635–639, 2012; E. R. Andresen, G. Bouwmans, S. Monneret, and H. Rigneault, “Toward endoscopes with no distal optics: video-rate scanning microscopy through a fiber bundle,” Opt. Lett., vol. 38, no. 5, pp. 609–611, 2013].

Cylindrical Multimode Waveguides as Focusing Interferometric Systems

“…1 2 (12) and in our experiment, d 1 + L + d 2 = 255.7 mm. Taking the values d 1 and L as given, the "focal length" d 2 after the waveguide, where a maximum is expected, can be calculated given the results of the ray-optical model as…”

“…Mixing of modes in the multimode waveguide may result in a large size of the output beam with a speckle pattern at the output. In the area of imaging with multimode fibers, researchers have learned to make use of the resulting speckle patterns at one end of a fiber by reconstructing images sent through the fiber from its other end. − Such experiments usually involve coupling radiation into and out of a multimode fiber using a microscope objective or lens at both ends and detecting the speckle pattern of the transmitted light through the fiber by an imaging CCD array, reconstruction of the transfer matrix of the fiber by sending many known images through the fiber, which may involve machine learning approaches with neural networks, and finally calculation of the original image based on the characterized properties of the multimode fiber. − However, if the goal is to focus radiation after a multimode waveguide, a speckle pattern represents a poor profile, which will be impractical for further focusing by lenses or parabolic mirrors.…”

Controlling light propagation in multimode fibers for imaging, spectroscopy, and beyond

“…Unlike in imaging applications, the light does not interact with an object to give a contrast. The approach is frequently used as a test bed for advanced computing algorithms, particularly artificial neural networks [197]. Using a convolutional network, Rahmani et al [130] reconstructed the input intensity (or phase) randomised by propagation through a 0.75 m segment of step index fiber having 50 µm in diameter.…”

Soft Fiber Electronics Based on Semiconducting Polymer

“…Future wireless signal transmission will rely heavily on advanced wearable devices, fiber-optic communication devices, and textile antennas. − In general, the design of existing textile antennas may be traced back to the printed circuit antenna proposed by Byron in 1970, which comprises a dielectric substrate with a conductor, and a printed grounded circular conductor sheet . This well-known structure was later termed the “Microstrip Antenna” and served as a model for designing small, integrable, easy-to-install, and lightweight wearable antennas.…”