Deep learning the high variability and randomness inside multimode fibers

Abstract

Multimode fibres (MMF) are remarkable high-capacity information channels owing to the large number of transmitting fibre modes 1-3 , and have recently attracted significant renewed interest in applications such as optical communication, imaging, and optical trapping [4][5][6][7][8][9][10][11][12][13][14][15] . At the same time, the optical transmitting modes inside MMFs are highly sensitive to external perturbations and environmental changes, resulting in MMF transmission channels being highly variable and random [16][17][18] . This largely limits the practical application of MMFs and hinders the full exploitation of their information capacity. Despite great research efforts made to overcome the high variability and randomness inside MMFs, any geometric change to the MMF leads to completely different transmission matrices, which unavoidably fails at the information recovery. Here, we show the successful binary image transmission using deep learning through a single MMF, which is stationary or subject to dynamic shape variations. We found that a single convolutional neural network has excellent generalisation capability with various MMF transmission states. This deep neural network can be trained by multiple MMF transmission states to accurately predict unknown information at the other end of the MMF at any of these states, without knowing which state is present. Our results demonstrate that deep learning is a promising solution to address the variability and randomness challenge of MMF based information channels. This deep-learning approach is the starting point of developing future high-capacity MMF optical systems and devices, and is applicable to optical systems concerning other diffusing media.We use a simple digital micromirror device (DMD) based single MMF transmission system 19 to demonstrate the idea, as shown in Fig. 1(a). The DMD can be controlled to display any binary pattern by turning 'ON' and 'OFF' individual micrormirrors, simulating a 2D object imaging from the DMD through the MMF to the camera. The experimental setup details are given in Methods. As the first step to verify our idea, we use transmission matrix (TM) calculated output speckle images for deep neural network training and prediction 20 . Firstly we calculate the MMF TM using 8000 input-output pairs collected from the experimental system. The measured TM is verified experimentally to have an average accuracy of 98% for output speckle pattern calculation (details in Methods). We then use the processed binary images of handwritten digits downloaded from the MNIST handwritten digits database 21,22 as the DMD input pattern. The binary processing is to convert the handwritten digit into 36×36 pixel image format that directly matches the fibre mode count and eigenchannels for optimal delivery 19 (details in Methods). As shown in Fig. 1(b), the MMF output speckle images are calculated using TM and the binary handwritten digits. We calculate 7,040 speckle images corresponding to different input binary images for the digits '0-9' (704 speckle i...

Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media

“…Indeed, as tested in our experiment, a single diffuser trained CNN does not capture sufficient statistical variations to interpret speckle patterns from other diffusers. Another closely related line of work is using DL to imaging through multi-mode fibers (MMF) [8,12]. Image transfer through a MMF also results in speckle patterns due to spatial mode mixing.…”

Artificial intelligence-assisted light control and computational imaging through scattering media

“…The authors further showed that such capability was observed for MMFs of up to 1 km long through training the DNNs with a database of 16,000 handwritten digits. Most recently, Fan et al 112 showed that a CNN can be trained under multiple MMF transmission states to accurately predict the input patterns at the proximal side of the MMF at any of these states, exhibiting a sig-ni¯cant generalization capacity for di®erent MMF states. The aforementioned recent studies are summarized in Table 1.…”

Transmission of natural scene images through a multimode fibre

“…This matrix connects certain orthogonal modes at the fibre input to the fibre output and can therefore, once known, be used to invert the speckle pattern back into the original image. This approach requires measurements of the full complex (amplitude and phase) profile of a large subset of modes and has been shown to work over fibre lengths of 0.3-1 m. Other approaches pioneered by Takagi et al [17], have recently been proposed that used artificial neural networks (ANNs) using deep learning encoders to infer images from the speckles patterns without any need for an a priori mathematical model of the fibre [18][19][20].…”