Advancing theoretical understanding and practical performance of signal processing for nonlinear optical communications through machine learning

Abstract

In long-haul optical communication systems, compensating nonlinear effects through digital signal processing (DSP) is difficult due to intractable interactions between Kerr nonlinearity, chromatic dispersion (CD) and amplified spontaneous emission (ASE) noise from inline amplifiers. Optimizing the standard digital back propagation (DBP) as a deep neural network (DNN) with interleaving linear and nonlinear operations for fiber nonlinearity compensation was shown to improve transmission performance in idealized simulation environments. Here, we extend such concepts to practical single-channel and polarization division multiplexed wavelength division multiplexed experiments. We show improved performance compared to state-of-the-art DSP algorithms and additionally, the optimized DNNbased DBP parameters exhibit a mathematical structure which guides us to further analyze the noise statistics of fiber nonlinearity compensation. This machine learning-inspired analysis reveals that ASE noise and incomplete CD compensation of the Kerr nonlinear term produce extra distortions that accumulates along the DBP stages. Therefore, the best DSP should balance between suppressing these distortions and inverting the fiber propagation effects, and such trade-off shifts across different DBP stages in a quantifiable manner. Instead of the common 'black-box' approach to intractable problems, our work shows how machine learning can be a complementary tool to human analytical thinking and help advance theoretical understandings in disciplines such as optics.

Recent Progress on Novel DSP Techniques for Mode Division Multiplexing Systems: A Review

“…Recently, DBP is realized in a deep neural network architecture [32]. The architecture of this deep neural network (DNN) is shown in Figure 17, where the inputs are the received signal samples and the outputs are the estimated symbols.…”

“…Thus, all the linear and nonlinear parameters in traditional DBP can be optimized in the DNN. The PDM-WDM transmission system is experimentally constructed to investigate the performance of this DNN [32]. The experiment setup is shown in Figure 18, where a 5 channel 50-GHz-spaced WDM system with 28 Gbaud 16-QAM modulation format is utilized.…”

“…The digital back-propagation (DBP) algorithm is the most common DSP method to compensate the fiber nonlinearities in single mode fiber. And fortunately, the extension of DBP based single-mode split-step Fourier method (SSFM) has recently been proposed to solve the multimode propagation problem in FMFs [30][31][32].…”

A Layer-Reduced Neural Network Based Digital Backpropagation Algorithm for Fiber Nonlinearity Mitigation

“…The convolution process of E and E 2 in the frequency domain is shown in the left side of the figure, which indicates that the inband spectrum of σ i is also distorted. Using the gradient-descent-based optimizer, LDBP learns a set of "M"-shape linear layers to limit the broadened bandwidth [18], [19]. However, the in-band distortion of σ i can not be completely removed by learned linear layers.…”

Artificial Intelligence in Optical Communications: From Machine Learning to Deep Learning

“…Among them, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), deep reinforcement learning (DRL), end-to-end learning based on autoencoder, and their variants have made a distinctive contribution to fields such as machine vision, natural language processing, drug discovery, genomics, speech recognition, information retrieval, affective computing, and automatic deriving (Deng, 2014). Meanwhile, to promote the development of artificial intelligence (AI) in optical communication, the evolution from ML to DL is making major advances in a wide variety of applications in both physical and network layers (Fan et al, 2020;Häger and Pfister, 2020;Saif et al, 2020).…”