LSTM networks enabled nonlinear equalization in 50-Gb/s PAM-4 transmission links

Dai, Xiaoxiao; Li, Xiang; Luo, Ming; You, Quan; Yu, Shaohua

doi:10.1364/ao.58.006079

Cited by 56 publications

(16 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of channel equalization, the LSTM was suggested in [35], [36] to reduce the transmission impairments in IM/DD systems with pulse amplitude modulation (PAM). The LSTM-based approach was developed further in [17], where, for the first time, the biLSTM was used in an optical coherent system to compensate for fiber nonlinearities, but only in a simulation environment.…”

Section: B Long Short-term Memory Nnsmentioning

confidence: 99%

“…It was demonstrated that the application of the NNs with different internal structures, such as multi-layer perceptron (MLP) [11], [12] (i.e. a simple densely connected feedforward NN architecture), convolutional NNs (CNN) [13], [14], echo state networks (ESN) [15], and long short-term memory (LSTM) NNs [16], is efficient in improving optical system-level performance. However, the test of similar NN architectures in coherent optical systems has been carried out, mainly, numerically [17]- [20], or in short-haul experiments [21]- [24].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Performance Versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

Freire

Osadchuk

Spinnler

et al. 2021

J. Lightwave Technol.

116

View full text Add to dashboard Cite

We present the results of the comparative performance-versus-complexity analysis for the several types of artificial neural networks (NNs) used for nonlinear channel equalization in coherent optical communication systems. The comparison is carried out using an experimental set-up with the transmission dominated by the Kerr nonlinearity and component imperfections. For the first time, we investigate the application to the channel equalization of the convolution layer (CNN) in combination with a bidirectional long short-term memory (biLSTM) layer and the design combining CNN with a multilayer perceptron. Their performance is compared with the one delivered by the previously proposed NN-based equalizers: one biLSTM layer, three-dense-layer perceptron, and the echo state network. Importantly, all architectures have been initially optimized by a Bayesian optimizer. First, we present the general expressions for the computational complexity associated with each NN type; these are given in terms of real multiplications per symbol. We demonstrate that in the experimental system considered, the convolutional layer coupled with the biLSTM (CNN+biLSTM) provides the largest Q-factor improvement compared to the reference linear chromatic dispersion compensation (2.9 dB improvement). Then, we examine the trade-off between the computational complexity and performance of all equalizers and demonstrate that the CNN+biLSTM is the best option when the computational complexity is not constrained, while when we restrict the complexity to some lower levels, the three-layer perceptron provides the best performance.

show abstract

Section: B Long Short-term Memory Nnsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Performance Versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

Freire

Osadchuk

Spinnler

et al. 2021

J. Lightwave Technol.

116

View full text Add to dashboard Cite

show abstract

“…In recent years, various DSP algorithms have been reported in IM/DD systems to alleviate or mitigate CD effect, such as feed-forward equalization (FFE), decision feedback equalization (DFE), Volterra equalizer (VE), maximum likelihood sequence estimation (MLSE), Tomlinson-Harashima pre-coding and neural network. Among these algorithms, neural network with the inherent advantage of approximating any nonlinear function is considered to have great potential to mitigate CD effect and device nonlinear effect in IM/DD systems [8][9][10][11]. The development of integrated photonics technology [12] offers great potential for the application of neural networks.…”

Section: Introductionmentioning

confidence: 99%

56-Gbit/s PAM-4 Optical Signal Transmission Over 100-km SMF Enabled by TCNN Regression Model

et al. 2021

View full text Add to dashboard Cite

Due to the interaction between chromatic dispersion (CD) and direct detection, the CD induced power-fading effect has been considered as the key point to limit transmission rate and distance in intensity modulation direct-detection (IM/DD) systems. Besides, the performance of the IM/DD systems will be further affected by the bandwidth limitation and nonlinearity effect. In this paper, we propose a low complexity nonlinear equalizer (NLE) based on temporal convolutional neural network (TCNN) regression model which combines dilated convolutions with residual connections. Then, the performance of our proposed equalizer is experimentally demonstrated in 56-Gbit/s 4-level pulse amplitude modulation (PAM4) intensity modulation direct-detection system. The results show that the receiver sensitivity with the help of our TCNN equalizer can be improved about 1.5 dB and 3.5 dB compared to 2D-CNN and 1D-CNN equalizer under 70-km single mode fiber (SMF) transmission. Furthermore, the proposed equalizer can extend maximum transmission distance to 100 km at bit error rate threshold of 3.8×10 -3 .

show abstract

“…The application of machine learning technique in optical communication systems has been studied in many fields in recent years [1,2]. In the field of optical communication systems, many parts of the system, such as performance monitoring, fiber nonlinearity mitigation, carrier recovery, and equalization, have been optimized by machine learning and a neural network [3][4][5][6]. In particular, as we all know, chromatic dispersion (CD) and nonlinear Kerr effects in the fiber are the main constraint in the improvement of the signal rate in the optical communication system today [7].…”

Section: Introductionmentioning

confidence: 99%

An Improved End-to-End Autoencoder Based on Reinforcement Learning by Using Decision Tree for Optical Transceivers

et al. 2021

View full text Add to dashboard Cite

In this paper, an improved end-to-end autoencoder based on reinforcement learning by using Decision Tree for optical transceivers is proposed and experimentally demonstrated. Transmitters and receivers are considered as an asymmetrical autoencoder combining a deep neural network and the Adaboost algorithm. Experimental results show that 48 Gb/s with 7% hard-decision forward error correction (HD-FEC) threshold under 65 km standard single mode fiber (SSMF) is achieved with proposed scheme. Moreover, we further experimentally study the Tree depth and the number of Decision Tree, which are the two main factors affecting the bit error rate performance. Experimental research afterwards showed that the effect from the number of Decision Tree as 30 on bit error rate (BER) flattens out under 48 Gb/s for the fiber range from 25 km and 75 km SSMF, and the influence of Tree depth on BER appears to be a gentle point when Tree Depth is 5, which is defined as the optimal depth point for aforementioned fiber range. Compared to the autoencoder based on a Fully-Connected Neural Network, our algorithm uses addition operations instead of multiplication operations, which can reduce computational complexity from 108 to 107 in multiplication and 106 to 108 in addition on the training phase.

show abstract

LSTM networks enabled nonlinear equalization in 50-Gb/s PAM-4 transmission links

Cited by 56 publications

References 10 publications

Performance Versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

Performance Versus Complexity Study of Neural Network Equalizers in Coherent Optical Systems

56-Gbit/s PAM-4 Optical Signal Transmission Over 100-km SMF Enabled by TCNN Regression Model

An Improved End-to-End Autoencoder Based on Reinforcement Learning by Using Decision Tree for Optical Transceivers

Contact Info

Product

Resources

About