We propose a novel digital pre-distortion (DPD) based on neural networks for high-baudrate optical coherent transmitters. We demonstrate experimentally that it outperforms an optimized linear DPD giving a 1.2 dB SNR gain in a 128GBaud PCS-256QAM single-channel transmission over 80km of standard single-mode fiber resulting in a record 1.61 Tb/s net data rate.

Nonlinear frequency division multiplexing (NFDM) techniques encode information in the so called nonlinear spectrum which is obtained from the nonlinear Fourier transform (NFT) of a signal. NFDM techniques so far have been applied to the nonlinear Schrödinger equation (NLSE) that models signal propagation in a lossless fiber. Conventionally, the true lossy NLSE is approximated by a lossless NLSE using the pathaverage approach which makes the propagation model suitable for NFDM. The error of the path-average approximation depends strongly on signal power, bandwidth and the span length. It can degrade the performance of NFDM systems and imposes challenges on designing high data rate NFDM systems. Previously, we proposed the idea of using dispersion decreasing fiber (DDF) for NFDM systems. These DDFs can be modeled by a NLSE with varying-parameters that can be solved with a specialized NFT without approximation errors. We have shown in simulations that complete nonlinearity mitigation can be achieved in lossy fibers by designing an NFDM system with DDF if a properly adapted NFT is used. We reported performance gains by avoiding the aforementioned path-average error in an NFDM system by modulating the discrete part of the nonlinear spectrum. In this paper, we extend the proposed idea to the modulation of continuous spectrum. We compare the performance of NFDM systems designed with dispersion decreasing fiber to that of systems designed with a standard fiber with path-average model. Next to the conventional path-average model, we furthermore compare the proposed system with an optimized path-average model in which amplifier locations can be adapted. We quantify the improvement in the performance of NFDM systems that use DDF through numerical simulations.Index Terms-Fiber-optic communication, nonlinear frequency division multiplexing, nonlinear Fourier transform, dispersion decreasing fiber.

High-symbol-rate coherent optical transceivers suffer more from the critical responses of transceiver components at high frequency, especially when applying a higher order modulation format. We recently proposed a neural network (NN)-based digital pre-distortion (DPD) technique trained to mitigate the transceiver response of a 128 GBaud optical coherent transmission system. In this paper, we further detail this work and assess the NN-based DPD by training it using either a direct learning architecture (DLA) or an indirect learning architecture (ILA), and compare performance against a Volterra series-based ILA DPD and a linear DPD. Furthermore, we deliberately increase the transmitter nonlinearity and compare the performance of the three DPDs schemes. The proposed NN-based DPD trained using DLA performs the best among the three contenders. In comparison to a linear DPD, it provides more than 1 dB signal-tonoise ratio (SNR) gains at the output of a conventional coherent receiver DSP for uniform 64-quadrature amplitude modulation (QAM) and PCS-256-QAM signals. Finally, the NN-based DPD enables achieving a record 1.61 Tb/s net rate transmission on a single channel after 80 km of standard single mode fiber (SSMF).

No abstract