The trade-off between more user bandwidth and quality of service requirements introduces unprecedented challenges to the next generation smart optical networks. In this regard, the use of optical performance monitoring (OPM) and modulation format identification (MFI) techniques becomes a common need to enable the development of next-generation autonomous optical networks, with ultra-low latency and selfadaptability. Recently, machine learning (ML)-based techniques have emerged as a vital solution to many challenging aspects of OPM and MFI in terms of reliability, quality, and implementation efficiency. This paper surveys ML-based OPM and MFI techniques proposed in the literature. First, we address the key advantages of employing ML algorithms in optical networks. Then, we review the main optical impairments and modulation formats being monitored and classified, respectively, using ML algorithms. Additionally, we discuss the current status of optical networks in terms of MFI and OPM. This includes standards, monitoring parameters, and the available commercial products with their limitations. Second, we provide a comprehensive review of the available ML-based techniques for MFI, OPM, and joint MFI/OPM, describing their performance, advantages, and limitations. Third, we give an overview of the exiting ML-based OPM and MFI techniques for the emerging optical networks such as the new fiber-based networks that use future space division multiplexing techniques (e.g. few-mode fiber), the hybrid radioover-fiber networks, and the free space optical networks. Finally, we discuss the open issues, potential future research directions, and recommendations for the potential implementation of MLbased OPM and MFI techniques. Some lessons learned are presented after each section throughout the paper to help the reader identifying the gaps, weaknesses, and strengths in this field.

In this paper, we study the separability of the commonly used features to monitor the performance of optical signal in coherent optical systems. Specifically, our study focuses on the histogram-based features; the asynchronous amplitude histogram (AAH) and the two-dimensional extension of AAH, which we call IQ histogram (IQH). We investigate the conditions under which the optical channel impairments can be monitored. This study utilizes a dimensionality reduction technique, known as the t-distribution stochastic neighbor embedding (t-SNE). Using t-SNE, we show that under certain conditions the histogrambased features cannot be used to distinguish between the high and low impairment effects, rendering these features useless for certain cases, regardless of the type or complexity of the implemented impairments' estimator/classifier. Extensive simulations results have been conducted to investigate single and multiple impairments conditions and the performance of histogram-based features in each case. The results show that both AAH and IQH can be used for monitoring all types of single Impairment except phase noise in case of AAH. Moreover, under multiple impairments conditions, polarization mode dispersion values can be monitored to some extent, while it is difficult to monitor optical signal-to-noise ratio and chromatic dispersion values especially in the case of concurrent presence of more than two impairments. The results of these investigations are validated by providing a quantitative measure that ties their usefulness to the actual monitoring performance through the estimation of channel impairments under different conditions using linear support vector machine (SVM) regression. It has been found that there is a match to a great extent between the separability investigation using t-SNE and estimation results obtained using SVM.

The unique orthogonal shapes of structured light beams have attracted researchers to use as information carriers. Structured light-based free space optical communication is subject to atmospheric propagation effects such as rain, fog, and rain, which complicate the mode demultiplexing process using conventional technology. In this context, we experimentally investigate the detection of Laguerre Gaussian and Hermite Gaussian beams under dust storm conditions using machine learning algorithms. Different algorithms are employed to detect various structured light encoding schemes including the use of a convolutional neural network (CNN), support vector machine, and k-nearest neighbor. We report an identification accuracy of 99% under a visibility level of 9 m. The CNN approach is further used to estimate the visibility range of a dusty communication channel.

This paper considers, for the first time, optical performance monitoring (OPM) in few mode fiber (FMF)-based optical networks. One dimensional (1D) features vector, extracted by projecting a two-dimensional (2D) asynchronous in-phase quadrature histogram (IQH), and the 2D IQH are proposed to achieve OPM in FMF-based network. Three machine learning algorithms are employed for OPM and their performances are compared. These include support vector machine, random forest algorithm, and convolutional neural network. Extensive simulations are conducted to monitor optical to signal ratio (OSNR), chromatic dispersion (CD), and mode coupling (MC) for dual polarization-quadrature phase shift keying (DP-QPSK) at 10, 12, 16, 20, and 28 Gbaud transmission speeds. Besides, M-ary quadrature amplitude modulation (M = 8 and 16) is considered. Also, the OPM accuracy is investigated under different FMF channel conditions including phase noise and polarization mode dispersion. Simulation results show that the proposed 1D projection features vector provides better OPM results than those of the widely used asynchronous amplitude histogram (AAH) features. Furthermore, it has been found that the 2D IQH features outperform the 1D projection features but require larger number of features samples. Additionally, the effect of fiber nonlinearity on the OPM accuracy is investigated. Finally, OPM using the 2D IQH features has been verified experimentally for 10 Gbaud DP-QPSK signal. The obtained results show a good agreement between both simulation and experimental findings.