ML-assisted QoT estimation: a dataset collection and data visualization for dataset quality evaluation

Bergk, Geronimo; Shariati, Behnam; Safari, Pooyan; Fischer, Johannes

doi:10.1364/jocn.442733

Cited by 13 publications

(18 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another approach that is employed to obtain probabilistic forecasting is the Bayesian inference method. This method assigns distribution to parameters based on prior experience before data collection and applies Bayes' theorem to revise the distribution after obtaining data [10]. The drawback of the Bayesian approach is its complex and high cost computation [20].…”

Section: B Uncertaintymentioning

confidence: 99%

Forecasting Lightpath Quality of Transmission and Implementing Uncertainty in the Forecast Models

Yousefi

Chouman

Djukic

et al. 2023

J. Lightwave Technol.

View full text Add to dashboard Cite

The recent popularity of using deep learning models for the forecasting of time series calls for methods to not only predict the target but also measure the uncertainty of the prediction accurately. Working with time series requires reliable and stable forecasters. An essential component of the reliability of machine learning (ML) and deep learning (DL) models is the estimation of the uncertainty. In this work, we address building and characterizing time series forecasters, including N-Beats, Long Short-Term Memory (LSTM) and Multilayer Perceptron (MLP) against the Naive model, and define the confidence margins, and uncertainty for the selected model.All the implementations are conducted in Python programming language. Random sampling is performed to avoid overfitting. Our target field data is North American Service Provider data sets (NASP). Among the implemented models, the MLP model is selected to measure the uncertainty and confidence level, and the Monte Carlo dropout, which approximates Bayesian uncertainty, is applied during inference to render the implementation of uncertainty calculations. Quantile Regression is also implemented on the MLP algorithm as a baseline to predict the confidence intervals and to evaluate our strategy for estimating uncertainty. To establish reliable uncertainty estimation in time series predictions, we performed uncertainty calibration. Motivated by recent developments in Expected Uncertainty Calibration Error (UCE), we modified the uncertainty calculated by the probabilistic Bayesian estimations. Detailed experiments and architectures of the solution are presented.

show abstract

Section: B Uncertaintymentioning

confidence: 99%

Forecasting Lightpath Quality of Transmission and Implementing Uncertainty in the Forecast Models

Yousefi

Chouman

Djukic

et al. 2023

J. Lightwave Technol.

View full text Add to dashboard Cite

show abstract

“…Recently, machine learning (ML), has emerged as a powerful tool towards alleviating the limitations of traditional PLMs. Specifically, ML was shown to be more appropriate for modeling the non-linear nature of physical layer impairments [4]- [7], leading to margin reduction and consequently to spectrum savings [8], [9]. In fact, the ability of ML to find deep QoT models of sufficient accuracy is validated both in the field [10], [11] and experimentally [5], [12] with the use of real datasets.…”

Section: Introductionmentioning

confidence: 97%

Representing Uncertainty in Deep QoT Models

Maryam

Panayiotou

Ellinas

2022

2022 20th Mediterranean Communication and Computer Networking Conference (MedComNet)

View full text Add to dashboard Cite

Quality-of-transmission (QoT) estimation of unestablished lightpaths has been extensively studied in the literature through the development of linear physical layer models (PLMs) and more recently through the application of machine learning (ML). ML has proven its superiority over PLMs on more accurately modeling the non-linear nature of physical layer impairments (PLIs), leading to margin reduction and consequently to network capacity savings. Currently, in optical networks, margins compensating for ML model uncertainties are estimated through empirical myopic approaches, largely ignoring the fact that diverse input lightpaths are subject to different levels of uncertainty. Clearly, to further exploit the advantages of ML, it is necessary that uncertainty is appropriately represented through margins that capture uncertainty over each unestablished lightpath. In this work, we present and compare two uncertainty representation frameworks for deep QoT estimation models. The first framework is based on deep quantile regression and the second on the Monte Carlo dropout technique. Both frameworks achieve further margin reductions and improve classification accuracy of unestablished lightpaths, as opposed to myopic margins traditionally considered. Further, it is shown that the Monte Carlo dropout technique results in better representation of the model uncertainty compared to the deep quantile framework. Importantly, the implementation strengths and limitations of each framework are for the first time revealed and discussed.

show abstract

“…Machine learning (ML), with proven capabilities on sufficiently modeling the non-linear nature of physical layer impairments (PLIs), has been extensively studied in the optical networks literature with the purpose of accurately estimating the quality-of-transmission (QoT) of unseen (unestablished) lightpaths [1][2][3][4]. This information, is then used to examine the feasibility of any unseen lightpath and take decisions accordingly; that is, establish or deny service to the unseen lightpath under consideration.…”

Section: Introductionmentioning

confidence: 99%