A tutorial on the EM algorithm for Bayesian networks: Application to self-diagnosis of GPON-FTTH networks

Abstract

Abstract-Network behavior modelling is a central issue for model-based approaches of self-diagnosis of telecommunication networks. There are two methods to build such models. The model can be built from expert knowledge acquired from network standards and/or the model can be learnt from data generated by network components by data mining algorithms. In a recent work, we proposed a model of architecture and fault propagation for the GPON-FTTH (Gigabit Passive Optical Network-Fiber To The Home) access network. This model is based on a Bayesian network which encodes expert knowledge. This includes dependencies that encode fault propagation and conditional probability distributions that encode the strength of those dependencies. In this paper we consider the problem of automatically tuning the above mentioned probability distributions. This is a parameter estimation problem under missing data conditions that we solve with the Expectation Maximization (EM) algorithm. Conditional probability distributions are learnt from the tremendous amount of alarms generated by an operating GPON-FTTH network during two months in 2015. Self-diagnosis is carried out to analyze the root cause of alarms. The performance of the diagnosis is evaluated with respect to an expert system based on deterministic decision rules currently used by the Internet Access Provider to diagnose network problems.

An Overview on Application of Machine Learning Techniques in Optical Networks

“…Other instances of application of Bayesian models to detect and diagnose failures in optical networks, especially GPON/FTTH, are reported in [94] and [95]. In [94], the GPON/FTTH network is modeled as a Bayesian Network using a layered approach identical to one of their previous works [114]. The layer 1 in this case actually corresponds to the physical network topology consisting of ONTs, ONUs and fibers.…”

Artificial intelligence (AI) methods in optical networks: A comprehensive survey

“…Cognition-based methods [97]: detects failures in centralized SDN-based networks by periodically exchanging the messages between controller and switches. Bayesian inference/networks [19], [98]: probabilistic modeling and machine learning for fault diagnosis in optical access networks…”

Application of probabilistic modeling and machine learning to the diagnosis of FTTH GPON networks

“…In those cases, only one ONT is connected to the OLT port, upstream and downstream "loss of signal" alarms are observed, but no optical power measurement could be collected. The "faulty ONT" decision is fully reasonable for a human expert, although those situations could also result from fiber attenuation [6].…”

“…The conditional dependencies of PANDA model have then been finely tuned by machine learning thanks to an Expectation Maximization (EM) algorithm which is detailed in [6]. This EM algorithm was run on 5121 diagnosis cases collected from Orange FTTH network and used as a learning dataset.…”

“…The principle is to mine the dataset so as to learn the conditional dependencies which will afterwards enable Bayesian inference on other diagnosis cases. Expectation Maximization implements maximum likelihood estimation for an incomplete dataset, and leads to model parameters that "best match" the dataset [6]. The initial model parameters used for running the EM algorithm were those of the first PANDA implementation presented in the previous section.…”