Unveiling network and service performance degradation in the wild with mplane

Abstract

Abstract-Unveiling network and service performance issues in complex and highly decentralized systems such as the Internet is a major challenge. Indeed, the Internet is based on decentralization and diversity. However, its distributed nature leads to operational brittleness and difficulty in identifying the root causes of performance degradation. In such a context, network measurements are a fundamental pillar to shed light and to unveil design and implementation defects. To tackle this fragmentation and visibility problem, we have recently conceived mPlane, a distributed measurement platform which runs, collects and analyses traffic measurements to study the operation and functioning of the Internet. In this paper, we show the potentiality of the mPlane approach to unveil network and service degradation issues in live, operational networks, involving both fixed-line and cellular networks. In particular, we combine active and passive measurements to troubleshoot problems in end-customer Internet access connections, or to automatically detect and diagnose anomalies in Internet-scale services (e.g., YouTube) which impact a large number of end-users.

Results from Running an Experiment as a Service Platform for Mobile Networks

“…Once at the server, the JSON files are stored in a NoSQL database. In addition to the default experiments, each node runs an mPlane [5] compatible passive traffic analysis tool called Tstat [9]. Tstat generates a series of logs that the nodes send to the mPlane repository, from where users can consume the data using the mPlane visualization solution.…”

“…At the node side, metadata distribution is implemented in a publish/subscribe pattern using ZeroMQ. 5 The metadata stream is available for experiments during their execution using the ZMQ subscription mechanisms. Metadata entries are generated in a single-line JSON format, which eases human analysis.…”

“…5 http://zeromq.org/6 http://cassandra.apache.org/ Experiment workflow covering the design, test and experimentation phases.experiment may be related to the network conditions during its execution, even if at that time not all the metadata was checked online. Researchers may also import the CSV dumps into their own tools for more specific data analyses.…”

A Data-Driven Customer Quality of Experience System for a Cellular Network

“…We adopted a method based on SPC, which is known to be very helpful for detecting abnormal behavior [34]. We used popular rules called the Western electric rules (for the zone rules and the asymmetric rules, refer to https://en.wikipedia.org/wiki/Western Electric rules), especially the zone the asymmetric rules, which are used to find abnormal patterns in SPC (for other approaches to detect anomalies, refer to [35,36]), for example, "two consecutive values higher (or lower) than 2 " or "four consecutive points out of five higher (or lower) than " to detect the bad perceived quality from frequent, consecutive, or intense defects at the subscriber level. Each call generates a sequence of eMOS scores, as shown in Figure 4.…”

Detecting anomalous packets in network transfers: investigations using PCA, autoencoder and isolation forest in TCP

“…Anomalous network transfers have been classified into three major groups-packet loss, packet duplication and retransmissions (Mellia et al 2008a;Casas et al 2016). The retransmissions could be triggered by lack of client acknowledgments or some other error in the link.…”