Towards highly reliable enterprise network services via inference of multi-level dependencies

Abstract

Localizing the sources of performance problems in large enterprise networks is extremely challenging. Dependencies are numerous, complex and inherently multi-level , spanning hardware and software components across the network and the computing infrastructure. To exploit these dependencies for fast, accurate problem localization, we introduce an Inference Graph model, which is well-adapted to user-perceptible problems rooted in conditions giving rise to both partial service degradation and hard faults. Further, we introduce the Sherlock system to discover Inference Graphs in the operational enterprise, infer critical attributes, and then leverage the result to automatically detect and localize problems. To illuminate strengths and limitations of the approach, we provide results from a prototype deployment in a large enterprise network, as well as from testbed emulations and simulations. In particular, we find that taking into account multi-level structure leads to a 30% improvement in fault localization, as compared to two-level approaches.

NSDMiner: Automated discovery of Network Service Dependencies

“…Thus, NSDMiner is not fully free of user input. Nevertheless, parameter tuning is required by all existing approaches such as Orion [8] and Sherlock [4]. Moreover, as we will see in our evaluation in Section III-D3 even with a very low threshold NSDMiner still reports much lower false positives than the best existing solutions.…”

“…Various types of dependencies in distributed environments were identified in [13], and Leslie Graph was used as an abstraction to describe complex dependencies between network components [3]. Sherlock was developed to learn service dependencies based on co-occurrence of network traffic and was employed for fault localization [4]. eXpose uses a modified JMeasure computation on partitioned packet trace to learn dependencies [11].…”

“…eXpose uses a modified JMeasure computation on partitioned packet trace to learn dependencies [11]. Both Sherlock [4] and eXpose [11] require a window size parameter, whose choice influences significantly the detection accuracy. Orion was recently developed to use spike detection analysis in delay distribution of flow pairs to infer dependencies [8].…”

“…To further evaluate our approach, we compared NSDMiner with two previous solutions, Orion [8] and Sherlock [4], which are the two more effective ones among the network-based approaches. We implemented both methods as described in their papers.…”

Self-diagnosis for large scale wireless sensor networks

“…SCORE [15] and Shrink [12] diagnose the network failures based on shared risk modeling. Sherlock [4] presents a multi-level inference model that achieves high performance. Giza [19] try to address the performance diagnosis problem in a large IPTV network and NetMedic [13] enables detailed diagnosis in enterprise networks.…”