Sifter

Las-Casas, Pedro; Papakerashvili, Giorgi; Anand, Vaastav; Mace, Jonathan

doi:10.1145/3357223.3362736

Cited by 26 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To ensure the captured data is useful, sampling decisions are coherent per request -a trace is either sampled in its entirety, capturing the full end-to-end execution, or not at all. Sampling effectively reduces computational overheads; these overheads are only paid if a trace is sampled, so they can be easily reduced by reducing the sampling probability (see [22][23][24][25] with references therein).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Diagnosis-Effective Sampling of Application Traces

Poghosyan,

Harutyunyan,

Davtyan

et al. 2024

Preprint

View full text Add to dashboard Cite

Distributed tracing is a cutting-edge technology for monitoring, managing, and troubleshooting native cloud applications. It offers a more comprehensive and continuous observability, surpassing traditional logging methods, and is indispensable for navigating modern complex software architectures. However, the sheer volume of generated traces is staggering in distributed applications, and direct storage and utilization of every trace are impractical due to associated operational costs. This entails a sampling strategy to select which traces warrant storage and analysis. Historically, sampling methods have included a rate-based approach, often relying heavily on a manual configuration. There is a need for a more intelligent approach, and we propose a hierarchical sampling methodology to address multiple requirements concurrently. Initial rate-based sampling mitigates the overwhelming volume of traces, as no further analysis can be performed on this level. In the next stage, more nuanced analysis is facilitated based on the previous foundation, incorporating information regarding trace properties and ensuring the preservation of vital process details even under extreme conditions. This comprehensive approach not only aids in the visualization and conceptualization of applications but also enables more targeted analysis in later stages. As we delve deeper into the sampling hierarchy, the technique becomes tailored to specific purposes, such as simplifying application troubleshooting. In this context, the sampling strategy prioritizes the retention of erroneous traces from dominant processes, thus facilitating the identification and resolution of underlying issues. The focus of this paper is to reveal the impact of the sampling on troubleshooting efficiency. Leveraging intelligent and explainable artificial intelligence solutions enables the detection of malfunctioning microservices and provides transparent insights into root causes. We advocate for using rule-induction systems, which offer explainability and efficacy in decision-making processes. By integrating advanced sampling techniques with machine learning-driven intelligence, we empower organizations to navigate the complexities of large-scale distributed cloud environments effectively.

show abstract

Section: Related Workmentioning

confidence: 99%

“…This approach, known as head-based sampling, avoids the runtime costs of generating trace data as it occurs uniformly at random, and the resulting data is simply a random subset of requests. In practice, sampling rates can be as low as 0.1% (see [22]).…”

Section: Related Workmentioning

confidence: 99%

Diagnosis-Effective Sampling of Application Traces

Poghosyan,

Harutyunyan,

Davtyan

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…For a review of tracing tools for large distributed environments, the reader is referred to the survey of Las-Casas et al 34 Numerous projects are available to achieve this goal, most notably Zipkin 35 and Opentelemetry. 36…”

Section: F I G U R Ementioning

confidence: 99%

Distributed computation of the critical path from execution traces

2023

View full text Add to dashboard Cite

Due to the ever‐increasing number of computer nodes in distributed systems, efficient and effective tools have become crucial for their analysis. Although several efficient methods have been proposed to monitor and profile distributed systems, tracing remains the most effective solution for in‐depth system analysis. Tracing is the act of collecting a trace, which is a sequence of low‐level events generated by the kernel or the userspace. After data collection, the most important part is the event analysis. The paradigm and choice of graphs determine the ability of the user to detect abnormal behaviors and identify their root cause. Although tracing is a highly effective approach to analyzing complex systems, the scalability of the current analysis tools is limited. As a consequence, tracing is often impractical for large distributed systems. This paper identifies the shortcomings of the current approaches, most notably the critical path computation and the trace file transfer between nodes. Then, this paper proposes new solutions to these drawbacks, most notably a distributed algorithm to compute the critical path, that does not aggregate all traces in a single node, and an efficient architecture to perform tracing on distributed systems. These new solutions are made publically available.

show abstract

“…The data used for observation is collected from multiple sources, meaning they must be aligned together carefully to reconstruct the entire sequence of events that led to the failure. Logs, metrics, and traces offer comprehensive visibility into the behavior of the different systems, where the correlation of traces among humans would be a tough one [3]. Uniting fragments of the process across services is a complex thing to do without a proper automated approach.…”

Section: Problem Statementmentioning

confidence: 99%

Automated Root Cause Analysis with Observability Data - A Comprehensive Review

Mahida

2023

J Eng App Sci Technol

View full text Add to dashboard Cite

Identifying the root cause analysis is the key to the timely detection of errors in massive, multiple-functional software systems. Meanwhile, network development will become more intricate and non-transparent, leaving the human algorithm behind. The paper dedicates its resources to discussing ways to automate root cause analysis based on observability data, such as logs, metrics, and traces. Technologies including causal inference, anomaly detection, and pattern recognition are specialized techniques that allow us to identify the breach in the background of thousands of connected events. Data-driven tools in research and industry that consume observability data as input, uncover anomalies, model system topology, and rank probable root causes use these technologies. This should give the customer a shorter mean repair time, higher reliability, and security. Deeper adoption is perfect for chain management and the performance gains of individual developers. The coverage includes strategies of algorithmic and implementation of root cause analysis with the observability data. Related topics like service maps and anomalous numbers are also discussed where necessary. Scaling out the automatic diagnosis entity is investigated as an automated means to replace the manual ones faced with elaborate models.

show abstract

Sifter

Cited by 26 publications

References 9 publications

Diagnosis-Effective Sampling of Application Traces

Diagnosis-Effective Sampling of Application Traces

Distributed computation of the critical path from execution traces

Automated Root Cause Analysis with Observability Data - A Comprehensive Review

Contact Info

Product

Resources

About