LOGAN: Problem Diagnosis in the Cloud Using Log-Based Reference Models

Tak, Byung Chul; Tao, Shu; Yang, Lin; Zhu, Chao; Ruan, Yaoping

doi:10.1109/ic2e.2016.12

Cited by 41 publications

(18 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fundamental knowledge of diverse approaches of error log processing is found in [27], [28]. Some simple mining methods include log key terms occurrence correlation [29], and modeling a multithreaded system behavior through graphs or sequences representing system calls. For instance, the authors of [27] deal with the problem of failure prediction through clustering similar log sequences.…”

Section: Mining Logs For Root Cause Classification and Diagnosticsmentioning

confidence: 99%

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Zasadzinski¹,

Solé²,

Brandon

et al. 2018

2018 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Performing diagnostics in IT systems is an increasingly complicated task, and it is not doable in satisfactory time by even the most skillful operators. Systems and their architecture change very rapidly in response to business and user demand. Many organizations see value in the maintenance and management model of NoOps that stands for No Operations. One of the implementations of this model is a system that is maintained automatically without any human intervention. The path to NoOps involves not only precise and fast diagnostics but also reusing as much knowledge as possible after the system is reconfigured or changed. The biggest challenge is to leverage knowledge on one IT system and reuse this knowledge for diagnostics of another, different system. We propose a framework of weighted graphs which can transfer knowledge, and perform high-quality diagnostics of IT systems. We encode all possible data in a graph representation of a system state and automatically calculate weights of these graphs. Then, thanks to the evaluation of similarity between graphs, we transfer knowledge about failures from one system to another and use it for diagnostics. We successfully evaluate the proposed approach on Spark, Hadoop, Kafka and Cassandra systems.

show abstract

Section: Mining Logs For Root Cause Classification and Diagnosticsmentioning

confidence: 99%

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Zasadzinski¹,

Solé²,

Brandon

et al. 2018

2018 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

show abstract

“…As the scale of modern IT services grows with increasing diversity of component architectures and behavioral patterns, the mission of seamless operation and efficient management becomes more challenging. A primary technique of achieving such goals is through log analytics [1][2][3][4][5][6]. The logs, generated from the system software, middleware, as well as applications, are the continuous stream of textual information that encode internal states of running applications.…”

Section: Introductionmentioning

confidence: 99%

Priolog: Mining Important Logs via Temporal Analysis and Prioritization

2019

View full text Add to dashboard Cite

Log analytics are a critical part of the operational management in today’s IT services. However, the growing software complexity and volume of logs make it increasingly challenging to mine useful insights from logs for problem diagnosis. In this paper, we propose a novel technique, Priolog, that can narrow down the volume of logs into a small set of important and most relevant logs. Priolog uses a combination of log template temporal analysis, log template frequency analysis, and word frequency analysis, which complement each other to generate an accurately ranked list of important logs. We have implemented this technique and applied to the problem diagnosis task of the popular OpenStack platform. Our evaluation indicates that Priolog can effectively find the important logs that hold direct hints to the failure cause in several scenarios. We demonstrate the concepts, design, and evaluation results using actual logs.

show abstract

“…Currently, identifying the root cause of failures in cloud-computing environments has been considered a critical process. TAK et al, 2016). In addition, the interactions between components and the large number of user requests served simultaneously make problematic and challenging the process of discovering the root cause, since a single problem can trigger multiple alarms (MI et al, 2012).…”

Section: Challengesmentioning

confidence: 99%

“…• Accuracy: RCA systems must ensure high accuracy in the root cause identification, at low false positive rates (TAK et al, 2016). Since data in cloud computing are collected from multiple components, multiple nodes and multiple levels of abstraction, it can result in a high data volume and data variety.…”

Section: Requirementsmentioning

confidence: 99%

See 1 more Smart Citation

A data-driven solution for root cause analysis in cloud computing environments.

Pereira¹

View full text Add to dashboard Cite

Maybe you are searching among the branches, for what only appears in the roots." Rumi AGRADECIMENTOS Iniciei meu mestrado com o seguinte pensamento: terminar o mais rápido possível. No entanto, por minha sorte, não foi isso que aconteceu. Esses anos que me dediquei ao mestrado foram transformadores. Embora tenha levado um pouco mais de tempo que eu gostaria, pude passar por experiências que agregaram muito em minha vida pessoal e profissional. Nesse período passei por atividades como a pesquisa, prototipação, escrita de artigos, reuniões de projetos, oportunidades de liderança, palestras, viagens, escrita de livro, cursos e hackathons. Meu sentimento era de que, cada vez que realizava alguma dessas atividades, eu agregava um pouco mais de conhecimento e aprendia algo novo que levaria para a vida.Hoje, olhando para essa trajetória, percebo o quanto passar por tudo isso foi importante para meu aperfeiçoamento, e fico imensamente grata por ter esse registro em minha história de vida. No entanto, esse mestrado foi até hoje um dos maiores desafios pelo qual já passei. Por esse motivo, sem o suporte dos amigos, família e profissionais, não tenho certeza se essa etapa teria sido completada.Muito obrigada Deus, pela saúde, pela força, pela vida.Pai e mãe, vocês são meu alicerce para todas as minhas realizações. Obrigada pelo amor incondicional e por me trazer ao mundo em uma família tão especial, com os amados irmãos Ivan e Ronaldo. Obrigada também à todos da minha família, pelo apoio e encorajamento contínuo. Marido Evandro, você passou por toda essa etapa comigo, na alegria e na tristeza, sempre com muito amor, carinho e compreensão. Sou imensamente feliz por viver ao seu lado.Agradeço minha orientadora, Profa. Tereza Cristina Melo de Brito Carvalho, por todas as conversas, mentoria, cuidado, oportunidades e incentivo. As conversas contigo foram sempre muito especiais, sempre um grande aprendizado. Muito obrigada pela confiança, pelo suporte e pela energia que me passa.Agradeço também a Karen Langona, uma grande amiga e profissional, da qual me inspira com suas ações, capacidade de resolver problemas, autenticidade e carisma. Foi um imenso prazer trabalhar contigo. Vocês são mulheres admiráveis.Agradeço ao Laboratório de Arquitetura e Redes de Computadores (LARC-USP), em especial ao prof. Wilson Vicente Ruggiero, pela oportunidade de atuar com pesquisa e por todo o auxílio dado para meu aperfeiçoamento profissional. Não há como esquecer dos grandes amigos que encontrei nesse laboratório: Akio, Nelson, orgulho de fazer parte desse time.Também agradeço meu orientador da graduação, Francisco Pereira Junior, por sempre acreditar que eu poderia conseguir o que queria. Estamos juntos nessa meu amigo. De coração, também agradeço a todos meus amigos que estão celebrando comigo a conquista dessa etapa. Conseguimos! Por fim, agradeço a Fundação para o Desenvolvimento Tecnológico da Engenharia (FDTE), Ericsson Research e Universidade de São Paulo (USP) pelo suporte à nossa pesquisa.

show abstract

LOGAN: Problem Diagnosis in the Cloud Using Log-Based Reference Models

Cited by 41 publications

References 7 publications

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Priolog: Mining Important Logs via Temporal Analysis and Prioritization

A data-driven solution for root cause analysis in cloud computing environments.

Contact Info

Product

Resources

About