2017
DOI: 10.4236/jsea.2017.1011047
|View full text |Cite
|
Sign up to set email alerts
|

Research on Establish an Efficient Log Analysis System with Kafka and Elastic Search

Abstract: Search logs in a timely and efficient manner are an important part of SRE (Site Reliability Engineer). Logs help us solve the problems during our development work. In this paper, we will introduce you a way how to build an efficient logs analysis system based on kafka and Elastic Search. We hope you can learn something through the iteration of the Version and get some inspiration with your own log analysis system.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
1
1
1

Citation Types

0
3
0

Year Published

2019
2019
2023
2023

Publication Types

Select...
3
2

Relationship

2
3

Authors

Journals

citations
Cited by 7 publications
(3 citation statements)
references
References 2 publications
0
3
0
Order By: Relevance
“…On the other hand, ResNet can solve the problem of degradation and can achieve better performance with a deeper architecture. However, ResNet has a more complex architecture and requires more computational resources for training [13].…”
Section: Comparison Between Cnn and Resnetmentioning
confidence: 99%
“…On the other hand, ResNet can solve the problem of degradation and can achieve better performance with a deeper architecture. However, ResNet has a more complex architecture and requires more computational resources for training [13].…”
Section: Comparison Between Cnn and Resnetmentioning
confidence: 99%
“…The Vertical-oriented Monitoring System (5Gr-VoMS) is an extension of 5G-TRANSFORMER monitoring platform (5GT-MP), designed with the objective of supporting heterogeneous set of services and technological domains; and, likewise, novel innovations devoted to enhance end-to-end reliability (via selfhealing and auto-scaling), vertical control-loops stability, and analytical features, such as forecasting, inference and anomaly detection. In order to provide such innovation support, 5GT-MP must be extended to include additional functionalities such as log aggregation, a scalable data distribution system and dynamic probe reconfiguration [7]. Elastic stack is included in the VoMS architecture to support log aggregation, Kafka distributed streaming platform as scalable data distribution system and Elastic Beats which will, together with Prometheus node exporter, assist to the dynamic reconfiguration of the monitoring probes.…”
Section: Vertical-oriented Monitoring System (5gr-voms)mentioning
confidence: 99%
“…This paper proposes a neural network-based cancer classification model, studying 802 people who were detected with different types of cancer, each sample contains more than 20K gene *These authors contributed equally to this work. Journal of Software Engineering and Applications expression values, and the ultimate goal is to build a robust classification model to identify each type of cancer, and we processed the collected cancer data by Kafka [2]. This will allow us to identify each type of cancer early, thereby reducing mortality.…”
Section: Introductionmentioning
confidence: 99%