Combining density-based clustering and wavelet methods for internal systems anomaly detection

Abstract

Internal information systems play an important role in keeping the enterprises running well. To detect system anomalies, previous research achieved good results with system symptoms; however, the presented results are primarily performed on a relatively small scale and within a short time period. To understand the system's long-term profiles, we collected four common symptom data including CPU usage, memory loading, disk I/O, and network I/O from more than 100 online internal systems that includes 300 servers for 9 months. We randomly selected 50 servers from these servers and analyze their data in order to understand each symptom's long-term features. Based on our findings in network I/O, we propose a new approach combining a density-based clustering and wavelet methods to detect system anomalies. We also select 44 other servers to evaluate the false positive rate and simulate three types of system anomalies to evaluate the detection rate. The experiment results show that our approach has a great improvement on both the false positive rate and the detection rate compared to another wavelet-based network anomaly detection approach.