Anomaly Detection and Diagnosis for Container-based Microservices with Performance Monitoring

Abstract

With emerging container technologies, such as Docker, microservices-based applications can be developed and deployed in cloud environment much agiler. The dependability of these microservices becomes a major concern of application providers. Anomalous behaviors which may lead to unexpected failures can be detected with anomaly detection techniques. In this paper, an anomaly detection system(ADS) is designed to detect and diagnose the anomalies in microservices by monitoring and analyzing real-time performance data of them. The proposed ADS consists of a monitoring module that collects the performance data of containers, a data processing module based on machine learning models and a fault injection module integrated for training these models. The fault injection module is also used to assess the anomaly detection and diagnosis performance of our ADS. Clearwater, an open source virtual IP Multimedia Subsystem, is used for the validation of our ADS and experimental results show that the proposed ADS works well.

Straddling the crevasse: A review of microservice software architecture foundations and recent advancements

“…In order to ensure that the system is functioning as expected, analyze the data and apply anomaly detection techniques to detect any deviations from the expected behavior. Anomalies are generally perceived in the form of memory leaks, monopolized use of CPU, bandwidth, and other resources . One method to detect anomalies for web‐based applications is to measure the response times.…”

Detecting and Localizing Anomalies in Container Clusters Using Markov Models

“…Du et al [8] use different machine learning techniques to detect anomalous behaviour for microservices with container deployment. Dullmann [10] provides an online performance anomaly detection approach that detects anomalies in performance data based on discrete time series analysis.…”

“…There are many studies regarding detecting anomalies in cloud and edge computing contexts [6,7], and specifically in containerized environments [8][9][10]. Some studies looked at detecting an anomaly's root cause in clouds at a virtual machine [7,11] or network level [12].…”

“…Some studies looked at detecting an anomaly's root cause in clouds at a virtual machine [7,11] or network level [12]. Detecting and localizing anomalies in clustered container deployments [13] is still recognized as a research gap [8,14].…”

A Container Orchestration Development that Optimizes the Etherpad Collaborative Editing Tool through a Novel Management System

“…Qingfeng et al [24] designed a container-based anomaly detection system (ADS) to detect and diagnose anomalies in microservices and monitor and analyze performance data in real-time from them. The proposed ADS consisted of a monitoring module that collected performance data from containers, a data processing module based on machine learning models, and an integrated fault injection module to train these models using machine learning techniques.…”