Automated Workload Characterization in Cloud-based Transactional Data Grids

Abstract

Abstract-Cloud computing represents a cost-effective paradigm to deploy a wide class of large-scale distributed applications, for which the pay-per-use model combined with automatic resource provisioning promise to reduce the cost of dependability and scalability. However, a key challenge to be addressed to materialize the advantages promised by Cloud computing is the design of effective auto-scaling and self-tuning mechanisms capable of ensuring pre-determined QoS levels at minimum cost in face of changing workload conditions. This is one of the keys goals that are being pursued by the Cloud-TM project, a recent EU project that is developing a novel, self-optimizing transactional data platform for the cloud. In this paper we present the key design choices underlying the development of Cloud-TM's Workload Analyzer (WA), a crucial component of the Cloud-TM platform that is change of three key functionalities: aggregating, filtering and correlating the streams of statistical data gathered from the various nodes of the Cloud-TM platform; building detailed workload profiles of applications deployed on the Cloud-TM platform, characterizing their present and future demands in terms of both logical (i.e. data) and physical (e.g. hardware-related) resources; triggering alerts in presence of violations (or risks of future violations) of predetermined SLAs. I. INTRODUCTIONThe Cloud Computing paradigm is profoundly changing both the architectures of the IT systems and the organization of the enterprise IT infrastructure management. Architectures of distributed computing platforms are moving from a traditional static model, where the amount of resources allocated to applications/services are a-priori estimated, towards an elastic model, where resources can be provisioned on-demand. The flexibility of Cloud computing's pay-per-use model is driving many enterprises to move their IT infrastructure and services to the "cloud". Anyway this new paradigm opens up new challenges.One of these is related to the design of effective autoscaling and self-tuning mechanisms capable of ensuring predetermined QoS levels at minimum costs in face of changing workload conditions. In fact, on one hand, an elastic platform provides a very cost-effective model to improve system performance and dependability by means of data and services replication. On the other hand, in order to take advantage of the elasticity of the underlying infrastructure, both the data and services replication management need to be automated by means of mechanisms able to guarantee the desirable Quality of Service (QoS) levels while minimizing the operational cost of the infrastructure. In such a context,

Risk-based framework for SLA violation abatement from the cloud service provider’s perspective

“…In this paper, we explain the process for determining and abating SLA violation, which forms part of the post-interaction phase. It is important to mention that various approaches in the literature have used techniques such as QoS prediction [29], workflow detection control model [40], machine learning regression technique [42], and workload analyzer [50] to ascertain the possibility of SLA violation.…”

Disaggregating Non-Volatile Memory for Throughput-Oriented Genomics Workloads

“…SMUFIN has been implemented to maximize sequential writes to the devices, and this behavior has been verified by analyzing its access pattern. A block trace sample of requested blocks to the device was generated using Linux's blktrace, and the trace was then fed to the algorithm provided by [3] to calculate the percentage of sequential write accesses. This method identified 88% of sequential writes after adapting the algorithm to consider accesses in which the final address matched the initial address of many immediately following requests, thus accounting for file appends.…”

Self-Tuning Transactional Data Grids: The Cloud-TM Approach

“…This information is exploited, in its turn, by the AdM, which can react triggering corrective actions aimed at altering the scale and/or configuration of the Data Platform. More details about the architecture of the WPM and the workload characterization can be found in [19], [20].…”

“…Finally, the WA integrates workload and resource demand prediction schemes: the WA includes algorithms for time-series forecasting (e.g., based on Kalman filter or on polynomial regression [19]), which allow predicting future workload's trends and allow the AdM to enact proactive self-tuning schemes. This functionality represents a fundamental building block for any proactive adaptation scheme, i.e., schemes triggering reconfigurations of the platform anticipating imminent workloads' changes, which are particularly desirable in case the platform's reconfiguration (as in the case of elastic scaling) can have non-negligible latencies.…”