ProRenaTa: Proactive and Reactive Tuning to Scale a Distributed Storage System

Liu, Ying; Rameshan, Navaneeth; Monte, Enric; Vlassov, Vladimir; Navarro, Leandro

doi:10.1109/ccgrid.2015.26

Cited by 13 publications

(21 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Service Level Agreement (SLA) cannot be guaranteed on reactive policies since the actions are only activated after recording an observation, while SLA are more likely to be achieved in proactive policies which predict workload needs in advance [65]. The robust of proactive policies are associated with the quality of the historic data and the accuracy of the corresponding model.…”

Section: Policy-based Efficient Storage Systemsmentioning

confidence: 99%

“…For example, IF <Condition: Storage available capacity = X > AND < Goal desired = capacity > T HEN < PolicyAction: Compressions = TRUE > is a reactive policy. Typically, SLA cannot be guaranteed on reactive policies, since the actions are only activated after recording an observation, while an SLA is more likely to be achieved in proactive policies that predict workload needs an advance [65]. It is impotent to be able to design robust proactive policies that can achieve desired goals.…”

Section: Policy-based Storage Managementmentioning

confidence: 99%

See 1 more Smart Citation

Transfer Learning Based Performance Modeling And Effective Storage Management In Big Data Ecosystems

Alsayoud¹

2021

Preprint

View full text Add to dashboard Cite

Big data ecosystems contain a mix of sophisticated hardware storage components to support heterogeneous workloads. Storage components and the workloads interact and affect each other; therefore, their relationship has to consider when modeling workloads or managing storage. Efficient workload modeling guides optimal storage management decisions, and the right decisions help guarantee the workload’s needs. The first part of this thesis focuses on workload modeling efficiency, and the second part focuses on cost-effective storage management.<div>Workload performance modeling is an essential step in management decisions. The standard modeling approach constructs the model based on a historical dataset collected from one set of setups (scenario). The standard modeling approach requires the model to be reconstructed from scratch with every time the setups changes. To address this issue, we propose a cross-scenario modeling approach that improves the workload’s performance classification accuracy by up to 78% through adopting the Transfer Learning (TL).<br></div><div>The storage system is the most crucial component of the big data ecosystem, where the workload’s execution process starts by fetching data from it and ends by storing data into it. Thus, the workload’s performance is directly affected by storage capability. To provide a high I/O performance in the ecosystems, Solid State Drive (SSD) are utilized as a tier or as a cache on big data distributed ecosystems. SSDs have a short lifespan that is affected by data size and the number of writing operations. Balancing performance requirements and SSD’s lifespan consumption is never easy, and it’s even harder when interacting with a huge amount of data and with heterogeneous I/O patterns. In this thesis, we analysis big data workloads I/O pattern impacts on SSD’s lifespan when SSD is used as a tier or as a cache. Then, we design a Hidden Markov Model (HMM) based I/O pattern controller that manages workload placement and guarantees cost-effective storage that enhances the workload performance by up to 60%, and improves SSD’s lifespan by up to 40%. </div><div>The designed transfer learning modeling approach and the storage management solutions improve workload modeling accuracy, and the quality of the storage management policies while the testing setup changes.<br></div>

show abstract

Section: Policy-based Efficient Storage Systemsmentioning

confidence: 99%

Section: Policy-based Storage Managementmentioning

confidence: 99%

Transfer Learning Based Performance Modeling And Effective Storage Management In Big Data Ecosystems

Alsayoud¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Although we found this isolation acceptable in terms of security and privacy, we identified resource isolation and admission control as an area of improvement to ensure the quality of the services provided (see Section 2). We did research and proposed solutions [28,29,53] for performance isolation, including resource prioritisation and pinning, elastic resource provisioning, reactive tuning, and admission control, but any of these are included yet in our software. Given the wide range of essential and optional software and services (IaaS, PaaS, and SaaS) for a decentralised CC, Cloudy can have a similar role in terms of standardisation and unification at the cloud infrastructure and service level as the guifi.net website has had at the network infrastructure level.…”

Section: Cloudy Software For User Devicesmentioning

confidence: 99%

Cloudy in guifi.net: Establishing and sustaining a community cloud as open commons

Baig¹,

Freitag²,

Navarro³

2018

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is non-trivial to build an accurate and efficient elasticity controller. Recent works have been focusing on improving the accuracy of elasticity controllers by building different control models with various monitored/controlled metrics [1][2][3][4][5][6][7][8]. However, none of the works have considered the practical usefulness of an elasticity controller, which involves the following challenges.…”

Section: Introductionmentioning

confidence: 99%

“…Additionally, a generic workload prediction module is also integrated to facilitate the decision making of OnlineElast-Man. It allows OnlineElastMan to scale the storage system well in advance to prevent SLO violations caused on workload increase and scaling overhead [7]. Specifically, the prediction module aggregates multiple prediction algorithms and chooses the most appropriate prediction algorithm based on the current workload pattern using a weight majority selection algorithm.…”

Section: Introductionmentioning

confidence: 99%

OnlineElastMan: Self-Trained Proactive Elasticity Manager for Cloud-Based Storage Services

Liu

Gureya

Al-Shishtawy

et al. 2016

2016 International Conference on Cloud and Autonomic Computing (ICCAC)

Self Cite

View full text Add to dashboard Cite

The pay-as-you-go pricing model and the illusion of unlimited resources in the Cloud initiate the idea to provision services elastically. Elastic provisioning of services allocates/de-allocates resources dynamically in response to the changes of the workload. It minimizes the service provisioning cost while maintaining the desired service level objectives (SLOs). Model-predictive control is often used in building such elasticity controllers that dynamically provision resources. However, they need to be trained, either online or offline, before making accurate scaling decisions.The training process involves tedious and significant amount of work as well as some expertise, especially when the model has many dimensions and the training granularity is fine, which is proved to be essential in order to build an accurate elasticity controller. In this paper, we present OnlineElast-Man, which is a self-trained proactive elasticity manager for cloud-based storage services. It automatically evolves itself while serving the workload. Experiments using Onli-neElastMan with Cassandra indicate that OnlineElastMan continuously improves its provision accuracy, i.e., minimizing provisioning cost and SLO violations, under various workload patterns.

show abstract

ProRenaTa: Proactive and Reactive Tuning to Scale a Distributed Storage System

Cited by 13 publications

References 15 publications

Transfer Learning Based Performance Modeling And Effective Storage Management In Big Data Ecosystems

Transfer Learning Based Performance Modeling And Effective Storage Management In Big Data Ecosystems

Cloudy in guifi.net: Establishing and sustaining a community cloud as open commons

OnlineElastMan: Self-Trained Proactive Elasticity Manager for Cloud-Based Storage Services

Contact Info

Product

Resources

About