Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems

Noorshams, Qais; Busch, Axel; Rentschler, Andreas; Bruhn, Dominik; Kounev, Samuel; Tůma, Petr; Reussner, Ralf

doi:10.1109/icdcsw.2014.26

Cited by 7 publications

(6 citation statements)

References 13 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The third class of solutions proposes an application-based solution, in [17] and [41] the authors propose to avoid I/O interference by coordinating the applications I/O requests. Noorshames et al [48] present an approach for modeling I/O interference that is the most closely related to ours. However, the authors focused on HDDs and did not consider SSD-related performance models.…”

Section: Rmentioning

confidence: 80%

“…At the system level, some studies [3], [63] have attempted to modify the I/O scheduler and the Linux cgroup I/O throttling policy. Finally, at the application level, in [48], the authors presented an approach for modeling I/O interference that is closely related to ours. However, the authors focused on HDDs and did not consider SSDs and their specific I/O interferences.…”

Section: Imentioning

confidence: 94%

“…One needs to consider the variables that have an influence on the I/O performance for the learning algorithms to find the (hidden) relationships between x and y (see Section II-C). We have selected 9 features listed below based on [48], [66], [71]:…”

Section: Multivariate Adaptive Regression Splines (Mars)mentioning

confidence: 99%

See 2 more Smart Citations

Investigating Machine Learning Algorithms for Modeling SSD I/O Performance for Container-Based Virtualization

Dartois

Boukhobza

Knefati

et al. 2021

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

One of the cornerstones of the cloud provider business is to reduce hardware resources cost by maximizing their utilization. This is done through smartly sharing processor, memory, network and storage, while fully satisfying SLOs negotiated with customers. For the storage part, while SSDs are increasingly deployed in data centers mainly for their performance and energy efficiency, their internal mechanisms may cause a dramatic SLO violation. In effect, we measured that I/O interference may induce a 10x performance drop. We are building a framework based on autonomic computing which aims to achieve intelligent container placement on storage systems by preventing bad I/O interference scenarios. One prerequisite to such a framework is to design SSD performance models that take into account interactions between running processes/containers, the operating system and the SSD. These interactions are complex. In this paper, we investigate the use of machine learning for building such models in a container based Cloud environment. We have investigated five popular machine learning algorithms along with six different I/O intensive applications and benchmarks. We analyzed the prediction accuracy, the learning curve, the feature importance and the training time of the tested algorithms on four different SSD models. Beyond describing modeling component of our framework, this paper aims to provide insights for cloud providers to implement SLO compliant container placement algorithms on SSDs. Our machine learning-based framework succeeded in modeling I/O interference with a median Normalized Root-Mean-Square Error (NRMSE) of 2.5%.

show abstract

Section: Rmentioning

confidence: 80%

Section: Imentioning

confidence: 94%

See 1 more Smart Citation

Investigating Machine Learning Algorithms for Modeling SSD I/O Performance for Container-Based Virtualization

Dartois

Boukhobza

Knefati

et al. 2021

IEEE Trans. Cloud Comput.

View full text Add to dashboard Cite

show abstract

“…SPA can automatically explore and quantitatively evaluate both workload-specific and system-specific performance influences. This information can in turn be used in an automated process for statistical analysis and regression-based performance modeling [3,5,6]. These models can be used to analyze the system behavior and predict the performance in different scenarios.…”

Section: Application Scenariosmentioning

confidence: 99%

“…The Storage Performance Analyzer (SPA) [1] is an approach for the systematic analysis of I/O performance in virtualized environments, which has been successfully applied in our previous work for performance measuring, monitoring, and modeling [2][3][4][5][6][7][8]. As illustrated in Figure 1, our SPA framework basically consists of a benchmark harness that coordinates and controls the execution of benchmarks as well as monitors and a tailored analysis library used to process and evaluate the collected measurements.…”

Section: The Spa Approachmentioning

confidence: 99%

The Storage Performance Analyzer

Noorshams

Busch

Kounev

et al. 2015

Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

The ever-increasing I/O resource demands pose significant challenges for today's system environments to meet performance requirements. The resource demand effects are even magnified in modern virtualized environments where workloads are consolidated to save hardware and operating costs. Tool-supported analysis approaches can help to understand I/O performance characteristics and avoid I/O performance and interference issues. In this demo paper, we present the Storage Performance Analyzer (SPA) -a tool for automated I/O performance analysis. SPA is equipped with tailored features for virtualized environments allowing to measure, monitor, and model both I/O performance and interference effects in modern environments. SPA is open-source and available for common operating systems.

show abstract

Weight‐based strategy for an I/O‐intensive application at a cloud data center

Peng

Liu

Wang

et al. 2018

Concurrency and Computation

View full text Add to dashboard Cite

Applications with different characteristics in the cloud may have different resources preferences.However, traditional resource allocation and scheduling strategies rarely take into account the characteristics of applications. Considering that an I/O-intensive application is a typical type of application and that frequent I/O accesses, especially small files randomly accessing the disk, may lead to an inefficient use of resources and reduce the quality of service (QoS) of applications, a weight allocation strategy is proposed based on the available resources that a physical server can provide as well as the characteristics of the applications. Using the weight obtained, a resource allocation and scheduling strategy is presented based on the specific application characteristics in the data center. Extensive experiments show that the strategy is correct and can guarantee a high concurrency of I/O per second (IOPS) in a cloud data center with high QoS. Additionally, the strategy can efficiently improve the utilization of the disk and resources of the data center without affecting the service quality of applications. KEYWORDS IOPS, process scheduling, random I/O, small files, weight INTRODUCTIONIn recent years, cloud computing has been widely used because of its advantages, such as its high scalability, low energy-consumption, and on-demand provision of virtual resources. Currently, companies such as Google, IBM, Amazon, Microsoft, and Alibaba are the leaders in the development and application of cloud computing technology and services. 1 As a new business mode, cloud computing provides computing, storage, and network services to Internet users by integrating resources. As a result, users conveniently make use of their applications with much lower purchase and maintenance costs. Additionally, cloud computing can greatly improve the resource utilization efficiency of society. Therefore, many enterprises as well as personal users have moved their applications to the cloud environment. In cloud computing, the data center is the key feature, as it owns a large amount of the resource pool that provides powerful resources and serves all types of applications.However, although the resources in a cloud data center are enormous, they are limited. Different consumptions of resources might considerably affect the performance and service capability of a cloud data center. Traditionally, resources are generally allocated based on a virtual machine (VM).Applications are executed on the virtual machine without considering their characteristics or features. Actually, different types of applications have different characteristics and consume different types and amounts of resources during their execution. These resource requirement difference among applications leads to serious problems for cloud resource allocations and task schedule strategies. For example, some types of resources may have been highly overloaded while other types of resources are barely consumed in different virtual machines or by different applications. This occur...

show abstract

Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems

Cited by 7 publications

References 13 publications

Investigating Machine Learning Algorithms for Modeling SSD I/O Performance for Container-Based Virtualization

Investigating Machine Learning Algorithms for Modeling SSD I/O Performance for Container-Based Virtualization

The Storage Performance Analyzer

Weight‐based strategy for an I/O‐intensive application at a cloud data center

Contact Info

Product

Resources

About