Predicting the performance of big data applications on the cloud

Abstract

Big data analytics have become widespread as a means to extract knowledge from large datasets. Such applications are often characterized by highly heterogeneous and irregular data access patterns, challenging existing software and hardware infrastructures to meet their dynamic resource demands. The cloud computing paradigm, in turn, offers a natural hosting solution to such applications as it provides flexibility and elasticity, adapting the allocated resources in response to the application's current needs. However, these properties impose extra challenge to the accurate performance prediction of cloud-based applications, which is a key step to adequate capacity planning and managing of the hosting infrastructure. In this article, we tackle this challenge by exploring three modeling approaches for predicting the performance of big data applications running on the cloud. We evaluate two queuing-based analytical models and a novel fast ad-hoc simulator in various scenarios based on different applications and infrastructure setups. The considered approaches are compared in terms of prediction accuracy and execution time. Our results indicate that our two best approaches can predict average application execution times with only up to a 7% relative error, on average. Moreover, both of them run very fast (requiring at least two orders of magnitude lower execution time than widely used tools while providing slightly better accuracy), being practical for online prediction.

CLUTCH: A Clustering-Driven Runtime Estimation Scheme for Scientific Simulations

“…Table 1 summarizes major similarities and differences between our work and the existing studies. First, the examined papers concern various application domains: load sharing facility (LSF) [8], parallel program [9], [11], [25], cloud [10], [29], HPC [2], [14], [15], [19], location-based services [20]- [23], databases [26]- [28], big data applications [29], [30], and scientific workloads [12], [13], [16]- [19]. The runtime estimation problem addressed in this paper applies to the scientific workloads domain.…”

“…Second, the target of estimation is slightly different across the studies we have investigated. Most of the existing works [2], [9]- [18], [29], [30] aim to predict the runtime. Some studies [2], [8] attempt to estimate the memory usage.…”

“…Third, tools used for analysis differed across these studies. Some of these [26], [27], [29] developed (pure) analytical models and assess the validity of the model. Many of the existing studies [20]- [24], [28] build neural-net based learning models and utilize them for deriving estimated time; many others [2], [10], [16]- [19], [25], [30] use tree and linear regression based machine learning models.…”

“…Fourth, the availability of hardware-specific parameter data differs from the existing works. The algorithms proposed in some of the studies [2], [12]- [14], [16], [17], [24], [29] require hardware information that may enhance the quality of the runtime estimation, while those proposed in some other studies [15], [18] require no such information.…”

Fixed-Point Iteration Approach to Spark Scalable Performance Modeling and Evaluation

“…The many details of the Spark framework, its scheduler, and the other parameters of the cluster require advanced mathematical models to accurately predict the job completion time of a workload. The three main approaches in existing literature on the topic are simulation [10], [11], machine learning [12], [13], [14], [15], and analytical modeling [16], [17], [18]. Efforts to simulate a Spark cluster have resulted in a number of comprehensive simulators that mimic the actual framework's behavior [10].…”