A Control Approach for Performance of Big Data Systems

Berekmeri, Mihaly; Serrano, Damián; Bouchenak, Sara; Marchand, Nicolas; Robu, Bogdan

doi:10.3182/20140824-6-za-1003.01319

Cited by 20 publications

(26 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, no such details are provided in the case of [72]. In contrast, the authors of [43,83] used a PI feedback controller for big data application. They focused to adjusts the computing nodes of a map reduce cluster to guarantee the desired service time of map reduce jobs.…”

Section: Classicmentioning

confidence: 99%

“…A solution can be centralized at one of the following three levels, i.e., Application, Node or Cloud. The solutions that focused on horizontal elasticity from the SP perspective are centralized at Application level (e.g., [43,44,48,50,61,72]), whereas the control solutions that cater CPs perspective runs centrally at Cloud level, where they could be responsible for different applications (e.g., [33,48,86]). The application level control solutions can be executed outside of the cloud environment and therefore they can control interactions with multiple control.…”

Section: Architecturementioning

confidence: 99%

“…The control solutions having the objective of disturbance rejection often assist another control solution (e.g., [40][41][42][43]). Irrespective of these types, the key objective of any elastic control solution is to improve the utilisation of computational resources whilst maintaining acceptable level of performance of the system and reducing its operational cost.…”

Section: Control Objectivementioning

confidence: 99%

“…Apart from Response time and Throughput, the use of some other performance metrics is also observed. This include Service time for data oriented applications [43,72], Job progress for scientific application [73] and Read operation latency for storage application [32,74].…”

Section: Reference Inputmentioning

confidence: 99%

See 3 more Smart Citations

A control theoretical view of cloud elasticity: taxonomy, survey and challenges

Ullah

Shen

et al. 2018

Cluster Comput

View full text Add to dashboard Cite

The lucrative features of cloud computing such as pay-as-you-go pricing model and dynamic resource provisioning (elasticity) attract clients to host their applications over the cloud to save up-front capital expenditure and to reduce the operational cost of the system. However, the efficient management of hired computational resources is a challenging task. Over the last decade, researchers and practitioners made use of various techniques to propose new methods to address cloud elasticity. Amongst many such techniques, control theory emerges as one of the popular methods to implement elasticity. A plethora of research has been undertaken on cloud elasticity including several review papers that summarise various aspects of elasticity. However, the scope of the existing review articles is broad and focused mostly on the highlevel view of the overall research works rather than on the specific details of a particular implementation technique. While considering the importance, suitability and abundance of control theoretical approaches, this paper is a step forward towards a stand-alone review of control theoretic aspects of cloud elasticity. This paper provides a detailed taxonomy comprising of relevant attributes defining the following two perspectives, i.e., control-theory as an implementation technique as well as cloud elasticity as a target application domain. We carry out an exhaustive review of the literature by classifying the existing elasticity solutions using the attributes of control theoretic perspective. The summarized results are further presented by clustering them with respect to the type of control solutions, thus helping in comparison of the related control solutions. In last, a discussion summarizing the pros and cons of each type of control solutions are presented. This discussion is followed by the detail description of various open research challenges in the field.

show abstract

Section: Classicmentioning

confidence: 99%

Section: Architecturementioning

confidence: 99%

Section: Control Objectivementioning

confidence: 99%

Section: Reference Inputmentioning

confidence: 99%

See 2 more Smart Citations

A control theoretical view of cloud elasticity: taxonomy, survey and challenges

Ullah

Shen

et al. 2018

Cluster Comput

View full text Add to dashboard Cite

show abstract

“…Considering the scale of these workloads and the uncertainties of their execution environments, this can quickly lead to a waste of resources since the static configuration does not adapt to the current runtime condition. Optimizing Hadoop execution has therefore attracted a lot of research attention resulting in a number of different approaches in particular in the domain of selfadaptive software systems [1,2,4,5,6,7,8,9]. However, this research effort is often hindered by the accidental complexity of setting representative Hadoop deployment in different distributed environments and comparing different approaches.…”

Section: Introductionmentioning

confidence: 99%

Hadoop-Benchmark: Rapid Prototyping and Evaluation of Self-Adaptive Behaviors in Hadoop Clusters

Zhang¹,

Křikava²,

Rouvoy³

et al. 2017

2017 IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)

View full text Add to dashboard Cite

Optimizing Hadoop executions has attracted a lot of research contributions in particular in the domain of selfadaptive software systems. However, these research efforts are often hindered by the complexity of Hadoop operation and the difficulty to reproduce experimental evaluations that makes it hard to compare different approaches to one another.To address this limitation, we propose a research acceleration platform for rapid prototyping and evaluation of self-adaptive behavior in Hadoop clusters. Essentially, it provides automated approach to provision reproducible Hadoop environments and execute acknowledged benchmarks. It is based on the stateof-the-art container technology that supports both distributed configurations as well as standalone single-host setups. We demonstrate the approach on a complete implementation of a concrete Hadoop self-adaptive case study.

show abstract