Performance analysis model for big data applications in cloud computing

Abstract

The foundation of Cloud Computing is sharing computing resources dynamically allocated and released per demand with minimal management effort. Most of the time, computing resources such as processors, memory and storage are allocated through commodity hardware virtualization, which distinguish cloud computing from others technologies. One of the objectives of this technology is processing and storing very large amounts of data, which are also referred to as Big Data. Sometimes, anomalies and defects found in the Cloud platforms affect the performance of Big Data Applications resulting in degradation of the Cloud performance. One of the challenges in Big Data is how to analyze the performance of Big Data Applications in order to determine the main factors that affect the quality of them. The performance analysis results are very important because they help to detect the source of the degradation of the applications as well as Cloud. Furthermore, such results can be used in future resource planning stages, at the time of design of Service Level Agreements or simply to improve the applications. This paper proposes a performance analysis model for Big Data Applications, which integrates software quality concepts from ISO 25010. The main goal of this work is to fill the gap that exists between quantitative (numerical) representation of quality concepts of software engineering and the measurement of performance of Big Data Applications. For this, it is proposed the use of statistical methods to establish relationships between extracted performance measures from Big Data Applications, Cloud Computing platforms and the software engineering quality concepts.

Testing MapReduce programs: A systematic mapping study

“…The execution time of MapReduce programs can also be obtained using the modeling language proposed by Barbierato et al The tester can also monitor the execution of MapReduce programs and test cases, obtaining charts to evaluate performance and potential bottlenecks . Villalpando et al propose a model for the Big Data application establishing a relationship between performance and reliability measures based on the international standard of quality ISO/IEC 25010…”

Context-aware data quality assessment for big data

“…Other approaches focus on performance optimization of map reduce jobs. In [24], the performance of big data applications are analyzed in order to detect the main factors that affect their quality, with the aim of detecting the source of the degradation of the applications as well as the limitations of the infrastructure hosting it. In [25], the application non functional requirements are considered by proposing a new cloud service for scaling the infrastructure to meet them.…”

A System for Evaluating Big Data Analytics Performance

“…This suggests the need for considering both efficiency and effectiveness in measuring the performance of big data systems. Efficiency is related to relate to system's availability and performance over efforts, whereby effectiveness is concerned with the impact of information on assisting users on performing their work (Heo&Haan, 2000).Therefore resource utilization, time related metrics and capacity (Villalpando et al, 2014), (Brunnert et al, 2014), as well as throughput, response time, latency (Onyeabor&Ta'a, 2018)can be regarded as regarded performance measures of big data systemswhereas measures, such as satisfaction, timeliness usefulness and result representation are used to scrutinize system's success from user perspective. It means individual and organizational objectives are attained along with system's objectives.…”