Improving the Testing of Clustered Systems Through the Effective Usage of Java Benchmarks

Portillo-Dominguez, A. Omar; Ayala-Rivera, Vanessa

doi:10.1109/conisoft.2017.00023

Cited by 3 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The AUTs used were DaCapo 4 and SPECJVM 5 , which are two well-known Java benchmarks that are highly used in the literature. In order to utilize them as web applications, we used a servlet-based tool that enables their access and execution via web, so that each program (within the benchmark) can be used as a different functional transaction [28]. The Dacapo test plan consisted of its 14 benchmark programs (shown in Table 1) configured to run using their small size, 1 iteration, and 2 threads.…”

Section: Methodsmentioning

confidence: 99%

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Huerta-Guevara

Ayala-Rivera

Murphy

et al. 2019

Testing Software and Systems

Self Cite

View full text Add to dashboard Cite

Performance testing is a critical task to ensure an acceptable user experience with software systems, especially when there are high numbers of concurrent users. Selecting an appropriate test workload is a challenging and time-consuming process that relies heavily on the testers' expertise. Not only are workloads application-dependent, but also it is usually unclear how large a workload must be to expose any performance issues that exist in an application. Previous research has proposed to dynamically adapt the test workloads in real-time based on the application behavior. By reducing the need for the trial-and-error test cycles required when using static workloads, dynamic workload adaptation can reduce the effort and expertise needed to carry out performance testing. However, such approaches usually require testers to properly configure several parameters in order to be effective in identifying workloaddependent performance bugs, which may hinder their usability among practitioners. To address this issue, this paper examines the different criteria needed to conduct performance testing efficiently using dynamic workload adaptation. We present the results of comprehensively evaluating one such approach, providing insights into how to tune it properly in order to obtain better outcomes based on different scenarios. We also study the effects of varying its configuration and how this can affect the results obtained.

show abstract

Section: Methodsmentioning

confidence: 99%

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Huerta-Guevara

Ayala-Rivera

Murphy

et al. 2019

Testing Software and Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, frequent Major GCs in any microservices indicate potential memory leaks or inadequate memory allocation to the JVM which deteriorates the overall application performance by interrupting the connected programs. Previous studies have aimed at developing engines to automatically produce benchmarks for capturing the intricacies in Major GC behaviour of java-based systems [10,11,12]. The major challenge experienced by the researchers is to have an ideal load balancing strategy or Java benchmark while testing Major GC related scenarios due to the non-deterministic nature of Major GC which tends to influence the configuration complexity of the application [13,14,15,16].…”

Section: Related Workmentioning

confidence: 99%

Aiops Framework for Alerting Performance Issues in Microservices Using Time Series Forecasting

Singh¹,

Shyamsundar²

2022

IJEAST

View full text Add to dashboard Cite

For large scale applications based on Microservices architecture and hosted on scalable platforms like Pivotal Cloud Foundry or OpenShift, it is imperative to monitor the performance of microservices on a continuous basis and trigger performance critical alerts upfront when abnormal patterns are observed in the immediate past. One of the critical application performance indicators is Major Garbage Collection (Major GC), an increase of which causes application performance bottlenecks and might indicate insufficient memory allocation to the JVM or a potential memory leak. In this paper, we discuss a solution built using machine learning Algorithms that forecasts Major GC cycles (Number of Major Garbage Collection) of business critical microservices for the near future time periods. This paper discusses various algorithms within python’s stats models. tsa package that were explored and details the concepts and results of forecast models trained using Seasonal Autoregressive Integrated Moving Average, or SARIMA algorithm that was finally considered for forecasting based on model performance. The paper also elucidates the Forecast accuracy of the model, showing results of the forecast vs actual occurrences of Major Garbage collection. The objective of this solution based on AIOps framework is to enable development and production support teams get proactive notification when forecast Major GC patterns breach the set threshold limits, so that corrective action can be taken to improve the application performance by reducing frequency of Major GC cycles.

show abstract

“…For instance, the DaCapo and SPECJVM suites (sponsored by companies such as IBM and Microsoft) are composed of 24 different real-world applications and specialised programs [15], [16]. Meanwhile, the authors of [17] proposed an approach to build clustered test environments by leveraging Java benchmarks. Finally, [18] presented an approach to generate GC benchmarks.…”

Section: Background and Related Workmentioning

confidence: 99%

Improving the Testing of Java Garbage Collection Through an Efficient Benchmark Generation

Portillo-Dominguez

Ayala-Rivera

2018

2018 6th International Conference in Software Engineering Research and Innovation (CONISOFT)

Self Cite

View full text Add to dashboard Cite

Garbage Collection (GC) is a core feature of multiple modern technologies (e.g., Java, Android). On one hand, it offers significant software engineering benefits over explicitly memory management, like preventing most types of memory leaks. On the other hand, GC is a known cause of performance degradation. However, it is considerably challenging to understand its exact impact on the overall application performance. This is because the non-deterministic nature of GC makes very complex to properly model it and evaluate its performance impacts. To help tackling these problems, we present an engine to generate realistic GC benchmarks by enabling to effectively capture the GC/memory behaviours experienced by real-world Java applications. We also demonstrate, through a comprehensive experimental evaluation, how such benchmarks can be useful to strengthen the evaluation of GC-related advancements.

show abstract

Improving the Testing of Clustered Systems Through the Effective Usage of Java Benchmarks

Cited by 3 publications

References 22 publications

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Towards an Efficient Performance Testing Through Dynamic Workload Adaptation

Aiops Framework for Alerting Performance Issues in Microservices Using Time Series Forecasting

Improving the Testing of Java Garbage Collection Through an Efficient Benchmark Generation

Contact Info

Product

Resources

About