Automating experimentation on distributed testbeds

Wang, Yanyan; Rutherford, Matthew J.; Carzaniga, Antonio; Wolf, Alexander L.

doi:10.1145/1101908.1101934

Cited by 29 publications

(24 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such a description allows to rapidly prototype an experiment and run it on a model of a platform, but is mostly limited to one-time studies. Finally, solutions like Weevil [9] or Plush [10] turn to high-level, declarative descriptions of experiments.…”

Section: ) Experiments Descriptionmentioning

confidence: 99%

“…Solutions similar to Expo [5] offer theoretically unlimited capabilities, but are not modular by design and extending them is difficult. High-level solutions tend to be self-contained and difficult to extend as well, but also offer much more extensive built-in functionality (e.g., Weevil [9] offers a powerful workload generation functionality). Finally, an example of OMF [7] shows that even a platform-dedicated and relatively high-level approach may be extensible, as is shown by its instrumentation, logging and interoperability capabilities.…”

Section: ) Modular Architecturementioning

confidence: 99%

See 1 more Smart Citation

A Workflow-Inspired, Modular and Robust Approach to Experiments in Distributed Systems

Buchert

Nussbaum²,

Gustedt³

2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

Section: ) Experiments Descriptionmentioning

confidence: 99%

Section: ) Modular Architecturementioning

confidence: 99%

A Workflow-Inspired, Modular and Robust Approach to Experiments in Distributed Systems

Buchert

Nussbaum²,

Gustedt³

2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

“…Likewise, XML+XSLT is advocated for code generation tasks in industry as well [16]. One of the closest to our approach is Weevil [8], which is also focus on workload generation and script creation. In fact, later they observed some of the limitation in their approach and proposed four enhancements to explore richer scenarios and to obtain results with greater confidence [5].…”

Section: Related Workmentioning

confidence: 99%

An automated approach to create, store, and analyze large-scale experimental data in clouds

Jayasinghe

Kimball

Choudhary

et al. 2013

2013 IEEE 14th International Conference on Information Reuse &Amp; Integration (IRI)

View full text Add to dashboard Cite

Abstract-The flexibility and scalability of computing clouds make them an attractive application migration target; yet, the cloud remains a black-box for the most part. In particular, their opacity impedes the efficient but necessary testing and tuning prior to moving new applications into the cloud. A natural and presumably unbiased approach to reveal the cloud's complexity is to collect significant performance data by conducting more experimental studies. However, conducting large-scale system experiments is particularly challenging because of the practical difficulties that arise during experimental deployment, configuration, execution and data processing. In this paper we address some of these challenges through Expertus -a flexible automation framework we have developed to create, store and analyze large-scale experimental measurement data. We create performance data by automating the measurement processes for large-scale experimentation, including: the application deployment, configuration, workload execution and data collection processes. We have automated the processing of heterogeneous data as well as the storage of it in a data warehouse, which we have specifically designed for housing measurement data. Finally, we have developed a rich web portal to navigate, statistically analyze and visualize the collected data. Expertus combines template-driven code generation techniques with aspect-oriented programming concepts to generate the necessary resources to fully automate the experiment measurement process. In Expertus, a researcher provides only the high-level description about the experiment, and the framework does everything else. At the end, the researcher can graphically navigate and process the data in the web portal.

show abstract

“…Similarly, Weevil tool [23] is a generic automation tool for the deployment and execution of distributed workloads.…”

Section: Related Workmentioning

confidence: 99%

Performance Test Design Process and Its Implementation Patterns for Multi-services Systems

Din¹,

Schieferdecker²,

Petre³

2008

Testing of Software and Communicating Systems

View full text Add to dashboard Cite

Abstract. Over the past years, the scope of telecommunication services has increased dramatically making network infrastructure-related services a very competitive market. Additionally, the traditional telecoms are combined with Internet technologies for providing a larger range of services. The obvious outcome is the increase of the number of subscribers and services demand. Due to this complexity, the performance testing of continuously evolving telecommunication services has become a real challenge and requires efficient and more powerful testing solutions. This ability highly depends on the performance test design and on the efficient use of hardware resources for test execution.This paper proposes a performance testing methodology which copes with the characteristics mentioned above. The methodology consists of a set of methods and patterns, exemplified for the TTCN-3 language, to realize adequate performance tests for multi-service systems. The effectiveness of this methodology is demonstrated throughout a case study on IP Multimedia Subsystem (IMS) performance testing.

show abstract

Automating experimentation on distributed testbeds

Cited by 29 publications

References 11 publications

A Workflow-Inspired, Modular and Robust Approach to Experiments in Distributed Systems

A Workflow-Inspired, Modular and Robust Approach to Experiments in Distributed Systems

An automated approach to create, store, and analyze large-scale experimental data in clouds

Performance Test Design Process and Its Implementation Patterns for Multi-services Systems

Contact Info

Product

Resources

About