A similarity measure for time, frequency, and dependencies in large-scale workloads

Abstract

Performance evaluations of large-scale systems require the use of representative workloads with certifiable similar or dissimilar characteristics. To quantify the similarity of the characteristics, we describe a novel measure comprising two efficient methods that are suitable for large-scale workloads. One method uses the discrete wavelet transform to assess the periodic time and frequency characteristics in the workload. The second method evaluates dependencies in descriptive attributes via association rule learning. Both methods are evaluated to find the limits of their similarity spaces. Additionally, the wavelet method is evaluated against existing similarity methods and tested for noise robustness and random bias. An empirical study using workloads from seven operational large-scale systems evaluates the measure's accuracy. The results show that our measure is highly resistant to noise, well-suited for large-scale workloads, covers 87% of the possible similarity space, and improves accuracy by 24.5% and standard deviation by 10.8% when compared to existing work.

ATLAS grid workload on NDGF resources: Analysis, modeling, and workload generation

“…For example, the traces from PWA are used as an input to simulation in [20][21][22]. The workloads from GWA are used for testing the dependency searching methodology proposed in [23], and for developing workload models based on user activity in [24]. The data from GO serves as the basis for developing a job re-submission model in [25].…”

FAST: Near Real-Time Searchable Data Analytics for the Cloud

“…The key idea behind FAST is to explore and exploit the correlation property within and among datasets via improved correlation-aware hashing [11] and flat-structured addressing [12] to significantly reduce the processing latency of parallel queries, while incurring acceptably small loss of accuracy. The approximate scheme for real-time performance has been widely recognized in system design [13]- [16] and high-end computing [17]- [19]. In essence, FAST goes beyond the simple combination of existing techniques to offer efficient data analytics via significantly increased processing speed.…”