Experience with using the Parallel Workloads Archive

Abstract

Science is based upon observation. The scientific study of complex computer systems should therefore be based on observation of how they are used in practice, as opposed to how they are assumed to be used or how they were designed to be used. In particular, detailed workload logs from real computer systems are invaluable for research on performance evaluation and for designing new systems.Regrettably, workload data may suffer from quality issues that might distort the study results, just as scientific observations in other fields may suffer from measurement errors. The cumulative experience with the Parallel Workloads Archive, a repository of job-level usage data from large-scale parallel supercomputers, clusters, and grids, has exposed many such issues. Importantly, these issues were not anticipated when the data was collected, and uncovering them was not trivial. As the data in this archive is used in hundreds of studies, it is necessary to describe and debate procedures that may be used to improve its data quality. Specifically, we consider issues like missing data, inconsistent data, erroneous data, system configuration changes during the logging period, and unrepresentative user behavior. Some of these may be countered by filtering out the problematic data items. In other cases, being cognizant of the problems may affect the decision of which datasets to use. While grounded in the specific domain of parallel jobs, our findings and suggested procedures can also inform similar situations in other domains.

Workload Modeling for Computer Systems Performance Evaluation

“…For example, consider the instantaneous utilization data at several large parallel machines shown in Figure 2.19, namely the fraction of processors that are actively serving some job at each instant; it is calculated by tracking the start and end time of each job, and summing the processors used by jobs that have started but have not yet terminated. Naturally, this should be bounded by the number of processors in the system, but the data seems to indicate that in many instances more than 100% of the processors were allocated [250]. However, such exceptions are not easily attributable to any specific subset of jobs, so it is not clear which jobs should be removed to fix the problem.…”

“…and experience with using it. For example, it is important to share information about problems in the data, and any data cleaning that was performed [250].…”

“…Systems may always become overloaded for certain periods due to fluctuations in the load [599,250]. )…”

A Periodic Portfolio Scheduler for Scientific Computing in the Data Center

“…The workloads we consider in this work match the cluster-based traces of the Parallel Workloads Archive [20]. We further assume that jobs are CPU-bound and their runtime depends linearly on the speed of the (virtual) processor where they are executed.…”

“…The synthetic workloads are short-term but with significantly different job arrival patterns, allowing us to better characterize the impact of the arrival process on portfolio scheduling. The real workload is a whole trace from the Parallel Workloads Archive [20] and allows us to gain valuable insight into the operation of our portfolio scheduler in realistic conditions. Synthetic Workloads: In this paper, we generate five types of workloads that have different user behaviors but the same (real) job run times.…”