Representative Multiprogram Workloads for Multithreaded Processor Simulation

Abstract

Almost all new consumer-grade processors are capable of executing multiple programs simultaneously. The analysis of multiprogrammed workloads for multicore and SMT processors is challenging and time-consuming because there are many possible combinations of benchmarks to execute and each combination may exhibit several different interesting behaviors. Missing particular combinations of program behaviors could hide performance problems with designs. It is thus of utmost importance to have a representative multiprogrammed workload when evaluating multithreaded processor designs.This paper presents a methodology that uses phase analysis, principal components analysis (PCA) and cluster analysis (CA) applied to microarchitectureindependent program characteristics in order to find important program interactions in multiprogrammed workloads. The end result is a small set of co-phases with associated weights that are representative for a multiprogrammed workload across multithreaded processor architectures. Applying our methodology to the SPEC CPU 2000 benchmark suite yields 50 distinct combinations for two-context multithreaded processor simulation that researchers and architects can use for simulation. Each combination is simulated for 50 million instructions, giving a total of 2.5 billion instructions to be simulated for the SPEC CPU2000 benchmark suite. The performance prediction error with these representative combinations is under 2.5% of the real workload for absolute throughput prediction and can be used to make relative throughput comparisons across processor architectures.

Selecting representative benchmark inputs for exploring microprocessor design spaces

“…When looking at multicore processors, special attention should be given regarding how to best select inputs to form representative multiprogram and multithreaded workloads. Prior work has selected multiprogram workloads randomly [Van Craeynest and Eeckhout 2011] or based on microarchitecture-independent characterization of benchmarks [Van Biesbrouck et al 2007]. Extending and/or combining these techniques with techniques for selecting representative inputs is left for future work.…”

Multi-program benchmark definition

“…The process repeats until the end of the sample of one program is reached, which means this approach is a variable instruction count methodology. Later work describes how to use a cluster analysis (somewhat similar to SimPoint [14]) to determine representative co-phases of programs [18].…”

“…To reduce the time required for the experiment, the experimenter may choose to sample the benchmark's execution. Sampling methodology is an active area of research (e.g., [5], [11], [13], [18], [19], [22]), but it is largely orthogonal to the focus of this paper. To sample a benchmark requires first defining the benchmark to be sampled.…”

“…Much prior multi-program work has explored sampling methodologies [5], [11], [13], [18], [19], [22]. This work explores how to select sub-sets of multi-program benchmarks to execute so as to obtain representative results.…”

Chip Multiprocessor Design Space Exploration through Statistical Simulation

“…Van Biesbrouck et al [27], [28], [29] propose the cophase matrix for guiding sampled simultaneous multithreading (SMT) processor simulation running multiprogram workloads. The idea of the cophase matrix is to keep track of the Fig.…”