SEED: A Statically Greedy and Dynamically Adaptive Approach for Speculative Loop Execution

Abstract

Abstract-Research on compiler techniques for thread-level loop speculation has so far remained on studying its performance limits: loop candidates that are worthy of parallelization are manually selected by the researchers or based on extensive profiling and pre-execution. It is therefore difficult to include them in a production compiler for speculative multithreaded multicore processors. In a way, existing techniques are statically adaptive ("realized" by the researchers for different inputs) yet dynamically greedy (since all iterations of all selected loop candidates are always parallelized at run time). This paper introduces a SEED (Statically GrEEdy and Dynamically Adaptive) approach for thread-level speculation on loops that is quite different from most other existing techniques. SEED relies on the compiler to select and optimize loop candidates greedily (possibly in an input-independent way) and provides a runtime scheduler to schedule loop iterations adaptively. To select loops for parallelization at run time (subject to program inputs), loop iterations are prioritized in terms of their potential benefits rather than their degree of speculation as in many prior studies. In our current implementation, the benefits of speculative threads are estimated by a simple yet effective cost model. It comprises a mechanism for efficiently tracing the loop nesting structures of the program and a mechanism for predicting the outcome of speculative threads. We have evaluated SEED using a set of SPECint2000 and Olden benchmarks. Compared to existing techniques with a program's loop candidates being ideally selected a priori, SEED can achieve comparable or better performance while aututomating the entire loop candidate selection process.

A Static Greedy and Dynamic Adaptive Thread Spawning Approach for Loop-Level Parallelism

“…When D > L/p, extra thread bubbles will be formed in the pipeline due to the potential of load imbalance, as shown in Fig.4(b). In such case, the parallel overhead will be as follows: To choose a loop to parallelize, the benefit of it is obtained from the difference between the sequential overhead and the estimated parallel overhead shown in (4).…”

“…Our approach devotes all processor cores to one loop at a time, i.e., we focus on the parallelism of the best loop level at each invocation. All spawned threads are observed with one-spawnee restriction, which has been proved to not only fully utilize the potential parallelism of each selected loop, but also maximize the coverage and benefits of the whole region [4] . Thus, all threads from L3 will be executed on all available processors.…”

“…Dynamically determining the right set of loop candidates is more effective and efficient in the TLS execution. Various hardware-based performance prediction [4,15] and dynamic optimization [9,19] are used to reduce the overhead due to the imprecision of static estimation. SEED [4] is similar to our work.…”

“…Loop-level speculation, the most straightforward way to extract each iteration of a loop as a thread, has been extensively studied [4][5][6][7][8] . Due to control and data dependence analysis, we often devote all processor cores to the best loop level at the cost of serializing other less profitable ones.…”

Using the Xeon Phi Platform to Run Speculatively-Parallelized Codes

“…Thread-level speculation (TLS) [8,34,35,42], also called speculative parallelization (SP) [14,18,22,46] or optimistic parallelism [25,26] tries to extract parallelism of loops that can not be considered fully parallel at compile time. TLS optimistically assumes that dependence violations will not occur, launching the parallel execution of the loop.…”