Tolerating memory latency through software-controlled pre-execution in simultaneous multithreading processors

Abstract

Hardly predictable data addresses in man), irregular applications have rendered prefetching ineffective. In many cases, the only accurate way to predict these addresses is to directly execute the code that generates them. As multithreaded architectures become increasingly popular, one attractive approach is to use idle threads on these machines to perform pre-execution--essentially a combined act of speculative address generation and prefetching-to accelerate the main thread. In this paper, we propose such a pre-execution technique for simultaneous multithreading (SMT) processors. By using software to control pre-execution, we are able to handle some of the most important access patterns that are ~'pically difficult to prefetch. Compared with existing work on pre-execution, our technique is significantly simpler to implement (e.g., no integration of pre-execution results, no need of shortening programs for pre-execution, and no need of special hardware to copy register values upon thread spawns). Consequentl3; only minimal extensions to SMT machines are required to support our technique. Despite its simplicit),, our technique offers an average speedup of 24% in a set of irregular applications, which is a 19% speedup over state-of-the-art software-controlled prefetching.

Template-based memory access engine for accelerators in SoCs

“…Researchers have proposed different techniques to hide the long memory access latency, such as software prefetching [4,5,6,7] and hardware prefetching [8,9,10,11,12]. In software prefetching, the compiler inserts prefetch instructions that bring the indicated block of data onto onchip memory.…”

Dynamic Helper Threaded Prefetching on the Sun UltraSPARC CMP Processor

“…Since each Panther processor encapsulates two identical cores, helper threads can run on the often under-utilized secondary core to prefetch data into the shared cache so as to improve the main thread performance. Unlike other helper threading schemes discussed for SMT [4], [7], [10], [13], [15], [29], thread communication for Panther has to be conducted through the L2 cache since the L2 cache is the closet level of shared caching. On the other hand, construction of helper threads on Panther is less constrained by resource contention as each core is a complete processing unit with its own functional units, TLB, L1 caches and register files; execution of the helper thread would have less negative impact on the main thread performance.…”

“…When the initial load arrives, the processor resumes execution from the checkpointed state. In software pre-execution (also referred to as helper threads or software scouting) [2], [4], [7], [10], [14], [24], [29], [35], a distilled version of the forward slice starting from the missing load is executed, minimizing the utilization of execution resources. Helper threads utilizing run-time compilation techniques may also be effectively deployed on processors that do not have the necessary hardware support for hardware scouting (such as checkpointing and resuming regular execution).…”

A Self-Repairing Prefetcher in an Event-Driven Dynamic Optimization Framework

“…Pre-execution code can be constructed statically [8,15,13] or dynamically [7,23]. Pre-computation code typically runs in a spare hardware context [7,23] or a dedicated hardware engine [17,1,21], in parallel with the main thread.…”

“…This decreases the observed latency, increases memory level parallelism, and allows cache-hit dominated performance even when the working set is larger than the cache. Software based prefetching [4,18,15,31,14,6,11,20] has been shown to be a promising technique to address this issue, and all modern high-performance instruction set architectures provide support for software prefetching.…”