Improving latency tolerance of multithreading through decoupling

Abstract

AbstractÐThe increasing hardware complexity of dynamically scheduled superscalar processors may compromise the scalability of this organization to make an efficient use of future increases in transistor budget. SMT processors, designed over a superscalar core, are therefore directly concerned by this problem. This work presents and evaluates a novel processor microarchitecture which combines two paradigms: simultaneous multithreading and access/execute decoupling. Since its decoupled units issue instructions inorder, this architecture is significantly less complex, in terms of critical path delays, than a centralized out-of-order design, and it is more effective for future growth in issue-width and clock speed. We investigate how both techniques complement each other. Since decoupling features an excellent memory latency hiding efficiency, the large amount of parallelism exploited by multithreading may be used to hide the latency of functional units and keep them fully utilized. Our study shows that, by adding decoupling to a multithreaded architecture, fewer threads are needed to achieve maximum throughput. Therefore, in addition to the obvious hardware complexity reduction, it places lower demands on the memory system. Since one of the problems of multithreading is the degradation of the memory system performance, both in terms of miss latency and bandwidth requirements, this improvement becomes critical for high miss latencies, where bandwidth might become a bottleneck. Finally, although it may seem rather surprising, our study reveals that multithreading by itself exhibits little memory latency tolerance. Our results suggest that most of the latency hiding effectiveness of SMT architectures comes from the dynamic scheduling. On the other hand, decoupling is very effective at hiding memory latency. An increase in the cache miss penalty from 1 to 32 cycles reduces the performance of a 4-context multithreaded decoupled processor by less than 2 percent. For the nondecoupled multithreaded processor, the loss of performance is about 23 percent.

Energy-efficient mobile GPU systems

“…Overall, they show that decoupled access/execute is an effective energy saving technique in many-core architectures as it requires simple hardware and, in addition, the number of thread contexts required to keep the functional units busy can be significantly reduced. It is worth noting that similar conclusions about the synergy of decoupling and multithreading were already suggested in [122].…”

“…This small amount of threads allows the architecture to hide the latency of the functional units and keep them busy, which is an issue that the access/execute decoupling mechanism does not address. It is worth noting that similar conclusions about the synergy of decoupling and multithreading were already suggested in [122]. Figure 3.14 shows the normalized energy obtained by the different memory latency tolerance schemes.…”

Hybrid latency tolerance for robust energy-efficiency on 1000-core data parallel processors

“…IBM's POWER6 architecture implements multithreading and a restricted form of hardware scout, called load-lookahead prefetching [12]. Other work proposes dynamic instruction partitioning into separate threads and leverages out-oforder hardware in order to provide memory latency tolerance [19]. Additionally, many designs have implemented hardware prefetching.…”

Outrider

“…Other contemporary decoupling work involves hardware partitioning and SMT [19]. In this work, the authors propose hardware partitioning of integer and floating-point instructions into separate threads in order to provide memory latency tolerance using large instruction queues to hold dependent floating-point instructions while they wait for the miss to return.…”