Interconnect design considerations for large NUCA caches

Abstract

The ever increasing sizes of on-chip caches and the growing domination of wire delay necessitate significant changes to cache hierarchy design methodologies. Many recent proposals advocate splitting the cache into a large number of banks and employing a network-on-chip (NoC) to allow fast access to nearby banks (referred to as Non-Uniform Cache Architectures -NUCA). Most studies on NUCA organizations have assumed a generic NoC and focused on logical policies for cache block placement, movement, and search. Since wire/router delay and power are major limiting factors in modern processors, this work focuses on interconnect design and its influence on NUCA performance and power. We extend the widely-used CACTI cache modeling tool to take network design parameters into account. With these overheads appropriately accounted for, the optimal cache organization is typically very different from that assumed in prior NUCA studies. To alleviate the interconnect delay bottleneck, we propose novel cache access optimizations that introduce heterogeneity within the inter-bank network. The careful consideration of interconnect choices for a large cache results in a 51% performance improvement over a baseline generic NoC and the introduction of heterogeneity within the network yields an additional 11-15% performance improvement.

Energy Behaviour of NUCA Caches in CMPs

“…The energy model is presented only for single core systems, however some works [23][24] [25] have proposed the use of that model also for CMP NUCA systems, showing the relevance of the need for a CMP NUCA energy model. In [21] a model for NUCA interbank network elements is devised.…”

“…Various previous works have shown that NUCA-based systems are effective in hiding delay effects on performance [14][15] [16][17] [18], and several works have shown interest in energy aspects of such architectures. In [19] [20] techniques are proposed dealing with the reduction of power consumption, while in [21], a model for NUCA interbank network elements is devised. In [22], an energy model is defined for a single core NUCA based architecture, taking into account both static and dynamic contributions of NUCA caches.…”

Optimization techniques of on-chip memory system based on UltraSPARC architecture

“…Increasing capacity of cache and complicated interconnection organization cause a new symptom that wire delay may affect the timing characteristic of accessing cache bank. Delay depends on the distance between requesting core and requested cache bank, which is called Non-Uniform Cache Access(NUCA) [5]. Such syndrome becomes more intractable as the capacity of on-chip cache is increasing and on-chip interconnection is more complicated.…”

Ring data location prediction scheme for Non-Uniform Cache Architectures

“…In a NUCA-based system, the L2 space is organized as a set of banks connected to each other using an on-chip network [3], [4], [5], [6], [7]. Each of these banks can be accessed independently, with access latency dependent on the physical distance of the bank from the CPU.…”