Multi-Core Cache Hierarchies

Balasubramonian, Rajeev; Jouppi, Norman P.

doi:10.1007/978-3-031-01734-6

Cited by 23 publications

(4 citation statements)

References 228 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper focuses on explicitly managed memory hierarchies, where copies across the hierarchy are handled by DMAEs. We refer the interested reader to [6], [7], [8], [9] for excellent surveys on cache-based memory systems. Caches and DMAEs often coexist in modern computing systems as they address different application needs.…”

Section: Introductionmentioning

confidence: 99%

A High-Performance, Energy-Efficient Modular DMA Engine Architecture

Benz,

Rogenmoser,

Scheffler

et al. 2024

IEEE Trans. Comput.

View full text Add to dashboard Cite

Data transfers are essential in today's computing systems as latency and complex memory access patterns are increasingly challenging to manage. Direct memory access engines (DMAEs) are critically needed to transfer data independently of the processing elements, hiding latency and achieving high throughput even for complex access patterns to high-latency memory. With the prevalence of heterogeneous systems, DMAEs must operate efficiently in increasingly diverse environments. This work proposes a modular and highly configurable open-source DMAE architecture called intelligent DMA (iDMA), split into three parts that can be composed and customized independently. The front-end implements the control plane binding to the surrounding system. The mid-end accelerates complex data transfer patterns such as multi-dimensional transfers, scattering, or gathering. The back-end interfaces with the on-chip communication fabric (data plane). We assess the efficiency of iDMA in various instantiations: In high-performance systems, we achieve speedups of up to 15.8× with only 1 % additional area compared to a base system without a DMAE. We achieve an area reduction of 10 % while improving ML inference performance by 23 % in ultra-low-energy edge AI systems over an existing DMAE solution. We provide area, timing, latency, and performance characterization to guide its instantiation in various systems.

show abstract

Section: Introductionmentioning

confidence: 99%

A High-Performance, Energy-Efficient Modular DMA Engine Architecture

Benz,

Rogenmoser,

Scheffler

et al. 2024

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…At the same time, private caches are still needed to keep the average latency low, which leads to cache hierarchy and the use of some coherence protocol, usually involving some degree of cache inclusion. The cache hierarchy design of a multicore CPU is, therefore, a problem with many variables [Balasubramonian, Jouppi, Muralimanohar, 2011].…”

Section: Introductionmentioning

confidence: 99%

Reducing miss rate in a non-inclusive cache with inclusive directory of a chip multiprocessor

Nedbailo¹,

Surchenko²,

Bychkov³

2023

CRM

View full text Add to dashboard Cite

Although the era of exponential performance growth in computer chips has ended, processor core numbers have reached 16 or more even in general-purpose desktop CPUs. As DRAM throughput is unable to keep pace with this computing power growth, CPU designers need to find ways of lowering memory traffic per instruction. The straightforward way to do this is to reduce the miss rate of the last-level cache. Assuming "non-inclusive cache, inclusive directory" (NCID) scheme already implemented, three ways of reducing the cache miss rate further were studied.The first is to achieve more uniform usage of cache banks and sets by employing hash-based interleaving and indexing. In the experiments in SPEC CPU2017 refrate tests, even the simplest XOR-based hash functions demonstrated a performance increase of 3.2 %, 9.1 %, and 8.2 % for CPU configurations with 16, 32, and 64 cores and last-level cache banks, comparable to the results of more complex matrix-, division-and CRC-based functions.The second optimisation is aimed at reducing replication at different cache levels by means of automatically switching to the exclusive scheme when it appears optimal. A known scheme of this type, FLEXclusion, was modified for use in NCID caches and showed an average performance gain of 3.8 %, 5.4 %, and 7.9 % for 16-, 32-, and 64-core configurations.The third optimisation is to increase the effective cache capacity using compression. The compression rate of the inexpensive and fast BDI*-HL (Base-Delta-Immediate Modified, Half-Line) algorithm, designed for NCID, was measured, and the respective increase in cache capacity yielded roughly 1 % of the average performance increase.All three optimisations can be combined and demonstrated a performance gain of 7.7 %, 16 % and 19 % for CPU configurations with 16, 32, and 64 cores and banks, respectively.

show abstract

“…Nowadays, chip multiprocessor (CMP) systems dominate the market in high-performance servers, desktop or embedded systems, and mobile devices [3]. Their most common design includes a multilevel cache hierarchy, ending with a shared last-level cache (SLLC) [4]. Looking specifically at the current top 10 supercomputers [5], all but one of them include general-purpose CMPs that have a multilevel cache: ARM A64FX [6], IBM Power9 [7], AMD EPYC [8] and Intel Xeon [9].…”

Section: Introductionmentioning

confidence: 99%

“…Fig 4. Mean SLLC miss rate for step 1 of NOPTb-miss (labelled as iteration 0) and several iterations of step 3.…”

mentioning

confidence: 99%

Near-optimal replacement policies for shared caches in multicore processors

et al. 2021

View full text Add to dashboard Cite

An optimal replacement policy that minimizes the miss rate in a private cache was proposed several decades ago. It requires knowing the future access sequence the cache will receive. There is no equivalent for shared caches because replacement decisions alter this future sequence. We present a novel near-optimal policy for minimizing the miss rate in a shared cache that approaches the optimal execution iteratively. During each iteration, the future access sequence is reconstructed on every miss interleaving the future per-core sequences, taken from the previous iteration. This single sequence feeds a classical private-cache optimum replacement policy. Our evaluation on a shared last-level cache shows that our proposal iteratively converges to a near-optimal miss rate that is independent of the initial conditions, within a margin of 0.1%. The best state-of-the-art online policies achieve around 65% of the miss rate reduction obtained by our near-optimal proposal. In a shared cache, miss rate optimization does not imply the optimization of other metrics. Therefore, we also propose a new near-optimal policy to maximize fairness between cores. The best state-ofthe-art online policy achieves 60% of the improvement in fairness seen with our near-optimal policy. Our proposals are useful both for setting upper performance bounds and inspiring implementable mechanisms for shared caches.

show abstract

Multi-Core Cache Hierarchies

Cited by 23 publications

References 228 publications

A High-Performance, Energy-Efficient Modular DMA Engine Architecture

A High-Performance, Energy-Efficient Modular DMA Engine Architecture

Reducing miss rate in a non-inclusive cache with inclusive directory of a chip multiprocessor

Near-optimal replacement policies for shared caches in multicore processors

Contact Info

Product

Resources

About