NoM: Network-on-Memory for Inter-Bank Data Transfer in Highly-Banked Memories

Rezaei, Seyyed Hossein SeyyedAghaei; Modarressi, Mehdi; Ausavarungnirun, Rachata; Sadrosadati, Mohammad; Mutlu, Onur; Daneshtalab, Masoud

doi:10.1109/lca.2020.2990599

Cited by 27 publications

(15 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This key takeaway shows that the workloads most well-suited for the UPMEM PIM architecture are those with little or no inter-DPU communication. Based on this takeaway, we recommend that the hardware architecture and the software stack be enhanced with support for inter-DPU communication (e.g., by leveraging new in-DRAM data copy techniques [39,207,218,219,222,223,245] and providing better connectivity inside DRAM [39,207]).…”

Section: Key Takeawaymentioning

confidence: 99%

Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture

Gómez-Luna¹,

Hajj²,

Fernández³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of main memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM).Recent research explores different forms of PIM architectures, motivated by the emergence of new 3Dstacked memory technologies that integrate memory with a logic layer where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. The UPMEM PIM architecture combines traditional DRAM memory arrays with general-purpose in-order cores, called DRAM Processing Units (DPUs), integrated in the same chip.This paper provides the first comprehensive analysis of the first publicly-available real-world PIM architecture. We make two key contributions. First, we conduct an experimental characterization of the UPMEM-based PIM system using microbenchmarks to assess various architecture limits such as compute throughput and memory bandwidth, yielding new insights. Second, we present PrIM (Processing-In-Memory benchmarks), a benchmark suite of 16 workloads from different application domains (e.g., dense/sparse linear algebra, databases, data analytics, graph processing, neural networks, bioinformatics, image processing), which we identify as memory-bound. We evaluate the performance and scaling characteristics of PrIM benchmarks on the UPMEM PIM architecture, and compare their performance and energy consumption to their stateof-the-art CPU and GPU counterparts. Our extensive evaluation conducted on two real UPMEM-based PIM systems with 640 and 2,556 DPUs provides new insights about suitability of different workloads to the PIM system, programming recommendations for software designers, and suggestions and hints for hardware and architecture designers of future PIM systems. CCS Concepts: • Hardware → Dynamic memory; • Computing methodologies → Model development and analysis; • Computer systems organization → Architectures.

show abstract

Section: Key Takeawaymentioning

confidence: 99%

Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture

Gómez-Luna¹,

Hajj²,

Fernández³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, heterogeneous architectures in modern SoCs can stress the interconnect through their imbalanced loads that are a consequence of largely different demands across many different types of applications and accelerators. Relatively new technology such as chiplets [119,120] or new types of memory [121][122][123] can create demands for efficient interconnection network designs that connect multiple memory nodes together. Especially processing-inmemory systems can require well-connected memory arrays via efficient interconnects to tightly couple computation and communication.…”

Section: Conclusion and Future Outlookmentioning

confidence: 99%

Energy-Efficient Deflection-based On-chip Networks: Topology, Routing, Flow Control

Ausavarungnirun¹,

Mutlu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

As the number of cores scales to tens and hundreds, the energy consumption of routers across various types of on-chip networks in chip muiltiprocessors (CMPs) increases significantly. A major source of this energy consumption comes from the input buffers inside Network-on-Chip (NoC) routers, which are traditionally designed to maximize performance. To mitigate this high energy cost, many works propose bufferless router designs that utilize deflection routing to resolve port contention. While this approach is able to maintain high performance relative to its buffered counterparts at low network traffic, the bufferless router design suffers performance degradation under high network load.In order to maintain high performance and energy efficiency under both low and high network loads, this chapter discusses critical drawbacks of traditional bufferless designs and describes recent research works focusing on two major modifications to improve the overall performance of the traditional bufferless network-on-chip design. The first modification is a minimally-buffered design that introduces limited buffering inside critical parts of the on-chip network in order to reduce the number of deflections. The second modification is a hierarchical bufferless interconnect design that aims to further improve performance by limiting the number of hops each packet needs to travel while in the network. In both approaches, we discuss design tradeoffs and provide evaluation results based on common CMP configurations with various network topologies to show the effectiveness of each proposal.

show abstract

“…This key takeaway shows that the workloads most well-suited for the UPMEM PIM architecture are those with little or no inter-DPU communication. Based on this takeaway, we recommend that the hardware architecture and the software stack be enhanced with support for inter-DPU communication (e.g., by leveraging new in-DRAM data copy techniques [117,121,125,126] and providing better connectivity inside DRAM [121,125]).…”

Section: Key Takeawaymentioning

confidence: 99%

Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware

Gómez-Luna¹,

Hajj²,

Fernandez³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Many modern workloads such as neural network inference and graph processing are fundamentally memorybound. For such workloads, data movement between memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM).Recent research explores different forms of PIM architectures, motivated by the emergence of new technologies that integrate memory with a logic layer, where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. The UPMEM PIM architecture combines traditional DRAM memory arrays with general-purpose in-order cores, called DRAM Processing Units (DPUs), integrated in the same chip.This paper presents key takeaways from the first comprehensive analysis [1] of the first publicly-available real-world PIM architecture. First, we introduce our experimental characterization of the UPMEM PIM architecture using microbenchmarks, and present PrIM (Processing-In-Memory benchmarks), a benchmark suite of 16 workloads from different application domains (e.g., dense/sparse linear algebra, databases, data analytics, graph processing, neural networks, bioinformatics, image processing), which we identify as memory-bound. Second, we provide four key takeaways about the UPMEM PIM architecture, which stem from our study of the performance and scaling characteristics of PrIM benchmarks on the UPMEM PIM architecture, and their performance and energy consumption comparison to their state-of-the-art CPU and GPU counterparts. More insights about suitability of different workloads to the PIM system, programming recommendations for software designers, and suggestions and hints for hardware and architecture designers of future PIM systems are available in [1].

show abstract

NoM: Network-on-Memory for Inter-Bank Data Transfer in Highly-Banked Memories

Cited by 27 publications

References 11 publications

Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture

Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture

Energy-Efficient Deflection-based On-chip Networks: Topology, Routing, Flow Control

Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware

Contact Info

Product

Resources

About