Survey on memory management techniques in heterogeneous computing systems

Abstract

A major issue faced by data scientists today is how to scale up their processing infrastructure to meet the challenge of big data and high-performance computing (HPC) workloads. With today's HPC domain, it is required to connect multiple graphics processing units (GPUs) to accomplish large-scale parallel computing along with CPUs. Data movement between the processor and on-chip or off-chip memory creates a major bottleneck in overall system performance. The CPU/GPU processes all the data on a computer's memory and hence the speed of the data movement to/from memory and the size of the memory affect computer speed. During memory access by any processing element, the memory management unit (MMU) controls the data flow of the computer's main memory and impacts the system performance and power. Change in dynamic random access memory (DRAM) architecture, integration of memory-centric hardware accelerator in the heterogeneous system and Processing-in-Memory (PIM) are the techniques adopted from all the available shared resource management techniques to maximise the system throughput. This survey study presents an analysis of various DRAM designs and their performances. The authors also focus on the architecture, functionality, and performance of different hardware accelerators and PIM systems to reduce memory access time. Some insights and potential directions toward enhancements to existing techniques are also discussed. The requirement of fast, reconfigurable, self-adaptive memory management schemes in the high-speed processing scenario motivates us to track the trend. An effective MMU handles memory protection, cache control and bus arbitration associated with the processors.

A study on impacts of hardware changes on performance of different multithreading libraries based on Gem5

“…Different multithreading libraries have different features, advantages, and disadvantages. Therefore, choosing an appropriate multithreading library is crucial to high performance [3].…”

Fine-grained address segmentation for attention-based variable-degree prefetching

“…Memory latency is becoming an overwhelming bottleneck in computer performance due to the "memory wall" [4,72] problem, especially with the advent of GPUs [43], TPUs [28], and heterogeneous architectures [17,52] that accelerate computation. Prefetching is critical in reducing program execution time and improving instructions per cycle (IPC) by hiding the latency.…”