Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

Gioiosa, Roberto; Kestor, Gokcen; Cicotti, Pietro; Laure, Erwin

doi:10.1109/ipdpsw.2017.115

Cited by 28 publications

(18 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the performance of Stable-NUMA did not improve significantly over Linux at graph500 and CG workload execution on every configuration. This was because the memory access pattern of graph500 has the characteristics of poor temporal and spatial locality [62]. Actually, by using the Linux perf [63] tool, we measured that the local memory access ratio at Server A with four NUMA nodes is nearly 25%.…”

Section: Discussionmentioning

confidence: 99%

A Performance-Stable NUMA Management Scheme for Linux-Based HPC Systems

et al. 2021

View full text Add to dashboard Cite

Linux is becoming the de-facto standard operating system for today's high-performance computing (HPC) systems because it can satisfy the demands of many HPC systems for rich operating system (OS) features. However, owing to features intended for the general-purpose OS, Linux has many OS noise sources such as page faults or thread migrations that can result in the unstable performance of HPC application. Furthermore, in the case of the non-uniform memory access (NUMA) architecture, which has different memory access latencies to local and remote memory nodes, the performance stability of the application can be more exacerbated by the OS noise. In this paper, we address the OS noise caused by Linux in the NUMA architecture and propose a novel performance-stable NUMA management scheme called Stable-NUMA. Stable-NUMA comprises three techniques for improving performance stability: two-level thread clustering, state-based page placement, and selective page profiling. Our proposed Stable-NUMA scheme significantly alleviates OS noise and enhances the local memory access ratio of the NUMA system as compared to Linux. We implemented Stable-NUMA in Linux and experimented with various HPC workloads. The evaluation results demonstrated that Stable-NUMA outperforms Linux with and without its NUMA-aware feature by up to 25% in terms of average performance and 73% in terms of performance stability.

show abstract

Section: Discussionmentioning

confidence: 99%

A Performance-Stable NUMA Management Scheme for Linux-Based HPC Systems

et al. 2021

View full text Add to dashboard Cite

show abstract

“…al. [18]. In contrast, for PageRank the difference between the two memory technologies becomes apparent at 64 threads, and the performance gap is almost 2x for 256 threads.…”

Section: A Mcdram Versus Dram On the Knlmentioning

confidence: 94%

Performance Impact of Memory Channels on Sparse and Irregular Algorithms

Green

Fox

Young

et al. 2019

2019 IEEE/ACM 9th Workshop on Irregular Applications: Architectures and Algorithms (IA3)

View full text Add to dashboard Cite

Graph processing is typically considered to be a memory-bound rather than compute-bound problem. One common line of thought is that more available memory bandwidth corresponds to better graph processing performance. However, in this work we demonstrate that the key factor in the utilization of the memory system for graph algorithms is not necessarily the raw bandwidth or even the latency of memory requests. Instead, we show that performance is proportional to the number of memory channels available to handle small data transfers with limited spatial locality.Using several widely used graph frameworks, including Gunrock (on the GPU) and GAPBS & Ligra (for CPUs), we evaluate key graph analytics kernels using two unique memory hierarchies, DDR-based and HBM/MCDRAM. Our results show that the differences in the peak bandwidths of several Pascal-generation GPU memory subsystems aren't reflected in the performance of various analytics. Furthermore, our experiments on CPU and Xeon Phi systems demonstrate that the number of memory channels utilized can be a decisive factor in performance across several different applications. For CPU systems with smaller thread counts, the memory channels can be underutilized while systems with high thread counts can oversaturate the memory subsystem, which leads to limited performance. Finally, we model the potential performance improvements of adding more memory channels with narrower access widths than are found in current platforms, and we analyze performance trade-offs for the two most prominent types of memory accesses found in graph algorithms, streaming and random accesses.

show abstract

“…An important trend is the addition of wider buses and asynchronous protocols for data movement in the form of NVMe and also the support for highbandwidth memory (HBM) [74]. HBM requires architecture changes, which are not backwards compatible with older hardware, but will bring significant benefit to bandwidth-bound applications [73]. With 3D NAND, the capacity of SSDs will improve further, which might eventually replace HDDs in data centers [41].…”

Section: Technologiesmentioning

confidence: 99%

Survey of Storage Systems for High-Performance Computing

Lüttgau

Kuhn

Duwe

et al. 2018

JSFI

View full text Add to dashboard Cite

In current supercomputers, storage is typically provided by parallel distributed file systems for hot data and tape archives for cold data. These file systems are often compatible with local file systems due to their use of the POSIX interface and semantics, which eases development and debugging because applications can easily run both on workstations and supercomputers. There is a wide variety of file systems to choose from, each tuned for different use cases and implementing different optimizations. However, the overall application performance is often held back by I/O bottlenecks due to insufficient performance of file systems or I/O libraries for highly parallel workloads. Performance problems are dealt with using novel storage hardware technologies as well as alternative I/O semantics and interfaces. These approaches have to be integrated into the storage stack seamlessly to make them convenient to use. Upcoming storage systems abandon the traditional POSIX interface and semantics in favor of alternative concepts such as object and key-value storage; moreover, they heavily rely on technologies such as NVM and burst buffers to improve performance. Additional tiers of storage hardware will increase the importance of hierarchical storage management. Many of these changes will be disruptive and require application developers to rethink their approaches to data management and I/O. A thorough understanding of today's storage infrastructures, including their strengths and weaknesses, is crucially important for designing and implementing scalable storage systems suitable for demands of exascale computing.

show abstract

Exploring the Performance Benefit of Hybrid Memory System on HPC Environments

Cited by 28 publications

References 12 publications

A Performance-Stable NUMA Management Scheme for Linux-Based HPC Systems

A Performance-Stable NUMA Management Scheme for Linux-Based HPC Systems

Performance Impact of Memory Channels on Sparse and Irregular Algorithms

Survey of Storage Systems for High-Performance Computing

Contact Info

Product

Resources

About