Challenges of memory management on modern NUMA systems

Gaud, Fabien; Lepers, Baptiste; Funston, Justin; Dashti, Mohammad; Fedorova, Alexandra; Quéma, Vivien; Lachaize, Renaud; Roth, Mark A.

doi:10.1145/2814328

Cited by 39 publications

(27 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performance Degradation through automatic NUMA Balancing NUMA Balancing is known to cause page thrashing if multiple NUMA nodes access the same pages in an alternating fashion [4]. NUMA Balancing then migrates pages back and forth between these nodes.…”

Section: Stencil Aware Page Placement and Work Distribution For Numa mentioning

confidence: 99%

“…NUMA aware program optimizations that address this problem are at the extremes of a spectrum. At one end are generic NUMA page placement (PP) schemes, such as First-Touch, Interleaved, and the Linux automatic NUMA Balancing feature which are known to exhibit pathological behavior in hard to predict situations [3,4]. At the other end of the spectrum are application specific memory optimizations such as shared variable privatization.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

NUMA Optimizations for Algorithmic Skeletons

Metzger

Cole

Fensch

2018

Euro-Par 2018: Parallel Processing

View full text Add to dashboard Cite

To address NUMA performance anomalies, programmers often resort to application specific optimizations that are not transferable to other programs, or to generic optimizations that do not perform well in all cases. Skeleton based programming models allow NUMA optimizations to be abstracted on a pattern-by-pattern basis, freeing programmers from this complexity. As a case study, we investigate computations that can be implemented with stencil skeletons. We present an analysis of the behavior of a range of simple and complex stencil programs from the NAS and Rodinia benchmark suites, under state-ofthe-art NUMA aware page placement (PP) schemes. We show that even though an application (or skeleton) may have implemented the correct, intuitive scheduling of data and work to threads, the resulting performance can be disrupted by an inappropriate PP scheme. In contrast, we show that a NUMA PP-aware stencil implementation scheme can achieve speed ups of up to 12x over a similar scheme which uses the Linux default PP, and that this works across a set of complex stencil applications. Furthermore, we show that a supposed PP performance optimization in the Linux kernel never improves and in some cases degrades stencil performance by up to 0.27x and should therefore be deactivated by stencil skeleton implementations. Finally, we show that further speed ups of up to 1.1x can be achieved by addressing a work imbalance issue caused by poor conventional understanding of NUMA PP.

show abstract

Section: Stencil Aware Page Placement and Work Distribution For Numa mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

NUMA Optimizations for Algorithmic Skeletons

Metzger

Cole

Fensch

2018

Euro-Par 2018: Parallel Processing

View full text Add to dashboard Cite

show abstract

“…Second, the pages are periodically unmapped from a process address space and, upon the next page fault, migrated to the requesting node. Carrefour [4,8] is another recent tool that consists of a memory-placement algorithm for NUMA systems that focuses on traffic management. As in our approach, Carrefour focuses on memory congestion as the primary source of performance loss in current NUMA systems.…”

Section: Related Workmentioning

confidence: 99%

A performance comparison of data and memory allocation strategies for sequence aligners on NUMA architectures

Lenis

Senar

2017

Cluster Comput

View full text Add to dashboard Cite

Over the last several years, many sequence alignment tools have appeared and become popular for the fast evolution of next generation sequencing technologies. Obviously, researchers that use such tools are interested in getting maximum performance when they execute them in modern infrastructures. Today's NUMA (Non-uniform memory access) architectures present major challenges in getting such applications to achieve good scalability as more processors/cores are used. The memory system in NUMA systems shows a high complexity and may be the main cause for the loss of an application's performance. The existence of several memory banks in NUMA systems implies a logical increase in latency associated with the accesses of a given processor to a remote bank. This phenomenon is usually attenuated by the application of strategies that tend to increase the locality of memory accesses. However, NUMA systems may also suffer from contention problems that can occur when concurrent accesses are concentrated on a reduced number of banks. Sequence alignment tools use large data structures to contain reference genomes to which all reads are aligned. Therefore, these tools are very sensitive to performance problems related to the memory system. The main goal of this study is to explore the trade-offs between data locality and data dispersion in NUMA systems. We have performed experiments with several popular sequence alignment tools on two This work has been supported by MINECO-Spain under contract TIN2014-53234-C2-1-R. widely available NUMA systems to assess the performance of different memory allocation policies and data partitioning strategies. We find that there is not one method that is best in all cases. However, we conclude that memory interleaving is the memory allocation strategy that provides the best performance when a large number of processors and memory banks are used. In the case of data partitioning, the best results are usually obtained when the number of partitions used is greater, sometimes combined with an interleave policy.

show abstract

“…Each processor consists of one or several groups of processor cores, each of which is physically associated with one or more memory controllers and memory devices. Such a group of processor cores is referred to as a NUMA node [13,16,25]. Although the NUMA nodes are generally connected by high-speed interconnect links such as QuickPath Interconnect (QPI) [32] and HyperTransport [25], accessing a remote NUMA node still needs a longer latency than accessing the data of the local memory device.…”

Section: Introductionmentioning

confidence: 99%

“…However, only considering the locality is not sufficient to improve performance on modern NUMA systems. Furthermore, maximizing the locality can degrade performance because it potentially increases the memory congestion [2,16].…”

Section: Introductionmentioning

confidence: 99%

Online MPI Process Mapping for Coordinating Locality and Memory Congestion on NUMA Systems

Agung

Amrizal

Egawa

et al. 2020

JSFI

View full text Add to dashboard Cite

Mapping MPI processes to processor cores, called process mapping, is crucial to achieving the scalable performance on multi-core processors. By analyzing the communication behavior among MPI processes, process mapping can improve the communication locality, and thus reduce the overall communication cost. However, on modern non-uniform memory access (NUMA) systems, the memory congestion problem could degrade performance more severely than the locality problem because heavy congestion on shared caches and memory controllers could cause long latencies. Most of the existing work focus only on improving the locality or rely on offline profiling to analyze the communication behavior. We propose a process mapping method that dynamically performs the process mapping for adapting to communication behaviors while coordinating the locality and memory congestion. Our method works online during the execution of an MPI application. It does not require modifications to the application, previous knowledge of the communication behavior, or changes to the hardware and operating system. Experimental results show that our method can achieve performance and energy efficiency close to the best static mapping method with low overhead to the application execution. In experiments with the NAS parallel benchmarks on a NUMA system, the performance and total energy improvements are up to 34% (18.5% on average) and 28.9% (13.6% on average), respectively. In experiments with two GROMACS applications on a larger NUMA system, the average improvements in performance and total energy consumption are 21.6% and 12.6%, respectively.

show abstract

Challenges of memory management on modern NUMA systems

Abstract: Optimizing NUMA systems applications with Carrefour.

Cited by 39 publications

References 3 publications

NUMA Optimizations for Algorithmic Skeletons

NUMA Optimizations for Algorithmic Skeletons

A performance comparison of data and memory allocation strategies for sequence aligners on NUMA architectures

Online MPI Process Mapping for Coordinating Locality and Memory Congestion on NUMA Systems

Contact Info

Product

Resources

About