Enhancing Inter-Node Process Migration for Load Balancing on Linux-Based NUMA Multicore Systems

Chiang, Mei‐Ling; Tu, Shu-Wei; Su, Wei-Lun; Lin, Chen-Wei

doi:10.1109/compsac.2018.10264

Cited by 6 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From the Linux kernel source code, the kernel scheduler always migrates the first‐fit task in the busiest core's runqueue that can be migrated to the target core in the load balancing procedure. Our preliminary study demonstrated that migrating the task with minimum memory usage is better than migrating the first‐fit task. This study takes further steps to gather memory usage information on each node for each task at runtime.…”

Section: Kmlb Mechanismmentioning

confidence: 91%

“…Four policies with different considerations are proposed in this study. For comparison, another two existent policies, including the default policy and the Total Min (TM) policy, are also presented in this section. (a) Default method …”

Section: Memory‐aware Selection Policies For Internode Task Migrationmentioning

confidence: 99%

“…To reduce remote memory access and/or resource contention, existing studies use various strategies. Most of them focus on either the scheduling techniques and thread mapping strategies or memory placement and data mapping policies.…”

Section: Introductionmentioning

confidence: 99%

“…There are many related studies on the issues of thread mapping and data mapping to NUMA nodes, and their strategies substantially improve performance and energy efficiency. However, lacking the consideration of an OS kernel's internode load‐balancing activity will reduce the effect on performance improvement.…”

Section: Introductionmentioning

confidence: 99%

“…However, sometimes, the first‐fit policy may happen to migrate the less appropriate tasks across nodes. Due to the high cost of remote memory access along with page fault handling and subsequent page migration, selecting suitable tasks for internode migration is critical, as presented in our previous preliminary study . It is because tasks may have different characteristics, such as memory access behavior, and their memory allocated on multiple nodes also differ, so their chances of accessing remote memory after internode migration are not the same.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Memory‐aware kernel mechanism and policies for improving internode load balancing on NUMA systems

Chiang

et al. 2019

Softw Pract Exp

View full text Add to dashboard Cite

Although nonuniform memory access architecture provides better scalability for multicore systems, cores accessing memory on remote nodes take longer than those accessing on local nodes. Remote memory access accompanied by contention for internode interconnection degrades performance. Properly mapping threads to cores and data accessed to their nodes can substantially improve performance and energy efficiency. However, an operating system kernel's load-balancing activity may migrate threads across nodes, which thus messes up the thread mapping. Besides, subsequent data mapping behavior pays for the cost of page migration to reduce remote memory access. Once unsuitable threads are migrated, it is detrimental to system performance. This paper focuses on improving the kernel's internode load balancing on nonuniform memory access systems. We develop a memory-aware kernel mechanism and policies to reduce remote memory access incurred by internode thread migration. The Linux kernel's load balancing mechanism is modified to incorporate selection policies in the internode thread migration, and the kernel is modified to track the amount of memory used by each thread on each node. With this information, well-designed policies can then choose suitable threads for internode migration. The purpose is to avoid migrating a thread that might incur relatively more remote memory access and page migration. The experimental results show that with our mechanism and the proposed selection policies, the system performance is substantially increased when compared with the unmodified Linux kernel that does not consider memory usage and always migrates the first-fit thread in the runqueue that can be migrated to the target central processing unit.

show abstract

Section: Kmlb Mechanismmentioning

confidence: 91%

Section: Memory‐aware Selection Policies For Internode Task Migrationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Memory‐aware kernel mechanism and policies for improving internode load balancing on NUMA systems

Chiang

et al. 2019

Softw Pract Exp

View full text Add to dashboard Cite

show abstract

LBMA and IMAR²: Weighted lottery based migration strategies for NUMA multiprocessing servers

Laso

Lorenzo

Rivera

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Multicore NUMA systems present on‐board memory hierarchies and communication networks that influence performance when executing shared memory parallel codes. Characterizing this influence is complex, and understanding the effect of particular hardware configurations on different codes is of paramount importance. In this article, monitoring information extracted from hardware counters at runtime is used to characterize the behavior of each thread for an arbitrary number of multithreaded processes running in a multiprocessing environment. This characterization is given in terms of number of operations per second, operational intensity, and latency of memory accesses. We propose a runtime tool, executed in user space, that uses this information to guide two different thread migration strategies for improving execution efficiency by increasing locality and affinity without requiring any modification in the running codes. Different configurations of NAS Parallel OpenMP benchmarks running concurrently on multicore NUMA systems were used to validate the benefits of our proposal, in which up to four processes are running simultaneously. In more than the 95% of the executions of our tool, results outperform those of the operating system (OS) and produces up to 38% improvement in execution time over the OS for heterogeneous workloads, under different and realistic locality and affinity scenarios.

show abstract

Characterization of SPEC2006 Benchmarks Under Multicore Platform to Identify Critical Architectural Aspects

Shukla¹,

Pant²

2023

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Enhancing Inter-Node Process Migration for Load Balancing on Linux-Based NUMA Multicore Systems

Cited by 6 publications

References 12 publications

Memory‐aware kernel mechanism and policies for improving internode load balancing on NUMA systems

Memory‐aware kernel mechanism and policies for improving internode load balancing on NUMA systems

LBMA and IMAR²: Weighted lottery based migration strategies for NUMA multiprocessing servers

Characterization of SPEC2006 Benchmarks Under Multicore Platform to Identify Critical Architectural Aspects

Contact Info

Product

Resources

About

Enhancing Inter-Node Process Migration for Load Balancing on Linux-Based NUMA Multicore Systems

Cited by 6 publications

References 12 publications

Memory‐aware kernel mechanism and policies for improving internode load balancing on NUMA systems

Memory‐aware kernel mechanism and policies for improving internode load balancing on NUMA systems

LBMA and IMAR2: Weighted lottery based migration strategies for NUMA multiprocessing servers

Characterization of SPEC2006 Benchmarks Under Multicore Platform to Identify Critical Architectural Aspects

Contact Info

Product

Resources

About

LBMA and IMAR²: Weighted lottery based migration strategies for NUMA multiprocessing servers