Nimble Page Management for Tiered Memory Systems

Abstract

Software-controlled heterogeneous memory systems have the potential to increase the performance and cost efficiency of computing systems. However they can only deliver on this promise if supported by efficient page management policies and mechanisms within the operating system (OS). Current OS implementations do not support efficient tiering of data between heterogeneous memories. Instead, they rely on expensive offlining of memory or swapping data to disk as a means of profiling and migrating hot or cold data between memory nodes. They also leave numerous optimizations on the table; for example, multi-threaded hardware is not leveraged to maximize page migration throughput, resulting in up to 95% under-utilization of available memory bandwidth. To remedy these shortcomings, we propose and implement a general purpose OS-integrated multi-level memory management system that reuses current OS page tracking structures to tier pages directly between memories with no additional monitoring overhead. We augment this system with four additional optimizations: native support for transparent huge page migration, multi-threaded migration of a page, concurrent migration of multiple pages, and symmetric exchange of pages. Combined, these optimizations dramatically reduce kernel software overheads and improve raw page migration throughput over 15×. Implemented in Linux and evaluated on x86, Power, and ARM64 systems, our OS support for heterogeneous memories improves application performance 40% over baseline Linux for a suite of real-world memory-intensive workloads utilizing a multilevel disaggregated memory system.

Adaptive Page Migration Policy With Huge Pages in Tiered Memory Systems

“…The memory bandwidth of the slow memory node is throttled and saturated to emulate the slow memory. AMP can improve the performance of selected applications by 23.8% compared to the LRU lists adopted in the prior work [13]. Furthermore, AMP can achieve 10.9, 6.4, 17.6% higher performance than LRU, LFU, and Random, respectively.…”

“…(i) Our investigation first shows that none of the policies excels the others for the range of applications, since each application has a different memory access behavior preferring a different migration policy. However, the page migration policies used by recent tiered memory studies are fixed to a single policy, such as a variant of LRU lists in Linux systems [13]. (ii) If a proper migration policy for each application is used, our investigation shows that huge pages can be effectively used for migration in tiered memory systems, reaping the benefit of efficient address translation.…”

“…While huge pages can drastically improve the address translation efficiency, their interaction with migration policies for tiered memory systems is yet to be investigated. Improving the accuracy of hotness measurement for huge pages and the efficiency of page migration for huge pages for tiered memory systems have been investigated by the prior studies [12], [13]. This paper explores the migration policy space with huge pages.…”

AutoTM: Automatic Tensor Movement in Heterogeneous Memory Systems using Integer Linear Programming

“…These approaches use a software managed DRAM cache to mitigate the slow performance and block level read/write granularity of NVM SSDs. Operating system support for managing heterogeneous memory [2,45] and support for transparent unified memory between GPU and CPU [26,33] have been studied extensively in the past. However, to the best of our knowledge, the proposed work is the first to explore the design space and cost-performance tradeoffs of large scale DNN training on systems with DRAM and PMM.…”

ECO TLB

“…Software approaches. Similarly, there are a number of approaches in the operating system [1,8,11,15,17,44,54,55,57,63] to optimize TLB shootdowns. Barrelfish [11], a research multi-kernel OS, uses message passing instead of IPIs to shoot down remote TLB entries.…”