Phase-Based Data Placement Scheme for Heterogeneous Memory Systems

Abstract

Heterogeneous memory systems are equipped with two or more types of memories, which work in tandem to complement the capabilities of each other. The multiple memories can vary in latency, bandwidth and capacity characteristics across systems and they come in various configurations that can be managed by the programmer. This introduces an added programming complexity for the programmer. In this paper, we present a dynamic phase-based data placement scheme to assist the programmer in making decisions about program object allocations. We devise a cost model to assess the benefit of having an object in one type of memory over the other and apply the cost model at every application phase to capture the dynamic behaviour of an application. Our cost model takes into account the reference counts of objects and incurred transfer overhead when making a suggestion. In addition, objects can be transferred across memories asynchronously between phases to mask some of the transfer overhead. We test our cost model with a diverse set of applications from NAS Parallel and Rodinia benchmarks and perform experiments on Intel KNL, which is equipped with a high bandwidth memory (MCDRAM) and a high capacity memory (DDR). Our dynamic phase-based data placement performs better than initial placement and achieves comparable or better performance than cache mode of MCDRAM.

ecoHMEM: Improving Object Placement Methodology for Hybrid Memory Systems in HPC

“…Dulloor et al [7] also demonstrated that a careful placement of data-structures across memory tiers is necessary to incorporate NVM into the processor memory hierarchy. Plenty of methodology has been proposed for this purpose, including profiling based on sampling of hardware counters [21], [35], runtime-assisted profiling [3], or object-level placement granularity [8], [29] versus its page-level counterpart [39], [40]. Wen et al [38] proposed ProfDP, a differential profiling mechanism (requiring three profiling runs) to estimate per-object latency and bandwidth sensitivity, and decide a priority to guide placement decisions.…”

Online Application Guidance for Heterogeneous Memory Systems

“…Some other projects employ application-level tools to tag and profile certain data structures, and then use heuristic models to assign objects to the appropriate tier [2,4,18,42,63,66]. While these efforts demonstrate that application guidance can be useful for certain usage scenarios, they require manual source code modifications or expensive online detection to attach recommendations to data objects.…”