Cache index-aware memory allocation

Abstract

Poor placement of data blocks in memory may negatively impact application performance because of an increase in the cache conflict miss rate [18]. For dynamically allocated structures this placement is typically determined by the memory allocator. Cache index-oblivious allocators may inadvertently place blocks on a restricted fraction of the available cache indexes, artificially and needlessly increasing the conflict miss rate. While some allocators are less vulnerable to this phenomena, no general-purpose malloc allocator is index-aware and methodologically addresses this concern. We demonstrate that many existing state-of-the-art allocators are index-oblivious, admitting performance pathologies for certain block sizes. We show that a simple adjustment within the allocator to control the spacing of blocks can provide better index coverage, which in turn reduces the superfluous conflict miss rate in various applications, improving performance with no observed negative consequences. The result is an index-aware allocator. Our technique is general and can easily be applied to most memory allocators and to various processor architectures.Furthermore, we can reduce inter-thread and inter-process conflict misses for processors where threads concurrently share the level-1 cache such as the Sun UltraSPARC-T2 TM and Intel "Nehalem" by coloring the placement of blocks so that allocations for different threads and processes start on different cache indexes.

Avoiding Scalability Collapse by Restricting Concurrency

“…The results show that GCR avoids the scalability collapse, which translates to substantial speedup (up to three orders of magnitude) in case of high lock contention for virtually every evaluated lock, workload and machine. Furthermore, we show empirically that GCR does not harm the fairness of underlying locks (in 1 We also discuss other waiting policies and their limitations later in the paper. fact, in many cases GCR makes the fairness better).…”

“…Thus, limiting the maximum number of threads by the number of cores does not help much. Finally, even when a saturated lock delivers a seemingly stable performance, threads spinning and waiting for the lock consume energy and take resources (such as CPU time) from other, unrelated tasks 1 .…”

A Survey on Cache Management Mechanisms for Real-Time Embedded Systems

“…The problem of developing OS-level memory allocators that can perform cache-aware allocations and feature a predictable execution time has been addressed in [Chilimbi et al 2000;Afek et al 2011;Herter et al 2011]. The common denominator for all the aforementioned techniques is to improve the predictability of real-time systems deployed on top of cache-based architectures, in order to provide better isolation guarantees for real-time embedded applications.…”

“…Cache-Index Friendly (CIF) also tries to improve the average execution time, instead of improving the predictability, by explicitly controlling the cache-index position of allocated memory blocks [Afek et al 2011]. The central idea in CIF is to insert small spacer regions into the array of blocks within the allocator to better distribute block indices and disrupting the regular ordering of block addresses, returned by the allocator.…”

TWA – Ticket Locks Augmented with a Waiting Array

“…An additional confounding fact is that under certain loads, the MCS unlock operator may execute futile CAS operations that generate unnecessary coherence traffic, and that the unlock operator may need to busy-wait to allow an arriving successor to update the next pointer in the owner's qnode. 2 Threads might also malloc and free queue nodes as needed, but most malloc allocators are not sufficiently scalable. Also, many malloc implementations themselves make use of POSIX locks, resulting in reentry and recursion if a lock implementation were to try to call malloc which in turn would need to acquire a lock.…”