Scalable locality-conscious multithreaded memory allocation

Abstract

We present Streamflow, a new multithreaded memory manager designed for low overhead, high-performance memory allocation while transparently favoring locality. Streamflow enables low overhead simultaneous allocation by multiple threads and adapts to sequential allocation at speeds comparable to that of custom sequential allocators. It favors the transparent exploitation of temporal and spatial object access locality, and reduces allocator-induced cache conflicts and false sharing, all using a unified design based on segregated heaps. Streamflow introduces an innovative design which uses only synchronization-free operations in the most common case of local allocations and deallocations, while requiring minimal, non-blocking synchronization in the less common case of remote deallocations. Spatial locality at the cache and page level is favored by eliminating small objects headers, reducing allocator-induced conflicts via contiguous allocation of page blocks in physical memory, reducing allocator-induced false sharing by using segregated heaps and achieving better TLB performance and fewer page faults via the use of superpages. Combining these locality optimizations with the drastic reduction of synchronization and latency overhead allows Streamflow to perform comparably with optimized sequential allocators and outperform-on a shared-memory system with four two-way SMT processors-four state-of-the-art multiprocessor allocators by sizeable margins in our experiments. The allocation-intensive sequential and parallel benchmarks used in our experiments represent a variety of behaviors, including mostly local object allocation-deallocation patterns and producer-consumer allocation-deallocation patterns.

A scalable architecture for ordered parallelism

“…In principle, we could build a task-aware allocator with per-core memory pools to avoid serialization. However, building high-performance allocators is complex [26,65]. Instead, the simulator allocates and frees memory in a task-aware way.…”

Avoiding cache thrashing due to private data placement in last-level cache for manycore scaling

“…Our performance study uses a baseline that already employs a scalable locality-aware memory allocator that maintains per-core lists of private pages for private data [9], [10]. By doing so, we isolate our results from previous studies that aim to reduce coherence traffic.…”

“…While separate address spaces [7], locality-aware memory allocators [8], [9], [10] and task schedulers [11], [12], [13] can reduce coherence traffic, they do not address capacity or conflict misses in the LLC. Even based upon a locality-aware memory allocator, we show that with 16 eight-way multi-threaded cores in an inclusive organization, 5-10% of cache sets in a 16-way associative LLC are severely contended for by private data, raising conflict misses and unnecessary L1 evictions in an inclusive organization.…”

On the detection of custom memory allocators in C binaries

“…Many research projects, like (Schneider et al 2006;Rauchwerger 2007, 2009;Liu and Chen 2012;Lyberis et al 2012), propose new memory managers designed for highperformance memory allocation. Other approaches, e.g., (Berger and Zorn 2006;Lvin et al 2008;Novark and Berger 2010;Novark et al 2009; Perence, B, Electric Fence, http://perens.com/ FreeSoftware/ElectricFence; Akritidis 2010; Serebryany et al 2012), have used custom memory managers tailored to improve the memory safety of applications using them.…”