A new approach to parallelising tracing algorithms

Abstract

Tracing algorithms visit reachable nodes in a graph and are central to activities such as garbage collection, marshalling etc. Traditional sequential algorithms use a worklist, replacing a nodes with their unvisited children. Previous work on parallel tracing is processororiented in associating one worklist per processor: worklist insertion and removal requires no locking, and load balancing requires only occasional locking. However, since multiple queues may contain the same node, significant locking is necessary to avoid concurrent visits by competing processors.This paper presents a memory-oriented solution: memory is partitioned into segments and each segment has its own worklist containing only nodes in that segment. At a given time at most one processor owns a given worklist. By arranging separate single-readersingle-writer forwarding queues to pass nodes from processor i to processor j we can process objects in an order that gives lock-free mainline code and improved locality of reference. This refactoring is analogous to the way in which a compiler changes an iteration space to eliminate data dependencies.While it is clear that our solution can be more effective on NUMA systems, and even necessary when processor-local memory may not be addressed from other processors, slightly surprisingly, it often gives significantly better speed-up on modern multi-cores architectures too. Using caches to hide memory latency loses much of its effectiveness when there is significant cross-processor memory contention or when locking is necessary.

A study of connected object locality in NUMA heaps

“…al. [20] partition the memory and create a work list per region. A processor must own a work list to trace the local objects; consequently, load balancing is achieved by stealing a complete work list.…”

“…Several memory-oriented tracing techniques have been devised to maintain object locality. They segregate the heap into partitions and each GC thread processes local objects in a partition [22,20]. However, these techniques are unaware whether the set of connected objects terminates or continues to cross other memory nodes.…”

Assessing the scalability of garbage collectors on many cores

“…This is the scheme implemented in OpenJDK, and Section 3 shows that it doesn't scale well. Oancea et al [12] associate work-lists to dedicated partitions of the heap. Only the owner of a worklist (the worker thread that works exclusively on that worklist) traces the objects in the corresponding heap partition.…”

Topology-Aware Parallelism for NUMA Copying Collectors

“…Alternatively, Shuf et al [25] push references to remote objects into a shared queue enabling other threads to process them. For load balancing, GC threads need to lock unprocessed queues to trace live objects [21]. However, a memory segment boundary might not match the physical memory page size nor it is assigned to local threads; therefore, further locality improvements are required.…”