Inferential Queueing and Speculative Push

Abstract

Communication latencies within critical sections constitute a major bottleneck in some classes of emerging parallel workloads. In this paper, we argue for the use of two mechanisms to reduce these communication latencies: Inferentially Queued locks (IQLs) and Speculative Push (SP). With IQLs, the processor infers the existence, and limits, of a critical section from the use of synchronization instructions and joins a queue of lock requestors, reducing synchronization delay. The SP mechanism extracts information about program structure by observing IQLs. SP allows the cache controller, responding to a request for a cache line that likely includes a lock variable, to predict the data sets the requestor will modify within the associated critical section. The controller then pushes these lines from its own cache to the target cache, as well as writing them to memory. Overlapping the protected data transfer with that of the lock can substantially reduce the communication latencies within critical sections. By pushing data in exclusive state, the mechanism can collapse a read-modify-write sequences within a critical section into a single local cache access. The write-back to memory allows the receiving cache to ignore the push. Neither mechanism requires any programmer or compiler support nor any instruction set changes. Our experiments demonstrate that IQLs and SP can improve performance of applications employing frequent synchronization.

Reflections from the 2013 Eckert-Mauchly Award Recipient

“…For example, recognizing that certain memory locations could be associated with a given lock suggested the possibility of speculative push, passing cache lines modified within the CS at the time the now-available lock was replaced. 9 Ravi Rajwar extended the notion of speculating about critical sections one step further, recognizing that a CS without data conflicts need not acquire the lock and therefore can share the cache line containing the lock, permitting concurrent execution of critical sections protected by a common lock. 10 I only realized how counterintuitive this was when I described it to knowledgeable colleagues who initially insisted this was impossible since the programmer explicitly invokes mutual exclusion.…”

Speculation Meets Checkpointing

“…The speculation methods are required to be very fast, while they do not necessary have to make correct predictions, as the cost of the mistakes is usually considered negligible. They include speculative pushes of shared objects to processing nodes before they would actually demand access [7], prefetching of the shared objects [1], or self-invalidation of shared objects [5] among other techniques.…”

Using Speculative Push for Unnecessary Checkpoint Creation Avoidance

“…Speculation, on the other hand, is a technique which promises to increase the speed of DSM operations and to reduce the gap between DSM and messagepassing systems. The speculation may involve speculative pushes of shared objects to processing nodes, before they actually demand access [10], prefetching of the shared objects with anticipation that the application process would need those objects ( [1]) or self invalidation of shared objects to reduce the frequency of "3-hop-misses" ( [8]) among other techniques.…”