Timestamp snooping

Abstract

Symmetric multiprocessor (SMP) servers provide superior performance for the commercial workloads that dominate the Internet. Our simulation results show that over one-third of cache misses by these applications result in cache-to-cache transfers, where the data is found in another processor's cache rather than in memory. SMPs are optimized for this case by using snooping protocols that broadcast address transactions to all processors. Conversely, directory-based shared-memory systems must indirectly locate the owner and sharers through a directory, resulting in larger average miss latencies. This paper proposes timestamp snooping, a technique that allows SMPs to i) utilize high-speed switched interconnection networks and ii) exploit physical locality by delivering address transactions to processors and memories without regard to order. Traditional snooping requires physical ordering of transactions. Timestamp snooping works by processing address transactions in a logical order. Logical time is maintained by adding a few bits per address transaction and having network switches perform a handshake to ensure on-time delivery. Processors and memories then reorder transactions based on their timestamps to establish a total order.We evaluate timestamp snooping with commercial workloads on a 16-processor SPARC system using the Simics full-system simulator. We simulate both an indirect (butterfly) and a direct (torus) network design. For OLTP, DSS, web serving, web searching, and one scientific application, timestamp snooping with the butterfly network runs 6-28% faster than directories, at a cost of 13-43% more link traffic. Similarly, with the torus network, timestamp snooping runs 6-29% faster for 17-37% more link traffic. Thus, timestamp snooping is worth considering when buying more interconnect bandwidth is easier than reducing interconnect latency.

A detailed comparison of two transaction processing workloads

“…Oracle8 Improvement [5,6,7,11,12]. Overall, the studies using these two classes of workloads have led to similar conclusions: high frequency of "unpredictable" branches, large memory stall component for both instructions and data, and significant communication misses.…”

Cache Coherence Protocols for Many-Core CMPs

“…It could be also used in conjunction with techniques that try to make the best use of the limited on-chip cache storage. Martin et al (2000) present a technique that allows snooping-based protocols to utilize unordered networks by adding logical timing to coherence requests and reordering them on destiny to establish a total order. Likewise, Agarwal et al (2009) propose In-Network Snoop Ordering (INSO) to allow snooping over unordered networks.…”

Improving Multiprocessor Performance with Coarse-Grain Coherence Tracking

“…These networks allow processors to control the timing of message delivery and pipeline messages without violating sequential consistency. Martin et al proposed Timestamp snooping [20], an improved technique that introduces the concept of "slack" to cope with network delays, and requires fewer messages to implement. Martin et al later proposed Token Coherence [21], which uses tokens to grant coherence permissions, eliminating the need for ordered networks or logical time, and creating a simple correctness substrate onto which speculative optimizations, such as destination-set prediction can be implemented [22].…”