Hierarchical Cache Directory for CMP

Abstract

As more processing cores are integrated into one chip and feature size continues to shrink, the average access latency for remote nodes using directory-based coherence protocol becomes higher, which greatly impacts system performance. Previous techniques such as data replication and data migration optimize the performance of the requesting core, but offer little improvement for neighbor nodes. Other techniques such as in-transit optimization try to reduce latency at the cost of increased storage. This paper introduces hierarchical cache directory into CMP (chip multiprocessor), which divides CMP tiles into multiple regions hierarchically, and combines it with data replication. A new directory organization is proposed to record the share status within a region and assist the regional home to complete operation efficiently. Simulation results show that for a 16-core CMP, compared to traditional directory, hierarchical cache directory reduces average access latency by 9% and on-chip network traffic by 34% on average with less storage. Theoretical analyses show that for a 2 n × 2 n tiled CMP, the average access latency in hierarchical cache directory asymptotically approaches a function that is independent of n, hence the architecture is highly scalable.

PS directory: a scalable multilevel directory cache for CMPs

“…When using compression, area is saved at expenses of using an inexact representation of the sharer vector, thus yielding to performance losses. Hierarchical [11] representation of the sharer vector has been also used for entry size reduction purposes. However, hierarchical organizations impose additional lookups on the critical path so hurting latency.…”

Cuckoo directory: A scalable directory for many-core systems

“…Additionally, hierarchical organizations reduce directory storage by using coarse bit-vectors at a primary location and exact sub-bit-vectors at secondary locations [44,45]. Hierarchical techniques save storage by breaking up large bit vectors and allocating only the necessary second-level sub-bit-vectors, at the cost of additional storage to replicate the tags multiple times, once for each allocated second-level entry.…”

“…Hierarchical techniques save storage by breaking up large bit vectors and allocating only the necessary second-level sub-bit-vectors, at the cost of additional storage to replicate the tags multiple times, once for each allocated second-level entry. Figure 4 demonstrates the storage scalability of the Sparse Coarse [17] design that precisely stores sharers in the available bits (2*log(#caches) bits) and falls back to a coarse vector representation in the case of overflow [24], and Sparse Hierarchical [44,45], a 2-level hierarchical directory organization. Although theoretically scalable, these schemes address only the vector storage inside each entry and not the total number of directory entries.…”

“…The Cuckoo organization dictates only the organization of the directory itself, not the contents of each entry or its home node. In this work, we constructed the Cuckoo directory with the coarse [17] and hierarchical [44,45] approaches, although the Cuckoo organization can be used in conjunction with any of these space-reduction techniques.…”

“…Hierarchical directory organizations [44,45] can enable area-efficient uncompressed vector storage through multiple serialized lookups. However, these techniques address only the size of the sharer vectors, not the number of vectors the directory must store.…”