Relaxed consistency and coherence granularity in DSM systems

Abstract

During the past few years, two main approaches have been taken to improve the performance of software shared memory implementations: relaxing consistency models and providing fine-grained access control. Their performance tradeoffs, however, we not well understood.This paper studies these tradeoffs on a platform that provides access control in hardware but runs coherence protocols in software, We compare the performance of three protocols across four coherence granularities, using 12 applications on a 16-node cluster of workstations. Our results show that no single combination of protocol and granularity performs best for all the applications. The combination of a sequentially consistent (SC) protocol and fine granularity works well with 7 of the 12 applications. The combination of a multiple-writer, home-based lazy release consistency (HLRC) protocol and page granularity works well with 8 out of the 12 applications. For applications that suffer performance losses in moving to coarser granularity under sequential consistency, the performance can usually be regained quite effectively using relaxed protocols, particularly HLRC. We also find that the HLRC protocol performs substantially better than a single-writer lazy release consistent (SW-LRC) protocol at cease granularity for many irregular applications. For our applications and platform, when we use the original versions of the applications ported directly from hardware-coherent shared memory, we find that the SC protocol with 256-byte granularity performs best on average. However, when the best versions of the applications are compared, the balance shifts in favor of HLRC at page granularity. IntroductionThere are two important issues in providing a coherent shared address space abstraction on a network of computers, consistency models and coherence granularity. Consistency models define how applications use the shared address space, whereas the degree of the relaxation of a consistency protocol and the granularity of coherence determine the efficiency of an implementation. This paper evaluates the performance tradeoffs of the combinations of three consistency models with four sizes of coherence granularity for software shared memory implementations on a real hardware platform. The original shared virtual memory (SVM) proposal and prototype [20] uses the traditional virtual memory access protection mechanisms to detect access misses and implements a sequential consistency model [17]. The main advantage of the approach is that it implements shared memory entirely in software on a network of commodity workstations [19] to run applications developed for hardware shared-memory multiprocessors. A disadvantage is that it restricts the coherence granularity to be a virtual memory page size. For systems with large page sizes, false sharing and fragmentation will occur in applications with multiple writer, fine-grained access patterns.During the past few years, two main approaches have been taken to address this problem: relaxing consistency models and providing access...

Addressing a workload characterization study to the design of consistency protocols

“…Although this period is much shorter than that those found in pure CGS, it is much longer (several orders of magnitude) than the lengths found in the other FGS workloads. Zhou et al [6] state that Ocean possesses a CGS pattern; however, our results indicate that in some intervals, Ocean is FGS. For example, the synchronization period in the intervals 5, 15, 16, 22, 27 and 29 appear in bursts (with a synchronization period less than 100 cycles).…”

“…The frequency of sharing metric is calculated as the average computation time between synchronization events [6]. We assume there is fine-grained synchronization (FGS) if the average computation time is close to the average synchronization time.…”

“…A synchronization operation can take several microseconds to be processed in a typical SVM system [6], which translates to thousands of clock cycles in current microprocessors. Taking this into account, we classify the workloads in the first group as having FGS synchronization and workloads in the second group as having CGS synchronization.…”

“…Previous research in the open literature has explored the behavior of parallel workloads as run on SVM systems [3,4,6]. Iftode et al [4] established a workload taxonomy based on sharing patterns and the granularity of sharing.…”

“…Jiang et al [3] modified source code based on the same set of described axes to improve the performance of SVM systems. Zhou et al [6] studied the behavior of workloads running on several protocols and systems, establishing a number of rules about the optimal granularity of sharing. In their workload taxonomy, they also studied the frequency of synchronization.…”