RAPID: Reconfigurable and Scalable All-Photonic Interconnect for Distributed Shared Memory Multiprocessors

Abstract

Abstract-In this paper, we describe the design and analysis of a scalable architecture suitable for large-scale distributed shared memory (DSM) systems. The approach is based on an interconnect technology which combines optical components and a novel architecture design. In DSM systems, numerous shared memory transactions such as requests, responses and acknowledgment messages propagate simultaneously in the network. As the network size increases, network contention results in increasing the critical remote memory access latency, which significantly penalizes the performance of DSM systems. In our proposed architecture called reconfigurable and scalable all-photonic interconnect for distributed-shared memory (RAPID), we provide high connectivity by maximizing the channel availability for remote communication to reduce the critical remote latency. RAPID provides fast and efficient unicast, multicast and broadcast capabilities using a combination of aggressively designed wavelength division multiplexing (WDM), time division multiplexing (TDM), and space division multiplexing (SDM) techniques. RAPID is wavelength-routed, permitting the same limited set of wavelength to be reused among all processors. We evaluated RAPID based on network characteristics, power budget criteria, and by simulation using synthetic traffic workloads and compared it against other networks such as electrical ring, torus, mesh, and hypercube networks. We found that RAPID outperforms all networks and still provides good performance as the network is scaled to very large numbers.Index Terms-Cache coherence, distributed shared memory, optical interconnects, scalable optical networks.

Reconfigurable and adaptive photonic networks for high-performance computing systems

“…We previously proposed a reconfigurable allphotonic interconnect for distributed and parallel systems (RAPID) [15]. RAPID topology maximizes bandwidth availability and lowers network latency.…”

“…An E-RAPID network [15] is defined by a 3-tuple ðC; B; DÞ, where C is the total number of clusters, B is the total number of boards per cluster, and D is the total number of nodes per board. The total number of nodes in E-RAPID is the multiplicative factor N ¼ C × D × B.…”

“…The wavelengths used in each home channel are the same, thereby reusing the same set of wavelengths. Multicast and broadcast implementation have previously been described in [15].…”

RAPID for high-performance computing systems: architecture and performance evaluation

“…The wavelength 0 is used for multicast and broadcast communication. 9 For remote traffic, the number of wavelengths required to obtain the connectivity mentioned above is B, i.e. ͑B Ϫ 1͒ wavelengths are required to communicate with every other board and one more wavelength for multicast communication.…”

Application of self organizing maps for investigating network latency on a broadcast-based distributed shared memory multiprocessor

“…Several broadcast-based optical DSM multiprocessors have been proposed (Artundo et al, 2006;Kodi & Louri, 2004a, 2004b. To achieve high levels of performance in these architectures, remote memory access times must be minimized (Hawkins, Wills, Liboiron-Ladouceur, & Bergman, 2007).…”