Parallel optical interconnects may reduce the communication bottleneck in symmetric multiprocessors

Abstract

We start with a detailed analysis of the communication issues in today's symmetric multiprocessor (SMP) architectures to study the benefits of implementing optical interconnects (OI) in these machines. We show that the transmission of block addresses is the most critical communication bottleneck of future large SMPs owing to the need to preserve the coherence of data duplicated in caches. An address transmission bandwidth as high as 200-300 Gb/s may be necessary in ten years from now; this requirement will represent a difficult challenge for shared electric buses. In this context we suggest the introduction of simple point-to-point OIs for a SMP cache-coherent switch, i.e., for a VLSI switch that would emulate the shared-bus function. The operation might require as much as 10,000 input-outputs (IOs) to connect 100 processors, particularly if one maintains the present parallelism of transmissions to preserve a large bandwidth and a short memory access latency. The interest for OIs comes from the potential increase of the transmission frequency and from the possible integration of such a high density of IOs on top of electronic chips to overcome packaging issues. Then we consider the implementation of an optical bus that is a multipoint optical line involving more optical technology. This solution allows multiple simultaneous accesses to the bus, but the preservation of the coherence of caches can no longer be maintained with the usual fast snooping protocols.

An optical interconnection network and a modified snooping protocol for the design of large-scale symmetric multiprocessors (SMPs)

“…One technology that can provide high communication bandwidth, low latency, and scalability is optical interconnection technology [9], [4], [5]. The recent advances in optical interconnect devices and packaging techniques such as multidimensional arrays of vertical cavity surface emitting lasers (VCSELs), arrays of photodetectors (PDs), and waveguide optics [10] are making optical interconnects a serious and potentially viable interconnect technology for parallel computing.…”

“…The photobus smart pixel interconnection system for shared-memory multiprocessors use optical buses for broadcasting the address requests, but arbitration is implemented using electronic buses leading to buffering of address requests at the smart pixel VLSI chip [15]. The constraints of access arbitration is eliminated in the U-bus [5] design for SMPs. U-bus extends the address bandwidth, but a new coherence protocol must be designed to maintain consistency across the caches.…”

“…Other fundamental problems such as impedance mismatch, stray capacitance, 4I noise, and crosstalk [3], [4] significantly affect the speed improvements of the shared bus. Therefore, the bus speed and the coherence overhead limit the rate at which address requests can be broadcast to all the processors/memory modules [5], [6]. This address rate/bandwidth is the main scaling limit, which cannot follow the increasing demands of faster and large number of processors, limiting the scalability of shared-bus-based SMPs.…”

Reconfigurable optical interconnects for parallel computer systems: design space issues

“…At 1 GHz bus speed, this maximum bus length would reduced to approximately 3 mm. Since this is insufficient, either multiple electronic buses or an optical bus have to be used [8].…”

SYMNET: an optical interconnection network for scalable high-performance symmetric multiprocessors

“…Therefore the bus speed and the coherence overhead limit the rate at which address requests can be broadcast to all the processors͞memory modules. 7,8 This in turn limits the number of processors that can share the bus by affecting the scalability of SMP systems. 8 This address rate͞bandwidth is the main scaling limit, which cannot keep pace with the increasing demands of faster and multiple processors, on the scalability of shared-bus based SMPs.…”