Energy-aware systems, DependabilityRecent trends using computing clusters point to a growing demand for high compute density environments in various application fields such as server appliances including distributed web servers.Distributed web servers need many server nodes and low-latency/high-bandwidth network for operating a massive amount of web services including distribution of high-definition movies. In these computing clusters, power consumption and system cost have increased. Therefore, it is vital to downsize computing cluster without loss of high dependability including fault tolerance.To realize high performance, power-aware and highly dependable network, we have proposed a small computing cluster for embedded systems. This small computing cluster has low power computing nodes such as PCs and embedded CPUs. It also has a communication link between the computing nodes called PEARL (PCI Express Adaptive and Reliable Link) [1].Commodity network devices such as Gigabit Ethernet (GbE) and InfiniBand are not sufficient for small computing clusters. InfiniBand is a switched fabric communication link used in high-performance computing and enterprise data centers. It achieves high reliability but power consumption is relatively high [2]. GbE is a cost and power rival of InfiniBand. However, GbE does not match transmission performance of InfiniBand.In order to achieve both high performance and low power consumption, PEARL uses PCI Express 2/28 (PCIe) [3], which is a high-speed serial I/O interface standard in PCs, not only for connecting peripheral devices but also as a communication link between computing nodes.To implement PEARL, a communication device, PEACH (PCI Express Adaptive Communication Hub) which acts as a switching device, has been developed. PCIe transfers packets point-to-point bidirectionally with high bandwidth. However it connects only between a Root Complex (RC) and Endpoints (EPs).Therefore, a problem exists in that PCIe interfaces on PCs cannot be connected to each other, because every node CPU in the computing node is an RC. To solve this problem, each node CPU is equipped with a network interface card with PEACH. The node CPUs are connected to each other by a PCIe cable.To pair an RC with EPs at each end of the PCIe cable, PEACH can switch the RC port and EP ports to connect two computing nodes peer-to-peer.PEACH can address two computing nodes as peers, breaking the traditional PCIe limit of only linking to a single master.
PEACH overviewThe communication device, PEACH, has four PCIe Rev.2.0 ports with four lanes each, and employs an eight-core control processor [4]. There are several advantages of using of PEACH in the proposed network. Four PCIe ports can broaden the scope of selection of network topology. The high bandwidth of 20Gbps/port is equal to that of InfiniBand DDR 4x. The multicore control processor performs fault handling and system monitoring / logging for dependability. The multicore processor also controls the network system for power awareness. Figure 1 shows a prototype...