Hardware fault containment in scalable shared-memory multiprocessors

Abstract

Current shared-memory multiprocessors are inherently vulnerable to faults: any significant hardware or system software fault causes the entire system to fail. Unless provisions are made to limit the impact of faults, users will perceive a decrease in reliability when they entrust their applications to larger machines. This paper shows that fault containment techniques can be effectively applied to scalable shared-memory multiprocessors to reduce the reliability problems created by increased machine size.The primary goal of our approach is to leave normal-mode performance unaffected. Rather than using expensive faulttolerance techniques to mask the effects of data and resource loss, our strategy is based on limiting the damage caused by faults to only a portion of the machine. After a hardware fault, we run a distributed recovery algorithm that allows normal operation to be resumed in the functioning parts of the machine.Our approach is implemented in the Stanford FLASH multiprocessor. Using a detailed hardware simulator, we have performed a number of fault injection experiments on a FLASH system running Hive, an operating system designed to support fault containment. The results we report validate our approach and show that in conjunction with an operating system like Hive, we can improve the reliability seen by unmodified applications without substantial performance cost. Simulation results suggest that our algorithms scale well for systems up to 128 processors. NodeControlle (MAGIC) x Memory Node FIGURE 2.1. Overview of the FLASH multiprocessor. FLASH consists of a set of nodes connected through a point-topoint interconnect. Each node contains a portion of the distributed main memory and a node controller that handles cache coherence and other communication within the node and with other nodes.

Deadlock-free dynamic reconfiguration schemes for increased network dependability

“…Using the formal definitions given in Section 3.3, the routing function, R, is defined as follows: 7 These routing subfunctions compose both the old and the new routing functions but provide different routing capabilities, depending on which routing function they compose. The routing subfunction R E o =A n allows escape routing on the E o =A n channels when it composes the old routing function, but allows fully adaptive routing on those same channels when it composes the new routing function.…”

“…Static reconfiguration, for example, consists of first stopping and draining all user traffic from the network before commencing and completing network-wide reconfiguration [4], [5], [6], [7]. With this approach, network drainage typically occurs by actively discarding all nondelivered packets not yet reaching their destination nodes.…”

Cellular disco

“…Software techniques by themselves cannot provide hardware fault containment; they require the cooperation of the hardware, as described by Teodosiu et al [1997]. We assume that the hardware exhibits fail-stop behavior, which implies that after a fault the hardware stops working without generating erroneous results.…”

“…A fault containment unit has to be self-sufficient; therefore, it cannot be smaller than a node because a node controller failure will render the entire node useless. Further details are beyond the scope of this paper; they are covered in Teodosiu et al [1997].…”

A methodology for developing deadlock-free dynamic network reconfiguration processes. Part II

“…In transitioning between the old and new routing functions during network reconfiguration, additional dependencies among network resources may be introduced, causing what is referred to as reconfiguration-induced deadlock. Current techniques typically handle this situation through static reconfiguration-meaning that application traffic is stopped and, usually, dropped from the network during the reconfiguration process (see, for example, [17], [18]). While this approach guarantees the prevention of reconfiguration-induced deadlock, it can lead to unacceptable packet latencies and dropping frequencies for many applications, like compute data-centers, high availability webservers, video-servers, and process control servers.…”