Understanding the propagation of hard errors to software and implications for resilient system design

Abstract

With continued CMOS scaling, future shipped hardware will be increasingly vulnerable to in-the-field faults. To be broadly deployable, the hardware reliability solution must incur low overheads, precluding use of expensive redundancy. We explore a cooperative hardware-software solution that watches for anomalous software behavior to indicate the presence of hardware faults. Fundamental to such a solution is a characterization of how hardware faults in different microarchitectural structures of a modern processor propagate through the application and OS.This paper aims to provide such a characterization, resulting in identifying low-cost detection methods and providing guidelines for implementation of the recovery and diagnosis components of such a reliability solution. We focus on hard faults because they are increasingly important and have different system implications than the much studied transients. We achieve our goals through fault injection experiments with a microarchitecture-level full system timing simulator. Our main results are: (1) we are able to detect 95% of the unmasked faults in 7 out of 8 studied microarchitectural structures with simple detectors that incur zero to little hardware overhead; (2) over 86% of these detections are within latencies that existing hardware checkpointing schemes can handle, while others require software checkpointing; and (3) a surprisingly large fraction of the detected faults corrupt OS state, but almost all of these are detected with latencies short enough to use hardware checkpointing, thereby enabling OS recovery in virtually all such cases.

Accurate microarchitecture-level fault modeling for studying hardware faults

“…We inject faults under µarch-level stuck-at, gate-level stuck-at, and gate-level delay fault models, and use the previously studied SWAT detection techniques to understand their system-level manifestation [12]. We show that, in general, µarch-level stuck-at faults do not result in similar system-level fault manifestation as gate-level stuckat or delay faults.…”

“…For this purpose, we use the SWAT symptom-based detection scheme because these detectors essentially capture how hardware faults manifest into the system level and software [12].…”

“…Examples include software-level symptom-based detection techniques such as SWAT that capture how hardware faults manifest to the system level [12,22], end-to-end error detection and correction [23], microarchitecture-level (µarch-level) redundancy [27], and circuit-level BIST techniques [6].…”

“…This paper concerns accurate models of hardware faults at the microarchitecture level to evaluate faulttolerant solutions at the microarchitecture and higher levels. Recently, several µarch-level solutions that tolerate hardware failures have been proposed [2,8,12,14,25,30]. The primary evaluation mode for these proposals has been through statistical fault injections in simulations either at the gate level [8,14,25] or the microarchitectural state elements (e.g., output latch of an ALU) [2,12,30].…”

“…Recently, several µarch-level solutions that tolerate hardware failures have been proposed [2,8,12,14,25,30]. The primary evaluation mode for these proposals has been through statistical fault injections in simulations either at the gate level [8,14,25] or the microarchitectural state elements (e.g., output latch of an ALU) [2,12,30]. While gate-level fault injections can accurately capture lower level faults, the long simulation time of these schemes prevents detailed evaluation of the propagation of gate-level faults through the hardware and into the software.…”