Assessment of the Impact of Cosmic-Ray-Induced Neutrons on Hardware in the Roadrunner Supercomputer

Michalak, Sarah; DuBois, Andrew; Storlie, Curtis B.; Quinn, Heather; Rust, William N.; DuBois, D. F.; Modl, David G.; Manuzzato, Andrea; Blanchard, Sean

doi:10.1109/tdmr.2012.2192736

Cited by 36 publications

(24 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Flip-flop soft errors can result in the following outcomes [26], [30], [25], [29], [31]: Vanished -normal termination and output files match error-free runs, Output Mismatch (OMM) -normal termination, but output files are different from errorfree runs, Unexpected Termination (UT) -program terminates abnormally, Hang -no termination or output within 2× the nominal execution time, Error Detection (ED) -an employed resilience technique flags an error, but the error is not recovered using a hardware recovery mechanism.…”

Section: A Reliability Analysismentioning

confidence: 99%

Tolerating Soft Errors in Processor Cores Using CLEAR (Cross-Layer Exploration for Architecting Resilience)

Cheng

Mirkhani

Szafaryn

et al. 2018

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

Abstract-We present CLEAR (Cross-Layer Exploration for Architecting Resilience), a first of its kind framework which overcomes a major challenge in the design of digital systems that are resilient to reliability failures: achieve desired resilience targets at minimal costs (energy, power, execution time, area) by combining resilience techniques across various layers of the system stack (circuit, logic, architecture, software, algorithm). This is also referred to as cross-layer resilience. In this paper, we focus on radiation-induced soft errors in processor cores. We address both single-event upsets (SEUs) and single-event multiple upsets (SEMUs) in terrestrial environments. Our framework automatically and systematically explores the large space of comprehensive resilience techniques and their combinations across various layers of the system stack (586 cross-layer combinations in this paper), derives cost-effective solutions that achieve resilience targets at minimal costs, and provides guidelines for the design of new resilience techniques. We demonstrate the practicality and effectiveness of our framework using two diverse designs: a simple, in-order processor core and a complex, out-of-order processor core. Our results demonstrate that a carefully optimized combination of circuit-level hardening, logic-level parity checking, and micro-architectural recovery provides a highly cost-effective soft error resilience solution for general-purpose processor cores. For example, a 50× improvement in silent data corruption rate is achieved at only 2.1% energy cost for an out-of-order core (6.1% for an in-order core) with no speed impact. However, (application-aware) selective circuit-level hardening alone, guided by a thorough analysis of the effects of soft errors on application benchmarks, provides a cost-effective soft error resilience solution as well (with ~1% additional energy cost for a 50× improvement in silent data corruption rate).Index Terms-Cross-layer resilience; soft errors I. INTRODUCTION HIS paper addresses the cross-layer resilience challenge for designing robust digital systems: given a set of resilience techniques at various abstraction layers (circuit, logic, architecture, software, algorithm), how does one protect a given design from radiation-induced soft errors using (perhaps) a combination of these techniques, across multiple abstraction layers, such that overall soft error resilience targets are met at minimal costs (energy, power, execution time, area)? Specific soft error resilience targets addressed in this paper are: Silent Data Corruption (SDC), where an error causes the system to output an incorrect result without error indication; and, Detected but Uncorrected Error (DUE), This research is supported in part by DARPA, DTRA, NSF, and SRC. The views expressed are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government.E. Cheng, S. Mirkhani, and S. Mitra are with Stanford University (e-mail: eccheng@ stanford.edu).H. Cho is w...

show abstract

Section: A Reliability Analysismentioning

confidence: 99%

Tolerating Soft Errors in Processor Cores Using CLEAR (Cross-Layer Exploration for Architecting Resilience)

Cheng

Mirkhani

Szafaryn

et al. 2018

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

View full text Add to dashboard Cite

show abstract

“…Despite a sizable amount of research into this topic, a consensus seems yet to have been established. A reasonable estimate appears to be that the frequency of SDC type errors is roughly an order of magnitude lower than that of errors leading to node failure [10,18]. On modern day clusters, DRAM memory and CPU caches are almost always protected at the architectural level using some form of error correction.…”

Section: Numerical Experimentsmentioning

confidence: 99%

Fault Tolerance in the Parareal Method

Nielsen

Hesthaven

2016

Proceedings of the ACM Workshop on Fault-Tolerance for HPC at Extreme Scale

View full text Add to dashboard Cite

Parallel-in-time integration is an often advocated approach for extracting parallelism in the solution of PDEs beyond what is possible using spacial domain decomposition techniques. Due to the comparatively low parallel efficiency of parallel-in-time integration techniques, they are primarily of interest as an extension for classical approaches at parallelism. As such, potential applications are expected to scale across several hundreds, or possibly thousands of nodes, making algorithmic resilience towards hardware induced errors highly relevant. In this work we develop a scheduling scheme for the parareal algorithm that is resilient to node-loss. The fault-tolerant scheme is based on a popular approach introduced by E. Aubanel in [1], modified with a set of MPI interface extensions for implementing recovery strategies available in the ULFM framework. In addition, we demonstrate how the parareal algorithm may be made resilient towards Silent-Data-Corruption (SDC) errors by viewing it as a point-iterative method, locally monitoring the residual between consecutive iterations so to discard potentially corrupt iterations.

show abstract

“…We use the FIT rates for crashes (DUEs) and SDCs of Michalak et al [29] for the Roadrunner supercomputer. Michalak et al obtained these rates via accelerated neutron-beam test.…”

Section: A the Estimation Of Failure Ratesmentioning

confidence: 99%

“…We use the benchmark FIT rates to calculate and specify the target reliability thresholds which are to be achieved by the App FIT heuristic. For instance, if the crash failure is 2.22 ×10 3 for 32 GBs as given in [29], then for 32 MB program input the crash failure would be 2.22, or for a task argument of 32 KB the crash failure would be 2.22 ×10 −3 . Finally a task's overall failure rates λ F (T ) and λ SDC (T ) are sum of all its arguments' failure rates respectively.…”

Section: A the Estimation Of Failure Ratesmentioning

confidence: 99%

“…Our framework is independent of the method for estimating/measuring the failure rates of each individual tasks or benchmarks. As stated above, we use DUE and SDC rates from the measurements of [29]. However, these rates can be obtained by any other methods such as the analysis of system failure (memory, storage, network) histories/logs or application/task-specific vulnerability analysis.…”

Section: A the Estimation Of Failure Ratesmentioning

confidence: 99%

See 1 more Smart Citation

A Runtime Heuristic to Selectively Replicate Tasks for Application-Specific Reliability Targets

Subaşi

Yalcin

Zyulkyarov

et al. 2016

2016 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Abstract-In this paper we propose a runtime-based selective task replication technique for task-parallel high performance computing applications. Our selective task replication technique is automatic and does not require modification/recompilation of OS, compiler or application code. Our heuristic, we call App FIT, selects tasks to replicate such that the specified reliability target for an application is achieved. In our experimental evaluation, we show that App FIT selective replication heuristic is low-overhead and highly scalable. In addition, results indicate that complete task replication is overkill for achieving reliability targets. We show that with App FIT, we can tolerate pessimistic exascale error rates with only 53% of the tasks being replicated.

show abstract

Assessment of the Impact of Cosmic-Ray-Induced Neutrons on Hardware in the Roadrunner Supercomputer

Cited by 36 publications

References 21 publications

Tolerating Soft Errors in Processor Cores Using CLEAR (Cross-Layer Exploration for Architecting Resilience)

Tolerating Soft Errors in Processor Cores Using CLEAR (Cross-Layer Exploration for Architecting Resilience)

Fault Tolerance in the Parareal Method

A Runtime Heuristic to Selectively Replicate Tasks for Application-Specific Reliability Targets

Contact Info

Product

Resources

About