Fault prediction under the microscope: A closer look into HPC systems

Gainaru, Ana; Cappello, Franck; Snir, Marc; Kramer, William

doi:10.1109/sc.2012.57

Cited by 75 publications

(61 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Providers continuously attempt to improve datacenter energy-efficiency; as stated in [2], a substantial amount of energy-waste is due to failures. First, it is ideal that dependability mechanisms deployed in datacenters do not significantly degrade the energyefficiency of the system.…”

Section: Application Of Workmentioning

confidence: 99%

“…According to the National Institute of Standards and Technology (NIST) [1], Cloud computing is defined as "a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction". The failure characteristics of such environments are of particular concern as failures can result in degradation of Quality of Service (QoS), availability, reliability and energywaste [2] that can ultimately lead to economic loss for both Cloud consumers and providers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Failure-Analysis of a Large-Scale Cloud Computing Environment

Garraghan

Townend

2014

2014 IEEE 15th International Symposium on High-Assurance Systems Engineering

View full text Add to dashboard Cite

Abstract-Cloud computing research is in great need of statistical parameters derived from the analysis of real-world systems. One aspect of this is the failure characteristics of Cloud environments composed of workloads and servers; currently, few metrics are available that quantify failure and repair times of workloads and servers at a large-scale. Workload metrics in particular are critical for characterizing and modeling accurate workload behavior, enabling more realistic workload simulation and failure scenarios of systems. This paper presents the analysis of failure data of a large-scale production Cloud environment (consisting of over 12,500 servers), and includes a study of failure and repair times and characteristics for both Cloud workloads and servers. Our results show that failure characteristics for workload and servers are highly variable and that production Cloud workloads can be accurately modeled by a Gamma distribution. Repair times range between 30 seconds to 4 days, and 25 minutes to 8 days, for workloads and servers respectively.

show abstract

Section: Application Of Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Empirical Failure-Analysis of a Large-Scale Cloud Computing Environment

Garraghan

Townend

2014

2014 IEEE 15th International Symposium on High-Assurance Systems Engineering

View full text Add to dashboard Cite

show abstract

“…The prediction of failures itself, however, was still an open issue. Recent results from the University of Illinois at Urbana-Champaign [54][55][56] and the Illinois Institute of Technology [101,102] clearly demonstrate the feasibility of error prediction for different systems: the Blue Waters CRAY system based on AMD processors and NVIDIA GPUs and the Blue Gene system based on IBM proprietary components. Failure prediction techniques have progressed by combining data mining with signal analysis and methods to spot outliers.…”

Section: Failure Predictionmentioning

confidence: 99%

Toward Exascale Resilience: 2014 update

Cappello

Geist

Gropp

et al. 2014

JSFI

View full text Add to dashboard Cite

Resilience is a major roadblock for HPC executions on future exascale systems. These systems will typically gather millions of CPU cores running up to a billion threads. Projections from current large systems and technology evolution predict errors will happen in exascale systems many times per day. These errors will propagate and generate various kinds of malfunctions, from simple process crashes to result corruptions.The past five years have seen extraordinary technical progress in many domains related to exascale resilience. Several technical options, initially considered inapplicable or unrealistic in the HPC context, have demonstrated surprising successes. Despite this progress, the exascale resilience problem is not solved, and the community is still facing the difficult challenge of ensuring that exascale applications complete and generate correct results while running on unstable systems. Since 2009, many workshops, studies, and reports have improved the definition of the resilience problem and provided refined recommendations. Some projections made during the previous decades and some priorities established from these projections need to be revised. This paper surveys what the community has learned in the past five years and summarizes the research problems still considered critical by the HPC community.

show abstract

“…To develop better solutions and reliability techniques, it is important to understand and characterize failure behavior. Indeed, failure characteristics can be used to inform failure predictors [8,2], or to improve fault-tolerance techniques [11,10,22].…”

Section: Related Workmentioning

confidence: 99%

Assuming Failure Independence: Are We Right to be Wrong?

Aupy

Robert

Vivien

2017

2017 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

This report revisits the failure temporal independence hypothesis which is omnipresent in the analysis of resilience methods for HPC. We explain why a previous approach is incorrect, and we propose a new method to detect failure cascades, i.e., series of non-independent consecutive failures. We use this new method to assess whether public archive failure logs contain failure cascades. Then we design and compare several cascadeaware checkpointing algorithms to quantify the maximum gain that could be obtained, and we report extensive simulation results with archive and synthetic failure logs. Altogether, not only are there but a few logs that contain cascades, but we show that the gain that can be achieved from this knowledge is not significant. The conclusion is that we can wrongly, but safely, assume failure independence!

show abstract

Fault prediction under the microscope: A closer look into HPC systems

Cited by 75 publications

References 21 publications

An Empirical Failure-Analysis of a Large-Scale Cloud Computing Environment

An Empirical Failure-Analysis of a Large-Scale Cloud Computing Environment

Toward Exascale Resilience: 2014 update

Assuming Failure Independence: Are We Right to be Wrong?

Contact Info

Product

Resources

About