Hard drive failure prediction using Decision Trees

Li, Jing; Stones, Rebecca J.; Wang, Gang; Liu, Xiaoguang; Li, Zhongwei; Xu, Ming

doi:10.1016/j.ress.2017.03.004

Cited by 61 publications

(19 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lin et al [47] and Li et al [42] predict node failures in large-scale cloud computing platforms by building machine learning models from temporal (e.g., CPU utilization metrics), spatial (e.g., location of a node), and config data (e.g., build data). Similarly, Li et al [40,41] build tree-based models to predict hard drive failures. Botezaku et al [11], Mahdisoltani et al [56], and Xu et al [82] leverage SMART-based analysis to build a machine learning pipeline for predicting hard drive failures in large-scale cloud computing platforms.…”

Section: Aiops Applicationsmentioning

confidence: 99%

Towards a Consistent Interpretation of AIOps Models

LyuYingzhe

Rajbahadur

Lin

et al. 2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Artificial Intelligence for IT Operations (AIOps) has been adopted in organizations in various tasks, including interpreting models to identify indicators of service failures. To avoid misleading practitioners, AIOps model interpretations should be consistent (i.e., different AIOps models on the same task agree with one another on feature importance). However, many AIOps studies violate established practices in the machine learning community when deriving interpretations, such as interpreting models with suboptimal performance, though the impact of such violations on the interpretation consistency has not been studied. In this article, we investigate the consistency of AIOps model interpretation along three dimensions: internal consistency, external consistency, and time consistency. We conduct a case study on two AIOps tasks: predicting Google cluster job failures and Backblaze hard drive failures. We find that the randomness from learners, hyperparameter tuning, and data sampling should be controlled to generate consistent interpretations. AIOps models with AUCs greater than 0.75 yield more consistent interpretation compared to low-performing models. Finally, AIOps models that are constructed with the Sliding Window or Full History approaches have the most consistent interpretation with the trends presented in the entire datasets. Our study provides valuable guidelines for practitioners to derive consistent AIOps model interpretation.

show abstract

Section: Aiops Applicationsmentioning

confidence: 99%

Towards a Consistent Interpretation of AIOps Models

LyuYingzhe

Rajbahadur

Lin

et al. 2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…Previous hard drive failure prediction methods [6][7][8][9][10][11][12][13][14] primarily use a single snapshot with SMART characteristics as the prediction input example to evaluate the health status of hard drive and predict the failure of a hard drive, without considering the dependence of different health conditions of the hard drive in the time range. However, hard drives usually do not fail suddenly but do so gradually.…”

Section: Related Workmentioning

confidence: 99%

“…The previous hard drive failure prediction [6][7][8][9][10][11][12][13][14] mostly takes a single snapshot with self-monitoring, analysis, and reporting technology (SMART) characteristics as the prediction input example. However, hard drive failure is not achieved instantly but is a time-varying process of the gradual decline of hard drive health.…”

Section: Introductionmentioning

confidence: 99%

Failure Prediction for Temporal Dependency of Hard Drives

2021

Proceedings of 2021 the 11th International Workshop on Computer Science and Engineering

View full text Add to dashboard Cite

Cloud service providers strive to predict hard drives' failure in advance to enhance user confidence in cloud storage resources. We explored the failure property from the self-monitoring, analysis, and reporting technology features of hard drives, finding that the long-term temporal changepoint dependency (LTCD) of hard drive failure creates new reconstruction challenges in failure prediction. The failure prediction for temporal dependency (FPTD) presented in this paper has three characteristics: primary identifying features and changepoint features and enhancing changepoint dependency, all of which make FPTD more sensitive to the failure of hard drives with LTCD. The experimental results show that the five evaluation metrics of FPTD are all above 94% while maintaining a low false alarm rate, among which Accuracy can reach 99.0% and Recall can reach 97.6% on average. In general, the FPTD has higher prediction quality and better stability, and is more suitable for predicting hard drive failures in the long-short temporal.

show abstract

“…The premise of applying DT is that the occurrence probability of various situations must be known. When there are too many categories, the probability of errors will increase 8,9 . In addition, Petri net (PNs) 10 and multi‐agent system (MAS) 11 have also been adopted to express the relationships of equipment faults.…”

Section: Introductionmentioning

confidence: 99%

Two‐layer model of equipment fault propagation in manufacturing system

Huang

2020

Quality & Reliability Eng

View full text Add to dashboard Cite

Equipment fault is a major factor that affects the reliability of manufacturing systems. Fault source identification and propagation path search are primary means to reduce and eliminate equipment faults. To this end, this paper proposed a two‐layer model of equipment fault propagation in a manufacturing system. One layer, modeled by a small‐world network, is a network of parts for a single equipment device. The other, represented by the production relationships of the manufacturing process, is a network of equipment devices for a manufacturing system. Hence, the propagation path of an equipment fault is divided into physical propagation inside a single equipment device and flow propagation within the manufacturing process. The intensity of physical propagation is given by the product of the fault load and the propagation probability between fault nodes within a single equipment device, whereas the flow propagation intensity is equivalent to the yield loss of the manufacturing system due to equipment fault. Therefore, the total intensity of fault propagation in a manufacturing system is given by the product of these two intensities. Subsequently, the ant colony algorithm is used to find the propagation paths with the maximum intensity. Finally, the proposed method is verified using an example of a camshaft manufacturing system.

show abstract

Hard drive failure prediction using Decision Trees

Cited by 61 publications

References 22 publications

Towards a Consistent Interpretation of AIOps Models

Towards a Consistent Interpretation of AIOps Models

Failure Prediction for Temporal Dependency of Hard Drives

Two‐layer model of equipment fault propagation in manufacturing system

Contact Info

Product

Resources

About