Development of Data Cleaning and Integration Algorithm for Asset Management of Power System

Abstract

Asset management technology is rapidly growing in the electric power industry because utilities are paying attention to which of their aged assets should be replaced first. The global trend of asset management follows risk management that comprehensively considers the probability and consequences of failures. In the asset management system, the risk assessment algorithm operates by interfacing digital datasets from various legacy systems. In this study, among the various electric power assets, we consider transmission cable systems as a representative linear asset consisting of different segments. First, the configurations and characteristics of linear asset datasets are analyzed. Second, six types of data cleaning functions are proposed for extracting dirty data from the entire dataset. Third, three types of data integration functions are developed to simulate the risk assessment algorithm. This technique supports the integration of distributed asset data in various legacy systems into one dataset. Finally, an automatic data cleaning and integration system is developed and the algorithm could repeat the cleaning and integration process until data quality is satisfied. To evaluate the performance of the proposed system, an automatic cleaning process is demonstrated using actual legacy datasets.

A Review of Asset Management using Artificial Intelligence based Machine Learning models with applications for the Electric Power and Energy System

“…Also, a sensitive examination of the transition rate has been completed, and the lifetime of transformers is calculated using data from real service. A thorough analysis of the various AM strategies applied to power distribution/transmission systems was carried out by the authors in [13]. The AM approach is divided into two groups, such as time-based and activity-based, based on how distribution networks are planned and operated.…”

A review of asset management using artificial intelligence‐based machine learning models: Applications for the electric power and energy system

“…Also, a sensitive examination of the transition rate has been completed, and the lifetime of transformers has been calculated using data from real service. The authors thoroughly analysed the various AM strategies applied to power distribution/transmission systems in [14]. The AM approach is divided into time‐ and activity‐based groups based on how distribution networks are planned and operated.…”

Research on Data Cleaning Algorithm Based on Multi Type Construction Waste

“…After normalization, to realize the unification of the data format, the data was normalized, as shown in Equation (2) [17]. Finally, the normalized matrix Z was obtained, as shown in Equation (3).…”

“…Schaub et al used data mining and machine learning algorithms to fill in data and realize the automation of data cleaning, which markedly improved cleaning efficiency. Ma Fengshi et al used the mathematical statistical method of mathematical extreme value, and judged abnormal data according to the characteristics of abnormal data appearing in the extreme value, which markedly improved the accuracy of outlier detection [3]. Yang Dongqing solved the problem in the process of data conversion, and developed a data cleaning tool [4].…”