An Application of Automatic Text Revision for Power Defect Log

Abstract

There are a large number of unstructured texts without data cleaning in the field of electric power. It is expensive to rely on manual ways to process the large amount of text data. In order to reduce the workload of data cleaning, we propose an intelligent method of automatic revision of power defect logs in this paper by adopting natural language processing technologies. We utilize entity recognition technology to recognize electrical equipment words on the text and utilize word similarity calculation to find out words with similar meaning in the standard vocabulary, which is the main process to revise Abnormal text. With the outstanding performance of entity recognition, the workload of data cleaning is reduced approximately 70% through our proposed method, which greatly improves the efficiency of unstructured data processing.

A Model for Enhancing Unstructured Big Data Warehouse Execution Time

“…By applying data cleaning techniques and algorithms, the paper obtains well-structured data that can be used for further analysis and visualization. The work presented in [53] proposed an approach for the automated modification of power failure records using natural language processing technology. This approach reduced the workload of data cleaning and significantly enhanced the efficiency of processing unstructured data.…”