Power Big Data Analysis Platform Design Based on Hadoop

Abstract

To better analyze and process power data to obtain effective information, a power big data analysis scheme based on Hadoop architecture is proposed. We analyze the cloud computing environment Hadoop distributed platform to obtain massive data of large-scale distributed power system. According to the characteristics of intelligent power consumption data, common data mining algorithm modules such as parallel classification algorithm and parallel real-time clustering algorithm are designed, and the implementation of clustering algorithm with different principles is further analyzed. Then, HBase is adopted to access data in a distributed way, and MapReduce is used to realize the visual management of power GIS data. The experimental results show that the parallel processing method of power big data based on Hadoop has high efficiency and good scalability, and the algorithm has good identification ability for massive data in the cluster mode.