A Parallel Computing Method for Entity Recognition based on MapReduce

Abstract

Abstract. With the rapid development of industrial automation, there are huge amounts of duplicate data refer to the same entity in the data sets have brought enormous challenges in data analysis. To accommodate the entity recognition of huge amounts of data, this paper presents a parallel computing method for entity recognition based on MapReduce. Through the detailed introduction to the MapReduce framework, running the applications on the Hadoop platform and parallel processing data sets to recognize the data entities. The experiments show that the proposed method greatly enhanced the recognition speed and accuracy, which has better effectiveness to meet the demand for entity recognition than other methods.