Data distribution and tensor influence analysis of different clustering methods

Abstract

At present, people are in the era of big data, which is changing people's views of the world. However, it has the characteristics of various types, huge scale, and complex relationships. In order to solve the repeated calculation caused by streaming data in the processing of tensor-based big data, there will also be dimension disasters. Therefore, in this paper, an incremental tensor train decomposition (ITTD) method is proposed to solve multi-clustering problem in tensor-based big data analysis systems. It mainly uses results of the tensor train decomposition obtained from the original tensor to calculate and updates the results of tensor train decomposition to avoid the repetitive decomposition of the original tensor and enhance the decomposition efficiency. The performance of ITTD method is tested through theoretical analysis, a large number of simulation data and a comparative experiment on the real data of public transportation in a region. The experimental results indicate that the execution time of ITTD is significantly shorter than that of nonincremental tensor train decomposition(NTTD) with time. However, as time goes by, there is no obvious difference in the approximation error and storage space between the two.. This shows that, compared to that of the traditional nonincremental method, if the approximation error and storage space are close, the execution time of the incremental method will be greatly shortened. It can improve the processing efficiency of multi-clustering problems in the tensor-based big data analysis system.