Mass data processing and multidimensional database management based on deep learning

Abstract

With the rapid development of the Internet of Things, the requirements for massive data processing technology are getting higher and higher. Traditional computer data processing capabilities can no longer deliver fast, simple, and efficient data analysis and processing for today’s massive data processing due to the real-time, massive, polymorphic, and heterogeneous characteristics of Internet of Things data. Mass heterogeneous data of different types of subsystems in the Internet of Things need to be processed and stored uniformly, so the mass data processing method is required to be able to integrate multiple different networks, multiple data sources, and heterogeneous mass data and be able to perform processing on these data. Therefore, this article proposes massive data processing and multidimensional database management based on deep learning to meet the needs of contemporary society for massive data processing. This article has deeply studied the basic technical methods of massive data processing, including MapReduce technology, parallel data technology, database technology based on distributed memory databases, and distributed real-time database technology based on cloud computing technology, and constructed a massive data fusion algorithm based on deep learning. The model and the multidimensional online analytical processing model of the multidimensional database based on deep learning analyze the performance, scalability, load balancing, data query, and other aspects of the multidimensional database based on deep learning. It is concluded that the accuracy of multidimensional database query data is as high as 100%, and the accuracy of the average data query time is only 0.0053 s, which is much lower than the general database query time.

Convolutional neural networks applied to data organized as OLAP cubes

“…With a CNN, it is possible to obtain effective representations of images, enabling the recognition of low‐level patterns, such as simple geometric shapes, and even high‐level patterns, such as human faces (Gu et al, 2018; Yamashita et al, 2018). CNN has also been used for massive data processing and multidimensional database management (Shen et al, 2022).…”

“…Tariq et al (2022) uses the 2-D convolutional neural network to predict the churned users. Shen et al (2022) proposes massive data processing and multidimensional database management based on CNN. Although CNN has obtained good results, there are still problems in the data that make classification difficult, as highlighted by Lee and Bang (2021), who propose the combination of under-sampling and CNN for the classification of imbalanced structured data.…”