FENCE: Fast, ExteNsible, and ConsolidatEd Framework for Intelligent Big Data Processing

Abstract

The proliferation of smart devices and the advancement of data-intensive services has led to explosion of data, which uncovers massive opportunities as well as challenges related to real-time analysis of big data streams. The edge computing frameworks implemented over manycore systems can be considered as a promising solution to address these challenges. However, in spite of the availability of modern computing systems with a large number of processing cores and high memory capacity, the performance and scalability of manycore systems can be limited by the software and operating system (OS) level bottlenecks. In this work, we focus on these challenges, and discuss how accelerated communication, efficient caching, and high performance computation can be provisioned over manycore systems. The proposed Fast, ExteNsible, and ConsolidatEd (FENCE) framework leverages the availability of a large number of computing cores and overcomes the OS level bottlenecks to provide high performance and scalability for intelligent big data processing. We implemented a prototype of FENCE and the experiment results demonstrate that FENCE provides improved data reception throughput, read/write throughput, and application processing performance as compared to the baseline Linux system.

Construction of Intelligent Search Engine for Big Data Multimedia Resource Subjects Based on Partial Least Squares Structural Equation

“…In the mode C partial least squares method, in the iterative calculation process, it uses a part of the weight relationship of mode A and mode B respectively. According to the different parts of the selected weight relationship, the researchers divide the Mode C partial least squares method into two categories: One is called mode 21 C partial least squares method; the other is called mode 12 C partial least squares method.…”

“…2) Mode 21 C partial least squares method Mode 12 C partial least squares is an iterative calculation, the formula (4) of mode A and formula (7) of mode B are respectively taken as the iterative calculation formula. Therefore, its mathematical formula specification is as follows:…”

“…In this part of the verification, the corresponding order of magnitude is measured by selecting the number of databases in the big data management platform of Siping Fire Bureau, 5G only corresponds to basic database, 100G corresponds to basic database + office database + public service library, 25T corresponds to system platform + data management and control platform, 50P corresponds to data management and control platform + system platform + business platform, the comparison of the search results is shown in Table 1. As can be seen from Table 1, with the continuous increase of the data order of magnitude, the advantages of the big data intelligent search engine based on the partial least squares structural equation algorithm are more obvious, there are significant improvements in retrieval time, precision and recall [12].…”