Evaluation Platform for DDM Algorithms With the Usage of Non-Uniform Data Distribution Strategies

Abstract

Huge amounts of data are collected in numerous independent data storage facilities around the world. However, how the data is distributed between physical locations remains unspecified. Downloading all of the data for the purpose of processing it is undesirable and sometimes even impossible. Various methods have been proposed for performing data mining tasks, but the main problem is the lack of an objective strategy for comparing them. The authors present current research on a novel evaluation platform for distributed data mining (DDM) algorithms. The proposed platform opens up a new field to evaluate algorithms in terms of the quality of the results, transfer used, and speed, but also for the use of a non-uniform data distribution among independent nodes during algorithm evaluation. This work introduces a ‘data partitioning strategy’ term referring to a specific, not necessarily uniform data distribution. A brief evaluation for three clustering algorithms is also reported, showing the usability and simplicity of identifying differences in processing with the use of the platform.

Image Identification and Error Correction Method for Test Report Based on Deep Reinforcement Learning and IoT Platform in Smart Laboratory

“…In the experiment, Python 3.4 was used to implement the algorithm model, and Tensorflow function module was used to develop a deep learning network. Use Python matplotlib 3.1 library to realize data and information visualization, with 64 GB of memory, CPU i7 4790, GPU GTX1070, and CUDA 9.0 and CUDNN 7.1 to speed up the calculation (Markiewicz & Koperwas, 2022). Parameters for the DRL network are set as follows: CNN filter is 3×3, quantity is 32, step size is 1, and pool size is 2×2.…”

The Construction of a Fire Monitoring System Based on Multi-Sensor and Neural Network

“…The traditional -sensor signal processing method usually adopts a simple mathematical algorithm, and the low-level data processing of multi-sensor information only describes the measured object from multiple isolated sides (Markiewicz & Koperwas, 2022). There are essential differences between multi-sensor data fusion and classical signal processing methods.…”

Application of Automatic Completion Algorithm of Power Professional Knowledge Graphs in View of Convolutional Neural Network

“…Its division is based on different aggregation methods. The former one extracts the local highest value, whereas the latter extracts the local mean value (Markiewicz & Koperwas, 2022). It also belongs to the convolution process.…”