Performance Optimization Technology of Fault Tolerance Mechanism in Distributed System Based on Neural Network

Abstract

In the era of big data, distributed systems(DS) have to deal with more and more data, and the possibility of system hardware failure is higher and higher. Traditional disk arrays can no longer meet the high reliability requirements of large-scale distributed clusters. Data fault tolerance has become an important problem in DSs. In order to ensure the reliability of data processing in DSs and continue to provide users with high-quality services, the performance optimization of fault-tolerant mechanisms has become an important research content in DSs. Aiming at the distributed core storage framework proposed in this paper, the fault-tolerant mechanism of its client module and metadata storage management module is designed and implemented, and the performance of the DS is tested through the performance optimization experiment of fault-tolerant mechanism. The experimental results of Prime-based and neural network-based data inpainting show that the neural network-based data inpainting technology can reduce the network cost of data inpainting. The experimental results of distributed computing delay based on radial basis neural network algorithm, C.T. algorithm and Pedone algorithm show that the delay time of RBF algorithm tends to converge with the increase of MRR.