N-SVRG: Stochastic Variance Reduction Gradient with Noise Reduction Ability for Small Batch Samples

Abstract

The machine learning model converges slowly and has unstable training since large variance by random using a sample estimate gradient in SGD. To this end, we propose a noise reduction method for Stochastic Variance Reduction gradient (SVRG), called N-SVRG, which uses small batches samples instead of all samples for the average gradient calculation, while performing an incremental update of the average gradient. In each round of iteration, a small batch of samples is randomly selected for the average gradient calculation, while the average gradient is updated by rounding of the past model gradients during internal iterations. By suitably reducing the batch size B, the memory storage as well as the number of iterations can be reduced. The experiments are compared with the state-of-the-art Mini-Batch SGD, AdaGrad, RMSProp, SVRG and SCSG, and it is demonstrated that N-SVRG outperforms SVRG and SASG, and is on par with SCSG. Finally, by exploring the relationship between the small values of different parameters n. B and k and the effectiveness of the algorithm, we prove that our N-SVRG algorithm has some stability and can achieve sufficient accuracy even in the case of small batch size. The advantages and disadvantages of various methods are experimentally compared, and the stability of N-SVRG is explored by parameter settings.

[Retracted] Higher Education Environment Monitoring and Quality Assessment Model Using Big Data Analysis and Deep Learning

“…The same transaction is stored only once in a matrix, and a matrix is created to store the number of iterations of the transaction to achieve the compression of transaction data. Calculate the support of each component according to formula (10), and screen the components that exceed the minimum support.…”

“…The next stage is to increase text data collecting and analysis based on the data already available and work to more accurately reflect the current state of education quality. (2) Combined with big data mining technology [ 10 ], the education quality monitoring model is improved. Constantly revise the core indicators of education quality, and improve the weight of each indicator through more sample data.…”

[Retracted] Ethnic Music Inheritance and Environmental Monitoring Using Big Data Analysis from the Cultural Perspective

“…People can discover other hidden rules from these vast amounts of data through analysis and processing, which will aid in the evolution of the entire society. Big data application technologies [ 7 , 8 ], such as different big data platforms, big data index systems, and other big data application technologies, are referred to as big data technology. On the one hand, big data technology is connected to a variety of industries, and its application promotes the general advancement of society.…”