Haijie Pan, Lirong Zheng*
CMES-Computer Modeling in Engineering & Sciences, Vol.131, No.1, pp. 493-512, 2022, DOI:10.32604/cmes.2022.019069
Abstract The machine learning model converges slowly and has unstable training since large variance by random using a sample estimate gradient in SGD. To this end, we propose a noise reduction method for Stochastic Variance Reduction gradient (SVRG), called N-SVRG, which uses small batches samples instead of all samples for the average gradient calculation, while performing an incremental update of the average gradient. In each round of iteration, a small batch of samples is randomly selected for the average gradient calculation, while the average gradient is updated by rounding of the past model gradients during internal iterations. By suitably reducing the… More >
Login
Register
