Open Access

ARTICLE

Performance Enhancement of Adaptive Neural Networks Based on Learning Rate

Swaleha Zubair¹, Anjani Kumar Singha¹, Nitish Pathak², Neelam Sharma³, Shabana Urooj^4,*, Samia Rabeh Larguech⁴
1 Department of Computer Science, Aligarh Muslim University, Aligarh, 202002, Uttar Pradesh, India
2 Department of Information Technology, Bhagwan Parshuram Institute of Technology (BPIT), GGSIPU, New Delhi, 110089, India
3 Department of Computer Science and Engineering, Maharaja Agrasen Institute of Technology (MAIT), GGSIPU, New Delhi, India
4 Department of Electrical Engineering, College of Engineering, Princess Nourah bint Abdulrahman University, P.O.Box 84428, Riyadh, 11671, Saudi Arabia
* Corresponding Author: Shabana Urooj. Email:

Computers, Materials & Continua 2023, 74(1), 2005-2019. https://doi.org/10.32604/cmc.2023.031481

Received 19 April 2022; Accepted 15 June 2022; Issue published 22 September 2022

Abstract

Deep learning is the process of determining parameters that reduce the cost function derived from the dataset. The optimization in neural networks at the time is known as the optimal parameters. To solve optimization, it initialize the parameters during the optimization process. There should be no variation in the cost function parameters at the global minimum. The momentum technique is a parameters optimization approach; however, it has difficulties stopping the parameter when the cost function value fulfills the global minimum (non-stop problem). Moreover, existing approaches use techniques; the learning rate is reduced during the iteration period. These techniques are monotonically reducing at a steady rate over time; our goal is to make the learning rate parameters. We present a method for determining the best parameters that adjust the learning rate in response to the cost function value. As a result, after the cost function has been optimized, the process of the rate Schedule is complete. This approach is shown to ensure convergence to the optimal parameters. This indicates that our strategy minimizes the cost function (or effective learning). The momentum approach is used in the proposed method. To solve the Momentum approach non-stop problem, we use the cost function of the parameter in our proposed method. As a result, this learning technique reduces the quantity of the parameter due to the impact of the cost function parameter. To verify that the learning works to test the strategy, we employed proof of convergence and empirical tests using current methods and the results are obtained using Python.

---

Keywords

Deep learning; optimization; convergence; stochastic gradient methods

---