Open Access iconOpen Access

ARTICLE

crossmark

Deep Neural Network Architecture Search via Decomposition-Based Multi-Objective Stochastic Fractal Search

Hongshang Xu1, Bei Dong1,2,*, Xiaochang Liu1, Xiaojun Wu1,2

1 School of Computer Science, Shaanxi Normal University, Xi’an, 710119, China
2 Key Laboratory of Modern Teaching Technology, Ministry of Education, Xi’an, 710062, China

* Corresponding Author: Bei Dong. Email: email

(This article belongs to this Special Issue: Artificial Intelligence Algorithm for Industrial Operation Application)

Intelligent Automation & Soft Computing 2023, 38(2), 185-202. https://doi.org/10.32604/iasc.2023.041177

Abstract

Deep neural networks often outperform classical machine learning algorithms in solving real-world problems. However, designing better networks usually requires domain expertise and consumes significant time and computing resources. Moreover, when the task changes, the original network architecture becomes outdated and requires redesigning. Thus, Neural Architecture Search (NAS) has gained attention as an effective approach to automatically generate optimal network architectures. Most NAS methods mainly focus on achieving high performance while ignoring architectural complexity. A myriad of research has revealed that network performance and structural complexity are often positively correlated. Nevertheless, complex network structures will bring enormous computing resources. To cope with this, we formulate the neural architecture search task as a multi-objective optimization problem, where an optimal architecture is learned by minimizing the classification error rate and the number of network parameters simultaneously. And then a decomposition-based multi-objective stochastic fractal search method is proposed to solve it. In view of the discrete property of the NAS problem, we discretize the stochastic fractal search step size so that the network architecture can be optimized more effectively. Additionally, two distinct update methods are employed in step size update stage to enhance the global and local search abilities adaptively. Furthermore, an information exchange mechanism between architectures is raised to accelerate the convergence process and improve the efficiency of the algorithm. Experimental studies show that the proposed algorithm has competitive performance comparable to many existing manual and automatic deep neural network generation approaches, which achieved a parameter-less and high-precision architecture with low-cost on each of the six benchmark datasets.

Keywords


Cite This Article

H. Xu, B. Dong, X. Liu and X. Wu, "Deep neural network architecture search via decomposition-based multi-objective stochastic fractal search," Intelligent Automation & Soft Computing, vol. 38, no.2, pp. 185–202, 2023.



cc This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
  • 212

    View

  • 80

    Download

  • 0

    Like

Share Link