Open Access

ARTICLE

Variable Importance Measure System Based on Advanced Random Forest

Shufang Song^1,*, Ruyang He¹, Zhaoyin Shi¹, Weiya Zhang²
1 School of Aeronautics, Northwestern Polytechnical University, Xi’an, 710072, China
2 AECC Sichuan Gas Turbine Establishment, Mianyang, 621700, China
* Corresponding Author: Shufang Song. Email:

Computer Modeling in Engineering & Sciences 2021, 128(1), 65-85. https://doi.org/10.32604/cmes.2021.015378

Received 14 December 2020; Accepted 17 March 2021; Issue published 28 June 2021

Abstract

The variable importance measure (VIM) can be implemented to rank or select important variables, which can effectively reduce the variable dimension and shorten the computational time. Random forest (RF) is an ensemble learning method by constructing multiple decision trees. In order to improve the prediction accuracy of random forest, advanced random forest is presented by using Kriging models as the models of leaf nodes in all the decision trees. Referring to the Mean Decrease Accuracy (MDA) index based on Out-of-Bag (OOB) data, the single variable, group variables and correlated variables importance measures are proposed to establish a complete VIM system on the basis of advanced random forest. The link of MDA and variance-based sensitivity total index is explored, and then the corresponding relationship of proposed VIM indices and variance-based global sensitivity indices are constructed, which gives a novel way to solve variance-based global sensitivity. Finally, several numerical and engineering examples are given to verify the effectiveness of proposed VIM system and the validity of the established relationship.

---

Keywords

Variable importance measure; random forest; variance-based global sensitivity; Kriging model

---

Citations