New Solution Generation Strategy to Improve Brain Storm Optimization Algorithm for Classification

: As a new intelligent optimization method, brain storm optimization (BSO) algorithm has been widely concerned for its advantages in solving classical optimization problems. Recently, an evolutionary classiﬁcation optimization model based on BSO algorithm has been proposed, which proves its effectiveness in solving the classification problem. However, BSO algorithm also has defects. For example, large-scale datasets make the structure of the model complex, which affects its classification performance. In addition, in the process of optimization, the information of the dominant solution cannot be well preserved in BSO, which leads to its limitations in classification performance. Moreover, its generation strategy is inefficient in solving a variety of complex practical problems. Therefore, we briefly introduce the optimization model structure by feature selection. Besides, this paper retains the brainstorming process of BSO algorithm, and embeds the new generation strategy into BSO algorithm. Through the three generation methods of global optimal, local optimal and nearest neighbor, we can better retain the information of the dominant solution and improve the search efficiency. To verify the performance of the proposed generation strategy in solving the classification problem, twelve datasets are used in experiment. Experimental results show that the new generation strategy can improve the performance of BSO algorithm in solving classification problems.


Introduction
Optimization problem is to solve the maximum (minimum) value of an objective function under certain constraints, which widely exists in scientific research and engineering applications. Data classification is a classic research topic in the field of machine learning, and its purpose is to extract a function or model from training samples to predict the label of unknown instances, which is similar to optimization problem. Some important classification methods such as support vector machine (SVM) [1], artificial neural network (ANN) [2], k-nearest neighbor (KNN) [3], decision tree (DT) [4], naive Bayesian classification (NBC) [5] have been proposed. These methods enrich the classification patterns and successfully solve various complex practical problems. However, many existing methods are fixed structures, and a considerable part of them are still fall into local optimal solutions. Swarm intelligence evolutionary algorithm uses evolutionary method to iteratively to find the global most accurate solution, which cannot be calculated by traditional methods. The classic evolutionary algorithms are particle swarm optimization (PSO) [6], genetic algorithm (GA) [7], artificial bee colony (ABC) [8], brain storm optimization (BSO) [9] and ant colony optimization (ACO) [10], etc. with traditional methods, swarm intelligence evolutionary algorithm has great advantages in algorithm performance and is widely used in solving practical problems. For example, Holden et al. [11] proposed a hybrid particle swarm optimization/ant colony optimization (PSO/ACO) algorithm to solve the classification problem. Cervantes et al. [12] introduced PSO algorithm into SVM classifier, which combining the best optimization technology and classification technology to achieve better classification performance. Dalal et al. [13] used genetic algorithm to optimize the kernel extreme learning machine, which has higher accuracy in classification of various forms of arrhythmia in Electrocardiogram. Therefore, the introduction of evolutionary algorithm into classification method can significantly improve the performance of classification method, but the improvement of evolutionary algorithm in classification method has limitations.
In the past few decades, evolutionary algorithms only improve the performance of classification by optimizing the parameters or structure of classifiers, and there are few other applications. Evolutionary classification optimization model is a new classification method proposed by Xue et al. [14], it uses BSO algorithm to solve the classification problem directly. This method verifies the effectiveness of evolutionary algorithm which can be directly used to solve the classification problem. However, Xue et al. [14] only focuses on how to construct the classification model, and the improvement of the classification performance of this method has not been involved too much. In addition, the experimental datasets used in [14] is still very limited, and the analysis of the method to solve the classification problem is not comprehensive enough. Therefore, the study of optimization evolutionary classification optimization model has a bright future.
The structure of evolutionary classification optimization model depends on the size of dataset dimension. Therefore, in large-scale datasets, complex structure may affect the performance of classification model. Feature selection is an effective way to solve such problems. For instance, Xue et al. [15] proposed an adaptive particle swarm optimization (SAPSO) algorithm for feature selection method, which achieves the ideal classification performance. Saravanan et al. [16] simplified feature subset obtained by feature selection is applied to monitor phishing websites, and its classification performance is superior to existing classifiers. Considering this, we introduce two feature selection methods to optimize the structure of the model. BSO algorithm is a new evolutionary algorithm proposed by Shi [17]. It has the ability of local search and global search, and has better optimization ability in solving practical problems. However, BSO also has a series of problems; for example, there is no direct interaction between different solutions. Thus, when solving the optimal solution problem, the information of other better solutions is not used well. Moreover, although BSO has the ability of local search and global search, there are still some deficiencies in the balance between global exploration and local exploration. In view of the shortcomings of BSO algorithm, most of the existing optimization work of BSO algorithm is limited to modifying parameters to improve its performance. Such as, Li et al. [18] designed a vector grouping learning BSO (VGLBSO) methods. The method used VGL schemes to generate individuals for improve population diversity to coordinate global exploration and local exploitation capabilities. Li et al. [19] also proposed a BSO algorithm with multiple information interactions (MIIBSO) which using multiple information interaction (MII) strategies to enhance various information interactions between individuals; Moreover, the original K-means algorithm of BSO algorithm is replaced by random grouping (RG) strategy, which further enhances the ability of information exchange between individuals, and introduced dynamic differential step size (DDS) for improve the search range and coordinate global exploration and local exploitation capabilities. Yang et al. [20] proposed a new method of combining BSO with chaotic local search (CLS), which has randomness and periodicity. The diversification of the CLS search method breaks the premature convergence of the BSO algorithm and maintains the diversity of the population, thus achieving a better balance between global exploration and local exploitation. These show that there is little research on the improvement of BSO search strategy. Therefore, this paper uses three generation methods: global optimization, local optimization and nearest neighbor to better retain the information of the dominant solution to improve the search efficiency and the performance of the algorithm. The improved algorithm is applied to the evolutionary classification optimization model, and is tested on 12 experimental datasets, which from UCI Machine Learning Repository.
The rest of the paper is organized as follows: Section 2 describes the outline of the BSO algorithm. Section 3 focuses on three generation strategies. Section 4 introduces the evolutionary optimization classification model. Section 5 introduces the design of experimental parameters and Section 6 gives the experimental results and analysis. Section 7 summarizes the research and prospects the future trend.

Brain Strom Optimization Algorithm
The BSO algorithm was proposed in 2011 by Shi [21], and it simulates the collective behavior of human beings to solve the problem by brainstorming. In BSO, each individual in the algorithm represents the solution of a potential problem, and the individual is updated through the evolution and fusion of individuals.
The implementation of the BSO algorithm is very simple: Firstly, n (population size) solutions are generated for the problem to be solved, and then, these n individuals are divided into m (preset parameters) clusters by clustering algorithm. Through the evaluation of n individuals, the individuals in each cluster are sorted, and the optimal individual in each cluster is selected as the central individual of the cluster. The central individual of a cluster is selected randomly, and whether it is replaced by a randomly generated individual is determined according to the probability value P . In the process of Finally, the new individuals are compared with the old ones in the current position, leaving the individuals with better fitness values.
The BSO algorithm framework is described in Algorithm 1:

New Solution Generation Strategy to Improve Brain Strom Optimization Algorithm (IBSO)
To improve the performance of BSO algorithm, researchers have proposed different improvement strategies. These improvements can be divided into three categories: individuals clustering, new individual generation and hybrid algorithm. Among them, the new individual generation strategy greatly affects the efficiency of BSO. By improving the new individual generation strategy, the search efficiency of BSO algorithm can also be improved. In order to solve different types of optimization problems, the algorithm needs to use the real-time search information adaptively. In the aspect of new individual generation, there are also a lot of research work that can be studied, such as modifying search step size and new individual generation method. In this paper, we use three new individual generation strategies, which have achieved good performance in previous applications.
3) Global optimal strategy: global optimal individuals guide the generation of new individuals, where optimal x is the global optimal individual obtained by comparing the cluster central individuals.
There is a probability value cluster P to decide whether the new individual is generated by the global optimal strategy or the others. If the new individual is not generated by the global optimal strategy, roulette is used to determine which cluster the new individual is generated from. In addition, the probability value one P used to select the cluster optimal strategy or the nearest neighbor strategy to generate new individuals.
The new individual generation strategy is described in Algorithm 2.

Evolutionary Classification Optimization Model
Given a dataset D , in which 70% of the samples are used as training sets T , write as: 11 where ( , ) i i x y is the th i samples, 1 2 , ,..., When we find such an approximate solution W , we can predict the label of the sample by the information of this sample. The process is as follows, and the δ an acceptable error range.  (10) In this paper, the evolutionary classification optimization model is implemented by BSO algorithm, which is described as classification based on BSO (CBSO).

Evolutionary Classification Optimization Model with Feature Selection
Feature selection is an effective way of data preprocessing, and is often used to solve classification problems. In this paper, feature selection is applied to this model in two ways.
(1) Taking the feature subset and weight parameters as a whole, and BSO algorithm is used to search for the optimal feature subset and weight parameters simultaneously. We define 1 represents that the corresponding feature is selected, otherwise, 0 represents that the corresponding feature is not selected. For example: in the Fig. 1, the orange part represents the feature subset part, and the green part represents the weight parameter part. Four random numbers between 0 and 1 correspond to four features and compared with 0.6 , it means that the first features are not selected and the other two feature are selected. In this case, the feature subset and weight parameters are regarded as an individual and BSO is used to find the optimal individual. This method is defined as FS-CBSO.
(2) The feature subset and weight parameters are regarded as two parts. BSO algorithm is used to search the optimal feature subset on the one hand, and on the other hand, it is used to search the optimal weight parameter under the corresponding feature subset. Firstly, BSO algorithm is used to generate the population of feature subsets. Different feature subsets represent different structures of the model. Secondly, under different structures, BSO algorithm is used to search the optimal weight parameters of each structure. In addition, each structure learns from each other and retains the optimal feature subset and its corresponding weight parameters. In this case, feature subset and weight parameter are considered as two parts to search separately, and the search of weight parameter depends on feature subset. This method is defined as CBSO-FS. The process of the two methods is shown in Fig. 2 and Fig. 3.  Table 1 70% instances of these dataset are randomly selected as the training datasets to learn the classification model, and the other 30% are used to test the prediction accuracy of the classification model after modeling.  DS1  bcw  683  9  2  DS2  wbcd  569  30  2  DS3  iono2  351  33  2  DS4  biodeg  1055  41  2  DS5  spect  267  44  2  DS6  sonar  208  60  2  DS7  MicroMass  360  1300  2  DS8  madelon  600  500  2  DS9  Musk1  476  66  2  DS10  ConnectionistBenchData 208  60  2  DS11 hill 606 100 2

Parameter Settings
The objective function used in this paper is

Experiment Results and Analysis
In order to prove the superiority of the new solution generation strategy, this section will compare with the original algorithm, and test the classification accuracy and solution size based on the evolutionary classification optimization model . The experimental results of CBSO, CBSO-FS, FS-CBSO,  IBSO, IBSO-FS and FS-IBSO are shown in Table 2 and Table 3. "mean" and "std" represent the average and standard deviation of classification accuracy (solution size) after the algorithm runs independently on each dataset for 30 times, respectively. Among them, on the same dataset, the results with the most average and standard deviation are represented in bold.    Table 2 shows the classification accuracy of each algorithm on the test set, and Table 3 shows the solution size of each algorithm on the test set. We will analyze the experimental results from classification accuracy and solution size.
Classification accuracy: As can be seen from Table 2, IBSO achieves the highest classification  accuracy on DS1 and DS5. IBSO-FS achieves the highest classification accuracy on DS2, DS7 and DS8. FS-IBSO achieves the highest classification accuracy on four datasets: DS4, DS6, DS10 and DS11. Both DS3 and DS9 achieved the highest classification accuracy on FS-CBSO. In general, the algorithm with new solution generation strategy achieves the best classification accuracy on eight datasets, and is inferior to the original algorithm only in two datasets.
Solution size: Because CBSO and IBSO do not use feature selection technology, it is not applicable (NA) in this section. It can be seen from Tab.3 that FS-IBSO has the minimum solution size on eight datasets, IBSO-FS has the minimum solution size on one dataset, and FS-CBSO has the minimum solution size on two datasets. In other words, the method with new solution generation strategy has better solution size on most datasets.
On the whole, the method with new solution generation strategy outperforms the original algorithm in classification accuracy and solution size. Therefore, it can be concluded that the new solution generation strategy to improve BSO algorithm for classification problem is feasible.

Conclusions and Future Work
In this paper, we propose the new solution generation strategy method to improve BSO algorithm. It introduces three new solution generation strategies into BSO to improve the classification performance of evolutionary classification optimization model. Experiments on 11 different datasets show that the method with new solution generation strategy has better classification performance and solution size. This means that the new solution generation strategy to improve the BSO algorithm for solving classification problems is effective.
In this paper, we have done a lot of research and optimization on the modeling of classification problem and the improvement of BSO algorithm. However, there are still some problems and need to be further explored. The experimental datasets selected in this paper have only two class labels, but the research on the problem of multiple class labels has not been carried out. In addition, the classification performance has not reached our ideal result; we need to further explore better algorithm improvement methods.