A Double Adaptive Random Spare Reinforced Sine Cosine Algorithm

1  Introduction

Recently, an enormous number of meta-heuristics algorithms have been proposed which simulate natural-based phenomena or biological behavior [1–6]. These meta-heuristics have been successfully applied to various domains of medical science [7–10], finance [11–13], energy [14–16], engineering [17–21], agriculture [22,23] and education [24–26]. Some examples of these algorithms are Artificial Bee Colony (ABC) [27], Genetic Algorithm (GA) [28], Ant Colony Optimization (ACO) [29], Particle Swarm Optimization (PSO) [30], Cuckoo Search (CS) [31,32], Grey Wolf Optimizer [33], Bat Algorithm (BA) [34], Runge Kutta Optimizer (RKN) [35], Colony Predation Algorithm (CPA) [36], Crow Search Algorithm (CSA) [37], Weighted Mean of Vectors [38], Snake Optimizer (SO) [39], Hunger Games Search (HGS) [40], Whale Optimization Algorithm (WOA) [41], Symbiotic Organism Search (SOS) [42], Lightning Search Algorithm (LSA) [43], Moth-Flame Optimization (MFO) [44], Slime Mould Algorithm (SMA) [45], Ant Lion Optimizer (ALO) [46], Harris Hawks Optimization (HHO) [47,48], Remora Optimization Algorithm (ROA) [49], and Wild Horse Optimizer (WHO) [50].

Sine Cosine Algorithm (SCA) is a novel population-based algorithm that has been introduced to solve real-world problems. For example, Elaziz et al. [51] developed the use of SCA to optimize the optimal parameters of Random Vector Functional Chain (RVFC) networks and use the obtained optimal model for predicting ACE inhibitory activity. Li et al. [52] developed a novel SCA-SVR model using the SCA in order to select the penalty & parameters of the kernel in SVR to improve the performance of generalization to unknown data. The model was able to have the optimal values of the SVR parameters with good results. Nayak et al. [53] introduced an enhanced Extreme Learning Machine (ELM) approach for the sine cosine algorithm (MSCA-ELM) and designed a new automated Magnetic Resonance Imaging (MRI) brain pathology detection system. Wang et al. [54] introduced a hybrid wavelet neural network based on the proposed Multi-Objective Sine Cosine Algorithm (MOSCA), which was successfully applied in wind speed prediction. Dasgupta et al. [55] introduced a newly designed SCA for determining the optimal voltage regulator systems of proportional-integral differential controller parameters, which can effectively tune the PID controller parameters. Chen et al. [16] combined opposition-based learning and local search strategies with SCA to estimate photovoltaic models parameters. Also, Sahlol et al. [56] tried to improve fish liver Enzymes prediction by using SCA to train feedforward neural networks. Also, Sindhu et al. [57] used an elitism strategy with SCA to solve the feature selection problem. Likewise, SCA has been applied to other problems e.g., predicting cervical hyperextension injury [58], predicting master students’ intention [26], hydrothermal scheduling [59], and image segmentation [60].

Many variants of SCA have been proposed such as a multi-objective SCA (MO-SCA) which was proposed by Tawhid and Savsani [61]. Also, Issa et al. [62] tried to design an adaptive SCA version that is combined with PSO called (ASCA-PSO). Likewise, Sine Cosine Crow Search Algorithm (SCSCA) [63], hybrid algorithms between sine cosine and particle swarm algorithm known as (H-PSO-SCAC) [64], hybrid algorithm between SCA and differential evolution (SCA-DE) [65], Opposition-Based SCA (OBSCA) [66], enhanced sine cosine algorithm called (MSCA) [67], and orthogonal-learning driven multi-swarm sine cosine called (OMGSCA) [68]. Abd Elaziz et al. [68] presented an improved version of SCA that incorporated backward learning to increase the exploration search space to generate more accurate solutions. Guo et al. [69] presented a Riesz derivative fractional sinusoidal algorithm abbreviated as that is based on Riesz fractional derivative variation mechanism and applyed it to the design of welded beams and pressure vessels. Gupta et al. [70] developed a novel and improved sine cosine algorithm using cross-exploitation techniques with the integration of individual optimal states of individual solutions, global search, and self-learning mechanisms and applied it effectively to multi-stage threshold segmentation in images. Gupta et al. [71] developed an improved SCA that is based on the logarithmic and adaptive components of the perturbation rate. Their algorithm was employed in order to solve five different engineering optimization problems and effectively solve the global optimization problem. Issa et al. [64] introduced a modified SCA version in which SCA was combined with PSO. This algorithm was used to solve the pairwise local alignment problem with the aim of looking for the longest continuous substring between two biological sequences, which has shown good performance in accuracy and calculation time. Long et al. [72] introduced inertial weights and a novel SCA version known as (ISCA) and applied it in solving high dimensional problems based on a Gaussian function with a nonlinear transformation parameter decreasing strategy. Compared to traditional SCA and other population-based algorithms, the ISCA algorithm has faster convergence and better ability to escape from local optimal solutions. Nenavath et al. [67] proposed a new differential evolutionary sine and cosine mixture optimization algorithm and confirmed its accuracy under various challenging conditions.

Although the success of all mentioned variants of SCA, it still has low convergence and may struggle in local optimal regions. This motivated us to propose a novel variant of SCA called RDSCA that employed two strategies: 1) double adaptive weight strategy in order to achieve a good balance between exploitation and exploration by using two weights w1 in the first half of iterations and w2 in the last half of iterations. 2) random spare strategy in which each individual in search space can replace the optimal solution in each dimension which helps in escaping from local optima. Moreover, 30 functions from CEC 2017 and 4 real-world constrained engineering problems are used to validate our algorithm. Also, a comparison with many SCA variants has been done with RDSCA. Chen et al. [73] has used the same strategies with WO and they argued that it has a very good results. So, we test the same mechansim on SCA.

To the best of our information & knowledge and according to the literature, this is the first time that these two strategies were introduced into SCA. The main contributions of this paper are shown below:

•   A new metaheuristic algorithm, called RDSCA, is proposed based on a random replacement and double adaptive weight strategy, combined with SCA.

•   In classical CEC2017 benchmark functions, experimental results with SCADE [74], OBSCA [68], CGSCA [75], ASCA_PSO [64] and CESCA [26] demonstrate that the RDSCA algorithm has efficient performance and competitive ability in optimization search.

•   To verify the performance of the algorithm on real-world problems, RDSCA’s experimental results on four engineering problems demonstrate that the optimizer can effectively solve complex real-world problems.

The rest of this paper is organized as follows: the basic SCA and its mathematical equation are given in Section 2 whereas RDSCA is discussed comprehensively in Section 3. Section 4 shows both experiments & the results whereas the conclusion & possible future works are given in Section 5.

2  Sine Cosine Algorithm

Recently many new meta-heuristics algorithms have been developed and have shown great potential in tackling complex optimization problems such as image segmentation [76,77], PID optimization control [78], economic emission dispatch problem [79], expensive optimization problems [80], combination optimization problems [81], bankruptcy prediction [12,82], plant disease recognition [83], traveling salesman problem [84], feature selection [85,86], multi-objective problem [87], detection of foreign fiber in cotton [88,89], medical diagnosis [7,8], fault diagnosis [90,91], parameter optimization for machine learning models [92,93], prediction problems in educational field [24,25], constrained optimization problems [94,95], scheduling problem [96,97], and object tracking [98,99]. SCA can be considered as one of the most powerful and well-known meta-heuristics algorithms which was firstly proposed in 2016 [100]. The main idea in SCA is Sine & Cosine operator, which update agent movement according to Eqs. (1a), (1b):

xit+1=xit+r1×sin⁡(r2)×|r3pit−xit| if r4<0.5(1a)

xit+1=xit+r1×cos⁡(r2)×|r3pit−xit| if r4≥0.5(1b)

where r2, r3, and r4 are random numbers in the interval (0,1], pit is the best solution position obtained in the i-th iteration, and r1 can be obtained from the following equation:

r1=a−taT(2)

where a refers to a constant number, and T, t refer to the maximum iteration number and the current iteration, respectively. Also, flowchart of SCA is given in Fig. 1.

Figure 1: Flowchart of SCA

3  Proposed Algorithm

3.1 Architecture of RDSCA

In this section, the structure of the suggested algorithm called RDSCA has been illustrated in detail. RDSCA is a modification of SCA with 2 strategies: 1) random spare/replacement. 2) double adaptive weight. The pseudo-code and flow chart are given in Algorithm 1 and Fig. 2. The integration of two mentioned strategies has a great impact on SCA in search capabilities and convergence curve.

:

Figure 2: Flowchart of RDSCA

3.1.1 Random Spare/Replacement

In this strategy, the current individual with n-th dimension position will be replaced with the corresponding dimension of the optimal individual vector. A certain probability is used to implement a random spare strategy, since not any dimension value in the optimal individual is optimal. Cauchy random is used as a ratio between the current evaluation number to total evaluation number.

3.1.2 Double Adaptive Weight

In this strategy, similar to PSO [30], a double adaptive weight was used to improve the ability of local & global search w1, w2 can be calculated from Eqs. (3) and (4). Furthermore, Fig. 1 in the supplement analyzes the balance of RDSCA and SCA.

w1=(1−FEsMaxFES)1−tan⁡(π×(rand−0.5)×sMaxFEs)(3)

w2=(2−2FEsMaxFES)1−tan⁡(π×(rand−0.5)×sMaxFEs)(4)

where FEs is the current number of iterations, MaxFES is the maximum number of iterations of the algorithm, tan⁡(π×(rand−0.5) denotes the Cauchy random number, s changes with the local optimum, and s automatically increases by 1 when an individual is not updated, otherwise the value of s is subtracted by half when an individual is updated. w1 and w2 are not linearly decreasing, but fluctuate in frequency as the algorithm falls into the local optimum, and the fluctuation range of w1 is [0, 1], w2 takes values in the range [0.5, 1]. Therefore, w1 is mainly used in the first half of the algorithm iteration to improve the global search ability of the algorithm, and w2 is used in the second half of the algorithm iteration to improve the ability of the algorithm to jump out of the local optimum. Eq. (5) is used in the first half of iterations whereas, Eq. (6) is used in the last half of iterations.

xit+1={xit+w1×r1×sin⁡(r2)×|r3pit−xit| if r4<0.5xit+w1×r1×cos⁡(r2)×|r3pit−xit| if r4≥0.5(5)

xit+1={xit+w2×r1×sin⁡(r2)×|r3pit−xit| if r4<0.5xit+w2×r1×cos⁡(r2)×|r3pit−xit| if r4≥0.5(6)

3.2 The Proposed RDSCA

In the algorithmic optimization search process, certain dimensional vectors of the current individual may be responsible for the superior quality of the individual, and in contrast, certain dimensions may also lead to the poor quality of the individual. This is explained by the fact that the result of the mapping from the decision space to the objective space is determined jointly by multiple dimensions of the decision variables. Therefore, we go to adopt the values of some dimensions of the current optimal individual to replace the values of the corresponding dimensions of the current individual with a certain probability in the optimization process of the algorithm. In addition, the timing of random replacement is also very important, and we use the mathematical description in lines 10 to 13 of Algorithm 1 to define a random replacement probability. The mathematical description is mainly used to determine whether the individual adopts the random replacement strategy through the Cauchy random number and the current number of iterations. At the beginning of the algorithm iteration, the Cauchy random number can improve the randomness, so we have a higher probability to replace some dimensions of the individual, thus speeding up the convergence of the algorithm; at the later stage of the algorithm iteration, the mathematical description in the third line of the pseudo-code will have a smaller probability to replace the dimensions of the individual, resulting in Individuals have more vectors of random dimensions, thus reducing the possibility of the algorithm falling into a local optimum.

In addition, the proposed RDSCA algorithm in which we introduce an double adaptive weight strategy is shown in lines 16 to 22 of Algorithm 1. w1 and w2 are affected by the change of individuals into local optimum, which can increase the ability of the algorithm to jump out of local optimum when individuals are detected to be in local optimum. The specific principle is that when an individual is not updated for a long time, the individual is more likely to fall into a local optimum, and then the values of weights w1 and w2 increase, increasing the range of random fluctuations in Eqs. (1a), (1b) and enhancing the ability of the algorithm to jump out of the local optimum. As a result, the algorithm can jump out of the local optimum faster and increase the speed of convergence. The random replacement strategy and the double adaptive weight strategy cooperate with each other in the iterative process of the algorithm, which well balances the algorithm between search and exploitation to improve the convergence ability and the ability to jump out of the local optimum.

In addition, it should be especially noted that RDSCA and SCA use the same algorithm framework, and SCA mainly accomplishes the balance of algorithm exploration and exploitation through Eqs. (1a), (1b), while RDSCA mainly balances algorithm exploration and exploitation through Eqs. (5) and (6). Eqs. (5) and (6) expand the perturbation frequency and amplitude of RDSCA by adding adaptive dual weights w1 and w2 without changing the conversion mechanism of the algorithm from exploration to exploitation.

3.3 Time Complexity of RDSCA

The complexity of any meta-heuristic algorithm is very crucial as it computes its needed time. The time complexity of RDSCA is is depended on the number of individuals (N), maximum iteration number (T), and number of dimensions (Dim). Therefore, RDSCA complexity is as follows:

O(RDSCA)=O(Initialization)+O(Evaluating )+O(Updating individual)=O(N×D)+O(N×D×T)+O(N×D×T)=O(N×D×T)

4  Experiment

In order to verify the effectiveness of the algorithm, a fair experiment is necessary, and the whole process must be controlled and have a verifiable data set [101]. In this section, a discussion on the performance of RDSCA is introduced based on some experiments using CEC 2017, they have been internationally recognized test data in the field of metaheuristic algorithms. References [102,103] are a benchmark and a number of compared algorithms. All experimental results are the average of the results of 30 independent runs of the algorithm on each benchmark function. All experiments of the algorithms were coded and run on the MAT-LAB R2014b software platform.

4.1 Benchmark Test Functions

In order to evaluate the performance of the proposed RDSCA, 30 functions from CEC 2017 benchmark were used. These functions have really complex parameters and a variety of real optimization difficulties that are close to reality. They can be classified into unimodal (F1–F3), multi-modal (F4–F10), hybrid (F11–F20), and composite (F21–F30) as given in Table S1 in the supplement, where ‘’Types” denotes the type of function; unimodal indicates that there is only one global optimum solution and no local optimum, which can test the convergence speed of the algorithm; multi-modal contains multiple local optima with one and only one global optimum; hybrid and composite functions are more complex, which often have multi-modal nature and have more difficult to jump out of the local optimum, which requires a better balance between exploration and exploitation of the algorithm. “Name” denotes the name of the function; “Opt” denotes the optimal objective value of the function. In all experiments, the number of population (n) is set to 30, dimension numbers (dim) is equal to 30, and the number of maximum evaluation of fitness function (MaxFEs) has been set to dim * 10000 = 300000 as it is shown in Table 1.

4.2 Comparative Algorithms

To evaluate the performance of RDSCA, a comparison against many other variants of SCA has been done using 30 functions from CEC 2017. Modified SCA versions are chosen for comparison purpose, which are mentioned as bellow:

•   SCADE [74]: differential evolution (DE) operators have been employed in SCA as local search operators.

•   OBSCA [68]: opposition-based learning (OBL) is used in the initialization and updating phases.

•   CGSCA [75]: exponential decreasing conversion parameter and linear decreasing inertia weight methods are employed within SCA to achieve a good balance between exploration and exploitation. Moreover, a greedy Levy mutation technique is also applied.

•   ASCA_PSO [62]: ASCA-PSO is an algorithm that hybridizes PSO with SCA in order to have more exploitation properties.

•   CESCA [26]: chaos theory concepts have been employed in SCA.

4.3 Measures of the Performance

Here, a number of statistical metrics are used to compute the RDSCA performance.

•   Average (mean):

Avg=1N∑i=1Ng∗i(7)

•   Std:

SD=1N−1∑i=1N(g∗i−Avg)2(8)

•   Min: the best solution obtained over all running.

best=mini=1Ng∗i(9)

where M refers to the iteration number to perform the experiments and g∗i refers to the result obtained from the fitness function at i-th iteration and Avg refers to the average of all running.

•   Max: the worst solution obtained over all running.

Worst=maxi=1Ng∗i(10)

where N refers to iteration number to perform the experiments and g∗i is the result obtain fron the fitness function at i-th iteration.

4.4 Discussion

4.4.1 Comparison between RDSCA and Other Algorithms

The comparison results between RDSCA and other SCA variants mentioned in Section 4.2 are shown in Table 2. From this table, it is notable that the suggested algorithm achieved better fitness value. RDSCA achieved the best average in 24 functions from 30 functions, second best in 4 other functions (F2, F11, F18, and F24) and the third best in F14. Furthermore, RDSCA ranked first in term of standard deviation in 19 functions and the second best in the other 11 functions.

Fig. 3 shows the convergence curve for some functions from all types (unimodal, multimodal, hybrid, and composite). It can be observed from this figure that RDSCA has a better speed convergence compared to other algorithms.

Figure 3: The average of convergence curves of unimodal & multi-modal functions

Also, Fig. 4 shows the 2-D appearance of the function in the first column whereas the search history is shown in the second one where we can observe that RDSCA’s search agent tends to exploit the promising regions of search space more extensively. The third column shows the first particle trajectory where we can see the first agent changes in the first dimension and the fourth and fifth columns show RDSCA’s average fitness and convergence curve, respectively. Furthermore, the result of banlance analysis of RDSCA and SCA is shown in Figure S1 in the supplement, which also shows the superiority of RDSCA.

Figure 4: (a) The average of convergence curves of unimodal & multi-modal functions 2; (b) The average of convergence curves of unimodal & multi-modal functions 2

4.4.2 Statistical Analysis

Besides the performance measured mentioned in Section 4.3, a non-parametric test called Wilcoxon signed-rank (WSR) [104] is used to be able to prove the significant superiority of the RDSCA. WRS considers all results as one group and then ranks are assigned to all scores in each group then, WRS test sums each group rank.

All algorithms are tested in all test functions for WSR. Each algorithm is performed 30 times independently on each test function, and the resulting solutions are available as data samples. WSR can test whether the two algorithms are significantly different on that test function based on these samples. First, we start with the null hypothesis, which is the assumption that there is no significant difference in the performance of the two algorithms. When p<0.05, the null hypothesis is rejected, and when p>0.05, the null hypothesis is accepted. Therefore, the two algorithms are significantly different when the p-value is greater than 0.05. Table S2 in the supplement represents the p-values for RDSCA and each comparison algorithm on each test function. From the table, we can find that most of the p-values are less than 0.05, therefore, we can conclude that RDSCA and other comparison algorithms are significantly different on the CEC2017 test functions and that RDSCA outperforms the comparison algorithms from the results in Table 2.

4.5 Engineering Problems

To validate our proposed meta-heuristic which known as RDSCA, we apply it to solve real engineering constrained problems which have various inequality and equality constraints. Furthermore, best values of such parameter of these problems remain unknown. Here, four different engineering optimization problems are used in order to validate RDSCA namely: tension/compression spring design (TCSD), welded beam design (WBD), pressure vessel design (PVD), and I-beam design (IBD).

4.5.1 TCSD Problem

The first engineering problem is called TCSD [105] (see Appendix A for the mathematical formulation). This problem aims to find the optimal and minimum weight and includes 3 design variables: mean coil diameter (D), active coils number (N), and diameter of wire (d).

Table S3 in the supplement shows the results of RDSCA against other peers. It can be noticed that the proposed algorithm achieved the best results.

4.5.2 WBD Problem

WBD problem is a well-known metal problem [106] which aims to obtain welded beam that has the minimum cost of fabrication. The mathematical modeling of WBD can be found in Appendix B. Here a comparison of our proposed approach has been done against other peers as shown in Table S4 in the supplement. It is obviously noted from the above mentioned table that RDSCA has got the minimum optimal solution, i.e., minimum fabrication cost.

4.5.3 PVD Problem

The purpose of this unconstrained engineering problem is to find the minimum cost of the cylindrical pressure vessel [107]. The PVD has 4 variables: head thickness (Th), shell thickness (Ts), inner radius (R), and cylindrical section length (L). Appendix C shows mathematical equations of PVD.

Table S5 in the supplement shows the comparison results among RDSCA, other peers. It is obviously noticed that RDSCA ranked first.

4.5.4 IBD Problem

In this section, our proposed algorithm is used to find the optimal design of I-beam problem [108] in order to have minimum deflection. This problem has four design parameters: height, length, and 2 thickness. The mathematical formulation is founded in section Appendix D. The results of RDSCA against other algorithms are shown in Table S6 in the supplement where we can observe that RDSCA ranked first.

5  Conclusion & Possible Future Works

In this study, a new metaheuristic algorithm, called RDSCA, is proposed by adding double adaptive weight and random replacement strategy to the algorithmic framework of SCA, where random replacement are used in the process of iterative optimization of the algorithm, and each individual has a certain probability to select a dimensional value and the corresponding dimension of the current optimal value for replacement. The basic principle of this strategy is that the cause of poor individual fitness values may be the value of a single dimension of an individual. And this method can effectively increase the convergence speed of the algorithm because the individuals are using as much information as possible from the current optimal solution. The double adaptive weight strategy mainly uses two adaptive weights w1 and w2 to enhance the ability of the algorithm to jump out of the local optimum, where the updates of w1 and w2 are related to the frequency of individuals falling into the local optimum and the Corsi random number. Once an individual falls into a local optimum, the algorithm can jump out faster, thus improving the convergence speed. Based on this, we experimentally compare RDSCA with other mainstream metaheuristic algorithms on CEC2017 and four restricted real-world engineering problems, and the experimental results prove that RDSCA has stronger performance and strong competitiveness.

However, RDSCA and SCA’s use the same algorithm framework, which also means that the algorithm is explored and developed in the same way of conversion as the original SCA, which may not be able to better balance the exploration and development of the algorithm when optimizing complex composite functions. In the future, we will use new techniques to improve the ability of algorithms to balance exploration and development. In the future, we will apply this version to many applications in different fields such as community service [109], multicore systems [110], etc. We can also try to propose a multi-objective version and binary version of RDSCA for other interesting scenarios. Also, it is possible in future to apply these combination of operators to other metaheurstics and comparing them.

Funding Statement: This study was supported in part by the Hangzhou Science and Technology Development Plan Project (Grant No. 20191203B30).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

Appendix A. Tension/compression spring design problem

Minimize: f(x)=(x3+2)x2x12

Subject to: g1(x)=1−(x23x3/71,785x14)≤0

g2(x)=(4x22−x1x2/12,566(x2x13−x14)+(1/5108x12))−10≤0

g3(x)=1−(140.45x1/x22x3)≤0

g4(x)=(x2+x1)/1.5−1≤0,

Variable Range

0.05≤x1≤2.00

0.25≤x2≤1.30

2.00≤x3≤15.00

Appendix B. Welded beam design problem

Minimize: f1(x)=1.10471∗x(1)2∗x(2)+0.04811∗x(3)∗x(4)∗(14.0+∗x(2))

Subject to: g1(x)=τ−13600

g2(x)=σ−30000

g3(x)=x(1)−x(4)

g4(x)=6000−p

Variable Range

0.125≤x1≤5

0.1≤x2≤10

0.1≤x3≤10

0.125≤x4≤5

Appendix C. Pressure vessel design problem

Minimize: f(x)=0.6224x1x3x4+1.7781x2x32+3.1661x12x4+19.84x12x3

Subject to: g1(x)=−x1+0.0193x

g2(x)=−x2+0/00954x3≤0

g3(x)=−πx32x4−(4/3)πx33+1,296,000≤0

g4(x)=x4−240≤0

Variable Range

0≤xi≤100,i=1,2

0≤xi≤200,i=3,4

Appendix D. I-beam design problem

Minimize: f(x)=5000tw(h−2tf)312+btf36+2btf(h−tf2)2

Subject to:

g=2btw+tw(h−2tf)≤0

Variable Range

10≤x1≤50

10≤x2≤80

0.9≤x3≤5

0.9≤x4≤5

Figure S1: Balance analysis of RDSCA and SCA

Table S1: CEC 2017 benchmark functions

Table S2: The calculated p-values from the Wilcoxon signed-rank test

Table S3: Results of TCSD problem

Table S4: Results of WBD problem

Table S5: Results of PVD problem

Table S6: Results of IBD problem