Dipper Throated Optimization Algorithm for Unconstrained Function and Feature Selection

: Dipper throated optimization (DTO) algorithm is a novel with a very efficient metaheuristic inspired by the dipper throated bird. DTO has its unique hunting technique by performing rapid bowing movements. To show the efficiency of the proposed algorithm, DTO is tested and compared to the algorithms of Particle Swarm Optimization (PSO), Whale Optimization Algorithm (WOA), Grey Wolf Optimizer (GWO), and Genetic Algorithm (GA) based on the seven unimodal benchmark functions. Then, ANOVA and Wilcoxon rank-sum tests are performed to confirm the effectiveness of the DTO compared to other optimization techniques. Additionally, to demonstrate the proposed algorithm’s suitability for solving complex real-world issues, DTO is used to solve the feature selection problem. The strategy of using DTOs as feature selection is evaluated using commonly used data sets from the University of California at Irvine (UCI) repository. The findings indicate that the DTO outperforms all other algorithms in addressing feature selection issues, demonstrating the proposed algorithm’s capabilities to solve complex real-world situations.


Introduction
Optimization is the process of obtaining the greatest or least objective function value for a set of inputs. It is the subject of various machine techniques that draw on artificial neural networks. Hundreds of famous optimization algorithms have become accessible, and dozens of technologies are available in major scientific code libraries. Given the problems of optimization, selecting what methods can thus be challenging [1]. Optimization is how a function has the lowest or highest output of input parameters or arguments. In machines where the input parameters of the tasks like the floating-point values are numerical, continuous functions optimization frequently arises. The function returns an evaluation of the parameter of real-life [2].
Continuous function optimization may be used to distinguish between such problems with discrete variables, known as combined optimization problems [3]. Different techniques may be resolved, organized, and called to optimize the problems involving continuous functions. The information about the objective function utilized and used throughout the optimization process depends on one technique of optimization classification [4]. The more information about the target function is generally understood, the easier it is to optimize because knowledge can be applied effectively [5]. Perhaps the significant difference between optimization techniques is identifying the destination function in one location [6]. It means that the first derivative of the feature may be used to identify a possible solution (gradient or route). It distinguishes itself from other not-calculated gradient data [7]. Metaheuristic optimization is the optimization process utilizing metaheuristic techniques. Almost every area of life is involved, much from engineering to business, holiday preparation to internet travel [8]. The use of those readily available resources must be maximized due to the continuous scarcity of money, resources, and time. The vast majority are non-linear, multimodal, and quite restrictive in real-life problems [9]. Different objectives frequently collide. Even if one goal is set, Optimum solutions are not always available [10]. Usually, a faultless or failed response is not simple to find. Many metaheuristic algorithms have been published, including swarm intelligence, anthrax optimization, optimization of the particulate swarm. In previous articles [11]. The feature selection issue may be seen as a multi-objective optimization problem in which two conflicting objectives must be met: picking the fewest possible features while attaining maximum classification accuracy [12]. The solution with the most distinctive features and the maximum classification accuracy is deemed optimal [13].

Literature Review
Meta-heuristics refers to generic methods that normally used to solve complex and challenging combinatorial search problems. Generally, the problems solved by metaheuristic algorithms are challenging for computer scientists due to the need to examining a huge number of combinations that usually exponential with conflicting objectives [14]. Many metaheuristic algorithms have been proposed to tackle real-world situation such as image segmentation [15], water allocation and crop planning [16], Nurse Rostering [17], power load dispatch [18], and Parkinson diagnosis [19]. Several survey papers are available for more information about metaheuristic algorithms [20][21][22][23].
Several research paper can be found in the literature tackling feature selection as in [37][38][39][40]. When it comes to feature selection, metaheuristic algorithms are instrumental because they deal with the dimensions of the data set to make predictions [38]. However, when the dimensionality of the data sets is increased, the performance of classification methods suffers because of this. Furthermore, highdimensional data sets have several drawbacks, including a long model creation time, redundant data, and reduced performance, making data analysis very challenging [40]. The feature selection step is a major preprocessing step that is used to resolve this problem. Its goal is to select a subset of features from an extensive data set while also increasing the accuracy of the classification or clustering model, resulting in the removal of noisy, extraneous, and ambiguous data. The following section present the proposed DTO optimizer.

Proposed Dipper Throated Optimization Algorithm
Dipper Throated bird is a member of the genus Cinclus in the bird family Cinclidae, so-called because of their bobbing or dipping movements see Fig. 1. They are unique among passerines for their ability to dive, swim, and hunts underwater. Besides, it can fly rapidly and straight without pauses or glides because it has short and flexible wings. Dipper Throated bird has its unique hunting technique, it performs rapid bowing movements, enhanced by the pure white of the breast. Once the prey is detected, it dives headfirst into the water, even into the turbulent and fast-flowing water. When it became on the bottom, it turns up stones and pebbles, to disturb aquatic invertebrates, aquatic insects, and small fish. the Dipper walks on the bottom by grasping stones. It often walks against the current, with the head downwards to locate prey, it can be stable for a long time with its strong feet, also, it can walk into the water and deliberately submerge, by using its wings effectively and walk along with the bottom keeping its head well down and its body oblique to secures its food.

Mathematical Formulation
Mathematically, the Dipper Throated Optimization (DTO) algorithm assumes the birds are swimming and flying to search for food resources N fs available for n birds. The birds' locations, BP, and velocities, BV , can be represented by the following matrices: where BP i,j indicates i th bird in the j th dimension for i ∈ 1, 2, 3, . . . , n and j ∈ 1, 2, 3, . . . , d. BV i,j indicates i th bird velocity in the j th dimension for i ∈ 1, 2, 3, . . . , n and j ∈ 1, 2, 3, . . .
where the fitness value indicates the quality of food source searched by each bird. The optimal value means mother bird. These values are then sorted in ascending order. The first best solution in declared to be BP best . The remaining solutions are supposed to be normal birds BP nd for follower birds. The global best solution in declared to be BP Gbest .
First DTO mechanism by this optimizer to update the swimming bird position is based on the following equation: where BP nd (t) is a normal bird position at iteration t and BP best (t) is the best bird position. The "." is pairwise multiplication. BP nd (t + 1) is the updated bird position for the solution.
The C 1 and C 2 are updated within the iterations by the following where c changes from 2 to 0 exponentially, r 1 is a random value in [0, 1] and T max in the total number of iterations.
Second DTO mechanism is based on updating the flying bird position and velocity by the following equations. The flying birds' positions are updated as where BP nd (t + 1) is the new bird position for normal birds, and the updated velocity of each bird BV (t + 1) is calculated as where C 3 is a weight value, C 4 and C 5 are constants. BP Gbest is the global best position and r 2 is a random number in [0; 1].
The DTO algorithm can be described by this equation Update position of current swimming bird as Update velocity of current flying bird as Update position of current flying bird as Return best bird BP Gbest

Experimental Results
The experiments in this section are explained in two sets. The first set of experiments is designed to evaluate the proposed DTO algorithm performance. The proposed DTO algorithm is tested compared to the algorithms of Particle Swarm Optimization (PSO), Whale Optimization Algorithm (WOA), Grey Wolf Optimizer (GWO), and Genetic Algorithm (GA) based on the seven unimodal benchmark functions [19]. Then, ANOVA and Wilcoxon rank-sum tests are performed to confirm the effectiveness of the proposed algorithm compared to other optimization techniques. The second is experimental in feature selection [20].

Evaluation of DTO Algorithm Unconstrained Function
Tab. 1 shows a list of the unimodal benchmark function tested in the first experiment. Tab. 1 shows the compared algorithms configuration. To be fair in the comparison, all the algorithms start with 20 agents, same objective function, 100 iterations, same dimensions, and boundaries.    2 shows the convergence curves of the proposed DTO algorithm compared to the PSO, WOA, and GWO algorithms for the benchmark mathematical functions. Note that, the best convergence is achieved by the DTO algorithm. Tab. 2 shows the mean and the standard deviation results based on the benchmark function, F1-F7, for different algorithms.  To test the statistical difference between the proposed DTO algorithm and the compared algorithms, a one-way analysis of variance (ANOVA) test is applied. In this test, two hypotheses are considered; the first is null hypothesis (H 0 : μ DTO = μ PSO = μ WOA = μ GWO = μ GA ) and the alternate hypothesis (H 1 : Means are not all equal). The ANOVA test results are shown in Tab. 3. Fig. 3 shows the ANOVA test for proposed and the compared algorithms vs. the objective function. From the results, the alternate hypothesis H 1 is accepted. The ANOVA test confirms the effectiveness of the proposed algorithm compared to other optimization techniques.   Wilcoxon's rank-sum test is employed between every two algorithms to get the p-values between the proposed DTO algorithm and other algorithms to show that DTO has a significant difference. The two hypotheses are the null hypothesis (H 0 : μ DTO = μ PSO , μ DTO = μ GWO , μ DTO = μ WOA , and μ DTO = μ GA ) and the alternate hypothesis (H 1 : Means are not all equal). Tab. 4 shows that the p-values are less than 0.05 between the proposed algorithm and other algorithms. This confirms the superiority of the DTO algorithm and that it is statistically significant. Thus, the alternate hypothesis H 1 is accepted.

Evaluation of DTO Algorithm on Feature Selection Problem
In the case of the feature selection issue, the output solution should be modified from a continuous solution to a binary solution utilizing the numbers 0 or 1. This function is often used to convert the continuous solution of an optimizer to a binary solution in an optimization problem.

Fitness Function
The quality of an optimizer's solutions is determined by the fitness function that has been given to it. The error rate of classification and regression, as well as the features that have been picked from the input dataset, are the primary determinants of the function. It is advisable to choose a solution based on the collection of characteristics that can provide the bare minimum of features with the lowest classification error rate. The following equation is used in this study to evaluate the quality of the solutions provided.
As part of the experiments and comparative findings, the DTO evaluated our proposed algorithm against six datasets from the UCI machine learning library to determine its effectiveness. The datasets were chosen because they had a diverse range of features and occurrences that were reflective of the many types of problems that the proposed approach would be evaluated against. For more details about the UCI datasets, refer to Tab. 5.

Algorithm 2: The bDTO Algorithm
Initialization DTO Algorithm configuration, including population and parameters Change solutions to binary (0 or 1) Evaluate fitness function While t ≤ iters max do Calculate objective function f n for each bird BP i Find best bird BP best While t ≤ Tmax do Apply DTO Algorithm Convert solutions to binary (0 or 1) using Eq. (9) Update parameters and best solution end while Return best solution

Evaluation Metrics
The evaluation metrics used in this research are presented in Tab. 6 as follows: The average error of several algorithms is shown in Tab. 7. Due to the decreased error, the optimizer has identified the optimal collection of features that can train the classifier while also producing a smaller error on the concealed test data. Tab. 8 bellow shows the average features selected. Tab. 9 for average fitness, Tab. 10 for standard deviation fitness, Tab. 11 for best fitness, and Tab. 12 for worst fitness. The DTO has been able to find the superiority fitness for all datasets.      In this paper, a novel Dipper Throated Optimization (DTO) is introduced which is inspired by the throated dipper bird. Six UCI machine learning database datasets and unconstrained function are used to prove the consistency of the suggested optimizer and guarantee that the proposed solution is dependable and stable to evaluate its quality and effectiveness. ANOVA and Wilcoxon rank-sum tests are used to compare the proposed algorithm to different optimization methods. The results showed clearly that, DTO outperformed all other compared methods. For future work, DTO needs more investigation by applying it to solve other well-known real-world combinatorial optimization problems. Based on the great success of DTO, the researchers can investigate the hybridizations between DTO and other metaheuristic optimization algorithms as it is proved that hyper metaheuristic algorithms perform well compared to single metaheuristic algorithms.