Open Access

ARTICLE

# A Pattern Classification Model for Vowel Data Using Fuzzy Nearest Neighbor

1 Department of Computer Science and Engineering, National Institute of Technology Srinagar, Hazratbal, 190006, Jammu and Kashmir, India

2 Department of Computer Science and Engineering, Aliah University, Kolkata, India

3 Department of Computer Science and Engineering, SRM University, Amaravati, 522240, AP, India

4 School of Computer Science SCS, Taylor’s University, Subang Jaya, 47500, Malaysia

5 Department of Computer Science, College of Computers and Information Technology, Taif University, P.O. Box 11099, Taif, 21944, Saudi Arabia

* Corresponding Author: NZ Jhanjhi. Email:

*Intelligent Automation & Soft Computing* **2023**, *35*(3), 3587-3598. https://doi.org/10.32604/iasc.2023.029785

**Received** 11 March 2022; **Accepted** 09 June 2022; **Issue published** 17 August 2022

## Abstract

Classification of the patterns is a crucial structure of research and applications. Using fuzzy set theory, classifying the patterns has become of great interest because of its ability to understand the parameters. One of the problems observed in the fuzzification of an unknown pattern is that importance is given only to the known patterns but not to their features. In contrast, features of the patterns play an essential role when their respective patterns overlap. In this paper, an optimal fuzzy nearest neighbor model has been introduced in which a fuzzification process has been carried out for the unknown pattern using*k*nearest neighbor. With the help of the fuzzification process, the membership matrix has been formed. In this membership matrix, fuzzification has been carried out of the features of the unknown pattern. Classification results are verified on a completely llabelled Telugu vowel data set, and the accuracy is compared with the different models and the fuzzy

*k*nearest neighbor algorithm. The proposed model gives 84.86% accuracy on 50% training data set and 89.35% accuracy on 80% training data set. The proposed classifier learns well enough with a small amount of training data, resulting in an efficient and faster approach.

## Keywords

Pattern classification has been a challenging task for the last decades. It is used in many practical applications (like pattern recognition, artificial intelligence, statistics, financial gaming, organization data, vision analysis, and medicine) [1]. There are many critical aspects in the pattern classification problem, like the accuracy of classification, computational time, learnability, generality, interpretation of parameters etc. Many approaches exist to create pattern classifiers, such as neural networks, statistical models, fuzzy logic systems, and evolutionary systems [2–5]. In the applications mentioned above, the classification of patterns is essential. But whenever classification data sets are highly overlying, and the boundaries of the classes are imprecisely defined, it becomes a challenging task, for example, in land cover classification, remote sensing images, or vowel classification [6,7].

Although, in various pattern recognition issues, the categorization of the input pattern depends upon the dataset, where, the actual sample size for every class is limited and perhaps not indicative of the actual probability distributions, regardless of whether they are known. In these conditions, numerous techniques depend on distance or similarity in feature sets, for example, discriminant analysis and clustering [8–10]. In various problems, machine learning methods such as neural network [11],

Meher [1] proposed a model for pattern classification by combining neighborhood rough sets and Pawlak’s rough set theory with fuzzy sets. Ghosh et al. [20] proposed a model based on neuro-fuzzy classification for fully and partially labeled data utilizing the feed-forward neural network algorithm. A neuro-fuzzy system was presented by Meher [21] for pattern classification by extracting features using rough set theory. An extreme learning machine was then used to efficiently classify partially and fully labeled data and remote sensing pictures. Pal et al. [22] developed a rough-fuzzy model depending on granular computing to classify fully and partially labeled data using rough set theory. The neuro-fuzzy model was also used in various other problems, i.e., to analyse biomedical data [23], Parkinson’s disease diagnosis [24], and analysis of gene expression data [25].

According to the k-nearest neighbor algorithm, the class labels of k-closest patterns decide the input pattern label. K-closest patterns are selected based on distance like Euclidean or Manhattan [26]. The k-nearest neighbor method is suboptimal. Although it has been demonstrated that in the infinite sample condition, the error value for the 1-nearest neighbor method is upper constrained by no more than double the optimum Bayes error value and as k rises, this error value reaches the optimal value [27]. There are some problems identified by Keller et al. [26] in the k-nearest neighbor rule, which are as follows: the first one is, that all k-nearest neighbors are equally essential to assign a class label to the unknown pattern, which causes a problem, where classification data set overlaps. It happens because atypical patterns have equal weight as true representatives of the groups. The second problem is that after an input pattern is allocated to a class, there is no sign of its “strength” of membership in that particular class. The above problems are addressed by Keller et al. [26], and resolved by using the fuzzy set theory in the k-nearest neighbor rule [26,28]. According to Keller et al. [26], the input pattern strength is calculated for each class then the class with maximum strength is assigned to the input pattern.

In this paper, a model is proposed for pattern classification. First, the model finds the nearest neighbors to the input pattern using the k-nearest neighbor algorithm. Next, the model finds membership values of features of the input pattern using fuzzy sets. Then, the model utilize the product reasoning rule followed by MAX operation to find the class label of the input pattern. The proposed model performance is verified using various classification models with the vowel data set. The performance of the proposed model is also compared with the fuzzy k nearest neighbor algorithm on the 50% and 80% training data sets. The motivation of this work is to utilize the fuzzification process that produces the importance of features of input patterns belonging to all classes rather than just one class.

The main contributions of this paper are as follows:

• A particular problem in the fuzzy k-nearest neighbor algorithm is addressed, i.e., when the data has highly overlapping classes.

• The identified problem is resolved by using a membership matrix and considering the importance of each pattern feature rather than considering the significance only.

• A pattern classification model is developed using the k-nearest neighbor algorithm. The model’s accuracy is verified using different classification models with the vowel data set.

The organization of the paper is as follows: In Section 2, steps of the proposed model have been discussed; in Section 3, the data set and the result and analysis are discussed, and the proposed model is compared with the fuzzy k-nearest neighbor algorithm, and the proposed model is also compared with five other classification models and conclusion is drawn in Section 4.

2 Framework of the Proposed Model

In this section, a model is proposed to classify unknown patterns, and the various steps are shown in Fig. 1. Initially, the proposed model finds out the nearest neighbors of the unknown pattern using the k-nearest neighbor algorithm. Now, selected nearest neighbors are provided as input to the fuzzification process. Then the reasoning rule and defuzzification process are carried out to find the class label for the unknown pattern. In this paper, the proposed model is implemented in MATLAB software. The succeeding subsections describe the classification process and the advantage of using it.

2.1 Nearest Neighbors of the Input Pattern

In this section, the nearest neighbours of the input pattern are chosen using the k nearest neighbors algorithm as shown in the first step of Fig. 1, where k is a positive integer. Let

The pattern

2.2 Fuzzification of Features of Input Pattern

In this section, fuzzification of features of a pattern is processed by using

where

where

The fact that the sum of a feature’s membership values in the

For example, when c = 2, if the membership values are near the value of 0.5, it indicates that the feature has a high level of membership in both of the classes; that is, the “bounding area” which isolates the classes from each other.

The output of the fuzzification process is a membership matrix M (as described in [7,32]), which uses fuzzy sets to assign membership degrees to aspects of a pattern to distinct classes by applying fuzzy sets [28]. On fuzzy sets, aggregation operations merge multiple fuzzy sets in a personalized manner to form a single fuzzy set. For issues in which all features contribute adequately to the desired class and cooperate in the decision-making approach, union and intersection (basic aggregate operations) are often unsatisfactory [33]. So, we have utilized the minimum reasoning rule (RR) over the attributes of the membership matrix as described in Li et al. [7]. Ghosh et al. [32] have explained and illustrated the advantage of utilizing product RR rather than minimal RR in different real-life datasets. In this work, product RR has been used for finding the class label. After using the product RR, the output obtained in the form of a vector is given by:

for

2.4 Rescaling and Defuzzification

Finally, the rescaled vector is obtained as

If

In this section, we will discuss the data set and performance of the proposed model. The performance of the presented model is shown in the context of percentage accuracy (PA), where percentage accuracy is the proportion of the testing data that the proposed model effectively categorizes. The known class label of testing data is compared with classified results from the proposed model for the model’s accuracy. The training and testing data of the data set are selected at random by partitioning the data set into two parts. Testing data is independent of training data.

This paper verifies the proposed model on the benchmarked Telugu vowel data set [35]. The data set is completely labeled and comprises 871 patterns, having three features and six highly overlapping classes. Features of the patterns are the sound which is uttered by human beings. The overlapping idea of these classes can be imagined from Fig. 2 of the vowel data set [1]. It has been observed that about 50% of boundary regions of class 5(/e/) is overlapped with neighbor’s class boundaries, for example, class 1 (/∂/), class 3(/i/) and class 6 (/o/).

3.2 Proposed Model’s Performance at the Varying Percentage of Training Data

The performance evaluated for different percentages of training data is illustrated in Tab. 1. The results from the Tab. 1 show that, for different percentages of training data, the percentage accuracy of the classification has been evaluated for the respective training data at m = 1.1 and k = 5. For better visualization of the performance accuracy, the respective bar chart is also shown in Fig. 3. From Fig. 3, it is explicable that as the percentage of training data increases, the accuracy in classifying the testing data also increases simultaneously.

3.3 Comparison of the Proposed Model with Various Classification Models

The proposed model’s performance is compared with various classification models. The models stated below have the benchmarked accuracy for the vowel data Set at 50% and 80% training data set [35–37]. Hence, the same benchmarked models have been used for the performance analysis on the 50% and 80% training data sets, which are stated below:

(a) Models used on 50% training data set

Model 1: Low, medium, and high (LMH) fuzzification (Meher [1]),

Model 2: LMH with fuzzy product aggregation reasoning rule (FPARR) classification (Meher [1]),

Model 3: Neuro-fuzzy (NF) classifier (Ghosh et al. [20]),

Model 4: LMH and Pawlak’s rough set theory with FPARR (Meher [1]),

Model 5: LMH and neighborhood rough set with FPARR (Meher [1]),

Model 6: A pattern classification model for vowel data using fuzzy nearest neighbor (This model).

(b) Models used on 80% training data set

Model 1: Neuro-fuzzy (NF) classifier (Ghosh et al. [20]),

Model 2: Class dependent fuzzification with Pawlek’s rough set feature selection (Pal et al. [22]),

Model 3: Class dependent fuzzification with neighborhood rough set (NRS) feature selection (Pal et al. [22]),

Model 4: NRS fuzzification and neural network classifier with extreme learning machine algorithm (Meher [21]),

Model 5: SSV decision tree (Duch et al. [37]),

Model 6: A pattern classification model for vowel data using fuzzy nearest neighbor (This model).

For the 50% and 80% training data set, the performance of all the classification models is shown in Tab. 2. In Tab. 2, at different percentages of the training data set, percentage accuracies for all the models have been calculated. From Figs. 4a–4b, it is visible that the percentage accuracy of model 6 is highest as compared to the rest of the five models for 50% and 80% training data set. In the experimental analysis, the efficiencies of the models have been demonstrated and it was found that the accuracy of the proposed model is superior to the previous models at m = 1.1 and k = 5 with the vowel data set.

3.4 Comparison of the Proposed Model with Fuzzy k-Nearest Neighbor Algorithm

The percentage accuracy of the presented model is compared with the fuzzy k-nearest neighbor algorithm proposed by Keller et al. [26] on the 50% and 80% training data sets. The accuracy of the presented model is calculated using a random subsampling technique. This technique randomly splits the data set into training and test data. The model has been trained for the training data for each split, and the accuracy is predicted using test data. The resulting performance accuracy is then averaged over the splits. A comparison of the proposed model with a fuzzy k-nearest neighbor algorithm over such splits is illustrated in Tab. 3.

From Tab. 3, it has been depicted that the average accuracy of the fuzzy k-nearest neighbor algorithm at 50% and 80% training data is 84.72477% and 85.05747%, respectively. In comparison, the average accuracy of the proposed model at 50% and 80% training data sets is 85.27523% and 88.50575%, respectively. The proposed model has better accuracy for both data sets than the fuzzy k-nearest neighbor algorithm. The results of the splits are shown in Figs. 5a–5b. In experimental analysis, the model’s efficiency has been demonstrated. It has been observed that the performance accuracy of the proposed model is superior to the fuzzy k-nearest neighbor algorithm at m = 1.1 and k = 5 with the vowel data set.

The pattern classification model for the vowel data using fuzzy set theory has been proposed, exploring the advantage of the explicit fuzzy classification technique and improving the model’s performance. Thus, the model explores the collective benefits of these techniques, which provide better class partition details, helpful for significantly overlapping data sets. The proposed model generates a membership matrix that represents the importance of features of input patterns belonging to all classes rather than just one class. As a result, the ability to generalize is improved. The efficiency of the proposed model was calculated through the percentage accuracy (PA), which was measured for a completely labeled vowel data set. Classification accuracy of the proposed model is also compared with the previous classification models and the fuzzy k nearest neighbor algorithm. The proposed model gives 84.86% accuracy on 50% training data set and 89.35% accuracy on 80% training data set. The learning ability of the proposed model from a small fraction of training data makes it applicable to tasks, including a high number of features and classes. This work can also be extended for organizational data, financial gaming, statistics etc.

Data Availability Statement: In this paper, the benchmarked Telugu vowel data set taken from [35] and available on GitHub repository https://github.com/Monika01p/Telugu-Vowel-Data-set.

Funding Statement: This work was supported by the Taif University Researchers Supporting Project Number (TURSP-2020/79), Taif University, Taif, Saudi Arabia.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

## References

- S. K. Meher, “Explicit rough–fuzzy pattern classification model,” Pattern Recognition Letters, vol. 36, pp. 54–61, 2014.
- S. Haykin and N. Network, “A comprehensive foundation,” Neural Networks, vol. 2, pp. 41, 2004.
- L. Kuncheva, “Fuzzy classifier design,” Springer Science & Buisness Media, vol. 49, 2000.
- S. Derivaux, G. Forestier, C. Wemmert and S. Lefevre, “Supervised image segmentation using watershed transform, fuzzy classification and evolutionary computation,” Pattern Recognition Letters, vol. 31, no. 15, pp. 2364–2374, 2010.
- S. Umer, P. P. Mohanta, R. K. Rout and H. M. Pandey, “Machine learning method for cosmetic product recognition: A visual searching approach,” Multimedia Tools and Applications, vol. 80, no. 28, pp. 34997–35023, 2021.
- F. Melgani, B. A. Al Hashemy and S. M. Taha, “An explicit fuzzy supervised classification method for multispectral remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 38, no. 1, pp. 287–295, 2000.
- C. Li, J. Zhou, Q. Li and X. Xiang, “A fuzzy cluster algorithm based on mutative scale chaos optimization,” in Int. Symp. on Neural Networks, Berlin, Germany, pp. 259–267, 2008.
- S. Russell and P. Norvig, “Artificial intelligence: A modern approach,” Prentice Hall, 2002.
- A. Iosifidis, A. Tefas and I. Pitas, “Multi-view action recognition based on action volumes, fuzzy distances and cluster discriminant analysis,” Signal Processing, vol. 93, no. 6, pp. 1445–1457, 2013.
- R. K. Rout, S. S. Hassan, S. Sindhwani, H. M. Pandey and S. Umer, “Intelligent classification and analysis of essential genes using quantitative methods,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), vol. 16, no. 1s, pp. 1–21, 2020.
- M. Khandelwal, D. K. Gupta and P. Bhale, “DoS attack detection technique using back propagation neural network,” in Int. Conf. on Advances in Computing, Communications and Informatics (ICACCI), Jaipur, Rajasthan, India, pp. 1064–1068, 2016.
- T. Cover and P. Hart, “Nearest neighbor pattern classification,” IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 21–27, 1967.
- M. Khandelwal, R. K. Rout and S. Umer, “Protein-protein interaction prediction from primary sequences using supervised machine learning algorithm,” in 12 th Int. Conf. on Cloud Computing, Data Science & Engineering (Confluence), Noida, India, pp. 268–272, 2022.
- S. Umer, R. Mondal, H. M. Pandey and R. K. Rout, “Deep features based convolutional neural network model for text and non-text region segmentation from document images,” Applied Soft Computing, vol. 113, pp. 107917, 2021.
- S. Umer, A. Sardar, B. C. Dhara, R. K. Rout and H. M. Pandey, “Person identification using fusion of iris and periocular deep features,” Neural Networks, vol. 122, pp. 407–419, 2020.
- H. Das, B. Naik and H. S. Behera, “Medical disease analysis using neuro-fuzzy with feature extraction model for classification,” Informatics in Medicine Unlocked, vol. 18, pp. 100288, 2020.
- H. Das, B. Naik and H. S. Behera, “A hybrid neuro-fuzzy and feature reduction model for classification,” Advances in Fuzzy Systems, vol. 2020, pp. 4152049:1–4152049:15, 2020.
- K. V. Shihabudheen and G. N. Pillai, “Recent advances in neuro-fuzzy system: A survey,” Knowledge-Based Systems, vol. 152, pp. 136–162, 20
- H. Patel and G. S. Thakur, “An improved fuzzy k-nearest neighbor algorithm for imbalanced data using adaptive approach,” IETE Journal of Research, vol. 65, no. 6, pp. 780–789, 20
- A. Ghosh, B. U. Shankar and S. K. Meher, “A novel approach to neuro-fuzzy classification,” Neural Networks, vol. 22, no. 1, pp. 100–109, 2009.
- S. K. Meher, “Efficient pattern classification model with neuro-fuzzy networks,” Soft Computing, vol. 21, no. 12, pp. 3317–3334, 2017.
- S. K. Pal, S. K. Meher and S. Dutta, “Class-dependent rough-fuzzy granular space, dispersion index and classification,” Pattern Recognition, vol. 45, no. 7, pp. 2690–2707, 2012.
- H. Das, B. Naik, H. S. Behera, S. Jaiswal, P. Mahato et al., “Biomedical data analysis using neuro-fuzzy model with post-feature reduction,” Journal of King Saud University-Computer and Information Sciences, 2020.
- H. L. Chen, C. C. Huang, X. G. Yu, X. Xu, X. Sun et al., “An efficient diagnosis system for detection of Parkinson’s disease using fuzzy k-nearest neighbor approach,” Expert Systems with Applications, vol. 40, no. 1, pp. 263–271, 2013.
- M. Khashei, A. Z. Hamadani and M. Bijari, “A fuzzy intelligent approach to the classification problem in gene expression data analysis,” Knowledge-Based Systems, vol. 27, pp. 465–474, 2012.
- J. M. Keller, M. R. Gray and J. A. Givens, “A fuzzy k-nearest neighbor algorithm,” IEEE Transactions on Systems,” Man, and Cybernetics, vol. SMC-15, no. 4, pp. 580–585, 1985.
- A. Kontorovich and R. Weiss, “A Bayes consistent 1-NN classifier,” in Artificial Intelligence and Statistics, PMLR, San Diego, California, USA, pp. 480–488, 2015.
- L. A. Zadeh, “Fuzzy sets,” in Fuzzy Sets, Fuzzy Logic, and Fuzzy Systems: Selected Papers by Lotfi A Zadeh, USA: World Scientific, vol. 6, pp. 19–34, 1996.
- S. S. Hassan, R. K. Rout, K. S. Sahoo, N. Jhanjhi, S. Umer et al., “A vicenary analysis of SARS-CoV-2 genomes,” Computers Materials & Continua, pp. 3477–3493, 2021.
- R. K. Rout, S. S. Hassan, S. Sheikh, S. Umer, K. S. Sahoo et al., “Feature-extraction and analysis based on spatial distribution of amino acids for SARS-CoV-2 protein sequences,” Computers in Biology and Medicine, vol. 141, pp. 105024, 2022.
- S. K. Sahu, D. P. Mohapatra, J. K. Rout, K. S. Sahoo, Q. Pham et al., “A LSTM-FCNN based multi-class intrusion detection using scalable framework,” Computers & Electrical Engineering, vol. 99, pp. 107720, 2022.
- A. Ghosh, S. K. Meher and B. U. Shankar, “A novel fuzzy classifier based on product aggregation operator,” Pattern Recognition, vol. 41, no. 3, pp. 961–971, 2008.
- H. J. Zimmermann, “Fuzzy set theory-and its applications,” Springer Science & Business Media, 2011.
- A. V. Patel, “Simplest fuzzy PI controllers under various defuzzification methods,” International Journal of Computational Cognition, vol. 3, no. 1, pp. 21–34, 2005.
- S. K. Pal and D. D. Majumder, “Fuzzy sets and decision making approaches in vowel and speaker recognition,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 7, no. 8, pp. 625–629, 1977.
- S. K. Pal and S. Mitra, “Multilayer perceptron, fuzzy sets, classification,” IEEE Transactions on Neural Networks, vol. 3, no. 5, 1992.
- W. Duch and Y. Hayashi, “Computational intelligence methods and data understanding,” Studies in Fuzziness and Soft Computing, vol. 54, pp. 256–270, 2000.

## Cite This Article

M. Khandelwal, R. K. Rout, S. Umer, K. S. Sahoo, N. Jhanjhi*et al.*, "A pattern classification model for vowel data using fuzzy nearest neighbor,"

*Intelligent Automation & Soft Computing*, vol. 35, no.3, pp. 3587–3598, 2023. https://doi.org/10.32604/iasc.2023.029785