Computer Systems Science & Engineering DOI:10.32604/csse.2023.027502
Article

Optimal Weighted Extreme Learning Machine for Cybersecurity Fake News Classification

Ashit Kumar Dutta1,*, Basit Qureshi2, Yasser Albagory3, Majed Alsanea4, Manal Al Faraj1 and Abdul Rahaman Wahab Sait5

1Department of Computer Science and Information Systems, College of Applied Sciences, AlMaarefa University, Ad Diriyah, Riyadh, 13713, Kingdom of Saudi Arabia
2Department of Computer Science, Prince Sultan University, Riyadh, 11586, Kingdom of Saudi Arabia
3Department of Computer Engineering, College of Computers and Information Technology, Taif University, Taif, 21944, Kingdom of Saudi Arabia
4Department of Computing, Arabeast Colleges, Riyadh, 11583, Kingdom of Saudi Arabia
5Department of Archives and Communication, King Faisal University, Al Ahsa, Hofuf, 31982, Kingdom of Saudi Arabia
*Corresponding Author: Ashit Kumar Dutta. Email: adotta@mcst.edu.sa
Received: 19 January 2022; Accepted: 23 March 2022

Abstract: Fake news and its significance carried the significance of affecting diverse aspects of diverse entities, ranging from a city lifestyle to a country global relativity, various methods are available to collect and determine fake news. The recently developed machine learning (ML) models can be employed for the detection and classification of fake news. This study designs a novel Chaotic Ant Swarm with Weighted Extreme Learning Machine (CAS-WELM) for Cybersecurity Fake News Detection and Classification. The goal of the CAS-WELM technique is to discriminate news into fake and real. The CAS-WELM technique initially pre-processes the input data and Glove technique is used for word embedding process. Then, N-gram based feature extraction technique is derived to generate feature vectors. Lastly, WELM model is applied for the detection and classification of fake news, in which the weight value of the WELM model can be optimally adjusted by the use of CAS algorithm. The performance validation of the CAS-WELM technique is carried out using the benchmark dataset and the results are inspected under several dimensions. The experimental results reported the enhanced outcomes of the CAS-WELM technique over the recent approaches.

Keywords: Cybersecurity; cybercrime; fake news; data classification; machine learning; metaheuristics

1  Introduction

Online data is often accessible as a result of few clicks away. With the unique independence provided to users for sharing stories, the complexity to describe the root of false data increases gradually. The existence of dramatic headlines and clickbait titles is at its highest point that assists in the broadcast of inaccurate and unprofessional news in response to advertising revenues. User, wants to be part of this hot discussion or topic, adapt the innovative message with intention or by mistake that eventually results in the distribution of rumor on the internet. Fake news is inscribed for a hoax that leads to political or gains or financial spreading data disguised as propaganda [1], one might be utilized to influence public perception towards falseness. Even this encourages the beliefs and people ideology to some range that might create several damages [2]. This persuading is popular when a news story breaks out, whereby the supporter usually tends to share data in its complete originality, while the one opinion doesn’t bring into line with the information mentioned resorting to share that similar data with few adjustments. Currently, media outlets are the only information resources. Specific contribution in news sharing has significantly developed over the last decade where it become ever more complex to discriminate news that originate from a reliable source from the one that is invented [3]. Consequently, fake news has gained several interests recently by organizations like Google, Twitter, Facebook, and by various authors, who are making continuous attempts in opposing the spread of fake stories. Fig. 1 illustrates the platform to detect fake news.

Figure 1: Platform to detect fake news

Artificial intelligence (AI) technique is the evolving technology that has transformed the view at business problems [4]. An increasing amount of businesses are transforming to innovative analysis and machine learning to resolve problems. With this development, natural language processing (NLP) describes great potential for business that is concerned with understanding human sentiment via the current information. NLP functions with each kind of social and natural communication, involving text, audio, and video. In order to identify trends and many valuable patterns in the textual data set, text mining assisted to perform in this way [5]. In present market setting, strategic use of NLP assist business to obtain relative benefits. AI and NLP assist in combating the large unstructured data of various fields involving education, healthcare, business sectors, fake news, trust and security, opinion from the public in the government sector [6]. The NLP assists human-to-machine communication very efficiently that sequentially improves the overall efficiency and decision-making of the businesses. The NLP relates to how individual interacts, that consist of emotions, speech, and text. Fake news detection has gained much consideration in the NLP research field to mitigate the time-consuming human activity and burdensome data verification [7]. Despite that, the process of estimating the validity of news remains a challenge even for automatic systems.

Kumar et al. [8] gather 1356 news samples from different clients through media sources and Twitter including PolitiFact and construct various data sets for the fake and real news stories. We compared many advanced methods including attention mechanism, convolution neural network (CNN), long short term memory (LSTM), and ensemble approaches. Roy et al. [9] developed deep learning (DL) algorithms to identify fake news and classify them to the pre-determined fine-grained classes. Firstly, we designed CNN and bidirectional LSTM (Bi-LSTM) based systems. The representation attained from these two methods is given to a multilayer perceptron (MLP) for the last classification.

Aslam et al. [10] presented an ensemble-based DL method for classifying news as real or fake. Because of the nature of dataset traits, two DL methods have been employed. For the textual attributes “statement,” Bi-LSTM-gated recurrent unit (GRU)-dense DL method has been utilized, for the residual characteristics, dense DL algorithm has been employed.

In Agarwal et al. [11], researchers have experimented and discussed word embedding (GloVe) for text pre-processing to establish lingual relationships and create a vector space of words. The presented method is the combination of CNN and recurrent neural network (RNN) frameworks that have accomplished standard outcomes in predicting fake news, with the effectiveness of word embedding complementing the overall method. Furthermore, to guarantee the prediction quality, several model parameters were recorded and tuned for the optimal result.

Khanam et al. [12] make research analytics based fake news detection and examine the conventional machine learning (ML) methods for choosing the best, to construct a method of a product using supervised ML method, which could categorize fake news as false or true, by utilizing python scikit-learn, NLP for text analysis. Bangyal et al. [13] developed a precise model for SA of fake news. The fake news datasets contain fake news; the study initiates by data pre-processing (replaces the stemming, tokenization, noise removal, and missing value). The study employed a semantic method with inverse document frequency and term frequency weighting for representing information. In the evaluation and measuring stage, we employed 8 ML approaches.

This study designs a novel Chaotic Ant Swarm with Weighted Extreme Learning Machine (CAS-WELM) for Cybersecurity Fake News Detection and Classification. The goal of the CAS-WELM system is to discriminate news into fake and real. The CAS-WELM technique initially pre-processes the input data and Glove technique is used for word embedding process. Then, N-gram based feature extraction technique is derived to generate feature vectors. Lastly, WELM model is applied for the detection and classification of fake news, in which the weight value of the WELM model can be optimally adjusted by the use of CAS algorithm. The performance validation of the CAS-WELM technique is carried out using the benchmark dataset and the results are inspected under several dimensions.

2  The Proposed Model

In this study, a novel CAS-WELM technique has been developed for Cybersecurity Fake News Detection and Classification. The CAS-WELM technique mainly intends to discriminate news into fake and real. The CAS-WELM technique undergoes different stages of operations namely pre-processing, Glove based word embedding, N-gram based feature extraction, WELM based classification, and CAS based parameter optimization. Besides, the weight value of the WELM model can be optimally adjusted by the use of CAS algorithm.

2.1 Pre-processing

The data set is considered into two groups, true category, and false category. Data visualization assists in comprehending comparative data mean by demonstrating information in visual contexts, namely graphs or maps. This makes it easy to spot outliers, trends, and patterns in massive datasets by creating the data to analyze for the human mind. The data set is categorized into two classes, original and fake news. The fake news class is denoted as ‘0’ and true news class is denoted as ‘1’. When certain words exist in the group of a corpus, then the word is removed [14]. Data pre-processing is a major phase that includes data manipulation beforehand it is implemented, to improve efficacy. It includes data transformation and cleansing. To remove the stop word from the sentence, the text can be separated into words, and then it is verified to understand whether the word exist in the Natural Language Toolkit (NLTK) list of stop words. Stemming represents the extraction of word root or stems form that may or may not completely reflects semantic intellectual. The procedure of lemmatization is the decrease of inflectional format generally useful word-to common form. Glove embedding and Keras embedding layer, utilized to train NN system on textual information. This is a flexible layer, utilized for loading pre-trained GloVe embedding of hundred dimensions.

2.2 N-gram Based Feature Extraction

Consider xi∈Rd represent the word vector for ith word in a sentence of d dimension. Where x∈RL×d signify the input sentence using length L . Take k as the filter length, also m∈Rk×d represent a filter for the convolutional process. For all the location, j in the sentence, a window vector wj consist of k successive word vectors are evaluated,

wj=[xj˙,xj+1.,…,xj+k−1] (1)

Now, the comma represents row vector concatenation. A filter m integrates to the window vector ( k−grams ) and all the locations in an approach to construct a feature map c∈RL−k+1 ; all the elements cj of feature maps for window vector wj is generated by:

cj=f(wj∘m+b) (2)

whereas ∘ indicates element-by-element multiplication, b∈R show a bias term and f denotes a non-linear conversion with probable kinds such as sigmoid, hyperbolic tangent, linear, rectified linear unit (ReLU), softmax, and so on. In this case, ReLU is employed. A filter amount is utilized for producing feature map [15]. For n filters of equivalent size, the generated n feature map is rearranged as feature representation for all the window wj,

W=[c1;c2;…;cn] (3)

Here, ci denotes the feature map generated using the ith filter and Semicolon signifies column vector concatenation. All the rows Wj of W∈R(L−k+1)×n represent the feature depiction generated from n feature for window vector at location provided by j .

2.3 WELM Based Classification

Beforehand elaborating on the WELM, firstly presented the fundamental extreme learning machine (ELM). Using the mapping datasets {χi′,yj}∈ℜp×ℜc(i=1,2,…,n) , the output of generalized single layer feed forward network (SLFN) using activation function h(x′) and q hidden node can be formulated by using the following equation. Fig. 2 demonstrates the structure of WELM.

Figure 2: Structure of WELM

oi=∑k=1qβkhk(χi′)=∑k=1qβkh(wk,bk,xi′), (4)

In which i=1,2,…,n,wk=[wk1,wk2,…,wkp]T characterizes the input weight connect the kth hidden and input nodes, bk signifies the bias of kth hidden node, βk=[βk1,βk2,…,βkc]T shows the output weight linking the kth hidden and output nodes, and 0i represents the predicted output of ith sample. The widely employed activation function in ELM includes multiquadric function, Gaussian RBF function, sigmoid function, and hard limit function [16].

Hβ=O, (5)

In which H represent the hidden neuron output matrix of SLFN

H=H(w1,…,wq,b1,…,bq,x1′,…,xn′)=[h(x1′)⋮h(xn′)] (6)

=[h(w1,b1,x1′)⋯h(wq,bq,x1′)⋮⋱⋮h(w1,b1,x1′)⋯h(wq,bq,x1′)]n×q

Here, the ith row of H represent the output of hidden node regarding the input samples xi′ , and the kth column of H shows the output of kth hidden node regarding the input sample x1′,x2′,,..,xn′ .

β indicates the weight matrix linking the output and hidden layers, as follows

β=[β1T⋮βqT]q×c (7)

O represents the predicted label matrix, and all the rows represent the output vector of single instance. O is determined by

O=[o1T⋮TOn]=[o11⋯01c⋮⋱⋮on1⋯0nc] (8)

The aim of trained SLFN is to reduce the output errors, that is, approximate the input sample with zero error

∑i=1n‖0i−yi‖=‖O−Y‖=0 (9)

Whereas y=[y1T⋮Tyn]=[y11⋯y1c⋮⋱⋮yn1⋯ync] represents the target output matrix.

Hβ=Y (10)

Aimed at ELM, the bias bk of hidden neurons and the weight wk of input connection is independently and arbitrarily chosen [17]. When this parameter is allocated, Eq. (10) is transformed to linear method and the β output weight matrix is systematically defined by detecting the least-square solutions of linear method as

minβ‖Hβ−Y‖ (11)

The optimum solution of Eq. (11) is

β^=H†Y=(HTH) (12)

Whereas H† means the Moore-Penrose generalized inverse of hidden neuron output matrix H . The attained β^ could guarantee minimally trained error, attain optimum generalization capability, and prevent plunging to local optimal because β^ is exclusive.

f(x′)=h(x′)β^=h(x′)H†y (13)

While constructing the ELM classification, we determine a n×n diagonal matrix W , that diagonal component Wii represents the weight of trained instance χi′ . Accurately, when χi′ belonging to the majority class, the Wii weight is comparatively lesser when compared to the samples that belong to the minority class.

β^=H†Y=(HTWH)−1HTWT (14)

Next, Eq. (13) becomes

f(x′)=h(x′)β^=h(x′)(HTWH)−1HTWT (15)

Mainly, it consists of two systems to assign the weight to the sample of the two classes:

W1=Wii={1/nPifxi′∈minorityclass1/nNifxi′∈majorityclass (16)

Or

W2=Wii={0.618/nPifxi′∈minorityclass1/nNifxi′∈majorityclass, (17)

Here, Wl and W2 denotes weighting systems, nP & nN indicates the amount of instances of the minority and majority classes, correspondingly.

2.4 Parameter Optimization Using CAS Algorithm

For tuning the weight values of the WELM model, the CAS is used. Recently, a SI optimization method named CAS approach is presented for solving the optimization issue according to chaos concept [18]. The CAS algorithm is mathematically modelled by the following equation:

yi(t)=yi(t−1)(1+ri),

zid(t)=Δexp⁡((1−exp⁡(−ayi(t)))(3−ΨdΔ))−

7.5Ψd×Vi+exp⁡(−2ayi(t)+b)×(pbestd(t−1)−zid(t−1)), (18)

In which yi(t) represent the organization parameter of the CAS and Δ=zid(t−1)+7.5/(Ψd×φi) . It handles chaotic behavior of one ant. ri indicates the organization variances of one ant that is a positive constant lesser than 1. Ψi defines the search array of ith ant in dth dimension. ∅i control the moving proportion of ith ant search region. pbest (t−1) denotes the optimal location that the single ant and neighbors have established with t−1 time step. Now the neighbor is fixed to be global neighbor; viz., each ant is the neighbor of one another. Usually, The ant exchanges data through direct or indirect transmission models. Owing to the efficient transmission, the effect of organization becomes robust as time changes. At last, each ant walks through the optimal route to forage food. As time grows, the impact of the organization parameter yi(t) on the behavior of all the ants become strong through the organization variable ri .

Lastly, with the impact of pbestd(t−1) and yi(t) , the state of zid(t) would converge to the global optimal location. ri and Ψd represent significant variables. ri has an impact on the convergence rate of CAS approach. When ri is smaller, the convergence rate of the CAS approach would be slower and the implementation time would take time. When ri is larger, the convergence rate of CAS approach would be faster thereby the optimum solution mightn’t be established. When ri is fixed to be 0, the behavior of ant would be chaotic continually and the CAS approach could not converge to a certain location. Moreover, slight variation of organization impact is chosen, ri is fixed to be 0≤ri≤0.5 . The actual equation of ri based on the runtime and certain issues.

To support ant to have a distinct organization variable, fix ri=0.1+0.2×rand , whereas rand represents a uniform distribution arbitrary value within [0,1]. Ψd has an impact on the search space of the CAS approach. When Ψd is smaller, the search space would be larger. when the value of Ψd is larger, the search space would be smaller. The search space is fixed to be [−wd/2,wd/2] , and wd ≂ 7.5/Ψd.

3  Experimental Validation

The experimental result analysis of the CAS-WELM technique is validated using benchmark dataset. The initial dataset is named as ISOT Fake News Dataset [19] (Sample Set-1), comprising 44,898 articles (21,417 instances under truthful articles and 23,481 under fake articles). The second Kaggle dataset [20] (Sample Set-2) includes 20,386 articles employed to train the dataset and 5,126 articles are applied to test the dataset. The third dataset [21] (sample set-3) comprises 3,352 articles, both fake and true. The final dataset (Sample Set-4) includes the combination of the dataset.

Tab. 1 and Fig. 3 demonstrate the accuracy analysis of the CAS-WELM technique with other ones [22]. The results indicated that the k-nearest neighbor (KNN) model has attained worse classification results than the other methods. In addition, the logistic regression (LR) model has obtained slightly improved classification performance over the KNN model. Moreover, the Localized Support Vector Machine (LSVM), MLP, and Bagging-decision tree (DT) model has accomplished moderately increased outcomes. Though the random forest (RF) model has resulted in competitive outcome, the CAS-WELM technique has outperformed the other methods with the higher accuracy of 99.46%, 96.32%, 96.58%, and 94.89% on the test sample sets 1–4 respectively.

Figure 3: Accuracy analysis of CAS-WELM technique with existing approaches

Tab. 2 and Fig. 4 illustrate the precision analysis of the CAS-WELM approach with other ones. The results indicated that the KNN technique has attained least classification outcomes over the other methods. Besides, the LR approach has reached somewhat higher classification performance over the KNN technique. Moreover, the LSVM, MLP, and Bagging-DT methodology have accomplished moderately increased outcomes. Then, the RF system has resulted in competitive outcome, the CAS-WELM system has demonstrated the other methods with the superior precision of 99.61%, 95.74%, 99.24%, and 95.35% on the test sample sets 1–4 correspondingly.

Figure 4: Precision analysis of CAS-WELM technique with existing approaches

Tab. 3 and Fig. 5 showcases the recall analysis of the CAS-WELM approach with other ones. The outcomes referred that the KNN algorithm has gained poor classification results over the other methods. Similarly, the LR technique has obtained slightly enhanced classification performance over the KNN technique. Likewise, the LSVM, MLP, and Bagging-DT approach has accomplished moderately increased outcomes. Eventually, the RF system has resulted in competitive outcome, the CAS-WELM method has exhibited the other techniques with the maximal recall of 100%, 98.24%, 100%, and 95.84% on the test sample sets 1–4 correspondingly.

Figure 5: Recall analysis of CAS-WELM technique with existing approaches on test sample sets 1–4

Tab. 4 and Fig. 6 illustrates the F-score analysis of the CAS-WELM technique with other ones. The results show that the KNN method has gained minimal classification outcomes over the other approaches. Besides, the LR technique has obtained somewhat enhanced classification performance over the KNN technique. Moreover, the LSVM, MLP, and Bagging-DT approach has accomplished moderately higher outcomes. At last, the RF system has resulted in competitive outcome, the CAS-WELM technique has outperformed the other methods with the increased F-score of 99.36%, 96.48%, 98.88%, and 96.23% on the test sample sets 1–4 correspondingly.

Figure 6: F-score analysis of CAS-WELM technique with existing approaches on test sample sets 1–4

Fig. 7 demonstrates the accuracy and loss graph analysis of the CAS-WELM technique on the test sample sets 1 and 2. The results show that the accuracy value tends to increase and loss value tends to decrease with an increase in epoch count. It is also observed that the training loss is low and validation accuracy is high on test sample sets 1 and 2.

Figure 7: Accuracy and loss analysis of CAS-WELM technique under test sample sets 1 and 2

Fig. 8 offers the accuracy and loss graph analysis of the CAS-WELM methodology on the test sample sets 3 and 4. The outcomes demonstrated that the accuracy value tends to be higher and loss value tends to lower with higher epoch count. It is also experiential that the training loss is minimum and validation accuracy is high on the test sample sets 3 and 4.

Figure 8: Accuracy and loss analysis of CAS-WELM technique under test sample sets 3 and 4

4  Conclusion

In this study, a novel CAS-WELM technique has been developed for Cybersecurity Fake News Detection and Classification. The CAS-WELM technique mainly intends to discriminate news into fake and real. The CAS-WELM technique undergoes different stages of operations namely pre-processing, Glove based word embedding, N-gram based feature extraction, WELM based classification, and CAS based parameter optimization. Besides, the weight value of the WELM model can be optimally adjusted by the use of CAS algorithm. The performance validation of the CAS-WELM technique is carried out using the benchmark dataset and the results are inspected under several dimensions. The experimental results reported the enhanced outcomes of the CAS-WELM technique over the recent approaches. In the future, advanced deep learning models can be utilized to classify and detect fake news in social networking platform.

Acknowledgement: The authors deeply acknowledge the Researchers supporting program (TUMA-Project-2021-27) Almaarefa University, Riyadh, Saudi Arabia for supporting steps of this work. The authors would like to acknowledge the support of Prince Sultan University for paying the Article Processing Charges (APC) of this publication.

Funding Statement: This research was supported by the Researchers Supporting Program (TUMA-Project-2021-27) Almaarefa University, Riyadh, Saudi Arabia. Taif University Researchers Supporting Project number (TURSP-2020/161), Taif University, Taif, Saudi Arabia.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.