[picture] | Computers, Materials & Continua | [picture]

DOI: 10.32604/cmc.2023.033045

Article

Parameter-Tuned Deep Learning-Enabled Activity Recognition for Disabled People

Mesfer Al Duhayyim*

Department of Computer Science, College of Sciences and Humanities-Aflaj, Prince Sattam Bin Abdulaziz University, Al-Aflaj, 16273, Saudi Arabia
Corresponding Author: Mesfer Al Duhayyim. Email: m.alduhayyim@psau.edu.sa
Received: 06 June 2022; Accepted: 07 July 2022

Abstract: Elderly or disabled people can be supported by a human activity recognition (HAR) system that monitors their activity intervenes and patterns in case of changes in their behaviors or critical events have occurred. An automated HAR could assist these persons to have a more independent life. Providing appropriate and accurate data regarding the activity is the most crucial computation task in the activity recognition system. With the fast development of neural networks, computing, and machine learning algorithms, HAR system based on wearable sensors has gained popularity in several areas, such as medical services, smart homes, improving human communication with computers, security systems, healthcare for the elderly, mechanization in industry, robot monitoring system, monitoring athlete training, and rehabilitation systems. In this view, this study develops an improved pelican optimization with deep transfer learning enabled HAR (IPODTL-HAR) system for disabled persons. The major goal of the IPODTL-HAR method was recognizing the human activities for disabled person and improve the quality of living. The presented IPODTL-HAR model follows data pre-processing for improvising the quality of the data. Besides, EfficientNet model is applied to derive a useful set of feature vectors and the hyperparameters are adjusted by the use of Nadam optimizer. Finally, the IPO with deep belief network (DBN) model is utilized for the recognition and classification of human activities. The utilization of Nadam optimizer and IPO algorithm helps in effectually tuning the hyperparameters related to the EfficientNet and DBN models respectively. The experimental validation of the IPODTL-HAR method is tested using benchmark dataset. Extensive comparison study highlighted the betterment of the IPODTL-HAR model over recent state of art HAR approaches interms of different measures.

Keywords: Human activity recognition; disabled person; artificial intelligence; computer vision; deep learning

1  Introduction

Recently, Human Activity Recognition (HAR) is one such effective method to assist disabled persons. Owing to the availability of accelerometers and sensors, low energy utilization and minimum cost, and developments in artificial intelligence (AI), computer vision (CV), Internet of Things (IoT), and machine learning (ML), several applications were framed with the help of human centered design monitoring for recognizing, detecting, and categorizing human behaviors, and research scholars have offered numerous techniques related to this topic [1]. HAR becomes a necessary tool in monitoring the dynamism of person, and it could be established with the help of ML techniques. HAR refers to a technique which automatically analyse and recognizes human activities on the basis of data obtained from several wearable devices and smartphone sensors, like location, gyroscope sensors, time, accelerometer sensors, and other environmental sensors [2]. Whenever compiled with other technologies, like IoT, it is utilized in an extensive range of application zones like sports, industry, and healthcare. World Health Organization (WHO) survey stated that nearly 650 million working age individuals across the globe were disabled [3]. Also, there are over six million people in Indonesia, as per a research carried out by Survei Sosial Ekonomi Nasional (Susenas) in 2012. There are presently inadequate amenities to meet the people needs with disability [4]. One is the need for a companion for monitoring its actions. Fig. 1 depicts the role of AI in disabled people.

Figure 1: Role of AI in disabled people

In recent times, latest smartphones fortified with various embedded sensors, like gyroscopes and accelerometers, were used as an alternate platform for HAR [5]. The smartphone-related HAR mechanism refers to an ML method which can be positioned on the subject’s smartphone and endlessly detects his or her actions while the smartphone can be attached to body parts of an individual [6]. This algorithm considers the benefits of the current smartphone computing sources for developing a real-time mechanism. In general, advancing these systems can be carried in 4 basic steps they are feature extraction, data collection, classification, and windowing. Feature extraction can be regarded as the most critical step, as it decides the model’s overall performance [7]. This step is established either utilizing conventional deep learning (DL) or ML techniques. In classical ML techniques, the field experts extract handcrafted or heuristic features manually in frequency as well as time domains. There exist several time domain features like standard deviation, mean, max, min, correlation, and so on. Additionally, there comes several frequency domain features namely time between peaks, energy, entropy, and much more. But handcrafted features contain certain restrictions in both fields. Firstly, they were dependent on human experience and field knowledge [8]. This knowledge can be helpful in some issues with particular settings; however, it could not be generalized for the same issue having distinct settings [9]. Moreover, the human experience was just utilized for extracting shallow features, namely statistical data, however, fails to distinguish among activities with approximately similar patterns (like standing and sitting actions in HAR). There were numerous studies which employed the conventional ML techniques to build smartphone-related HAR [10]. To overcome this limitation, DL techniques can be utilized. In DL techniques, the features are automatically learned by utilizing more than one hidden layer rather than extracted manually by the field specialist.

Yadav et al. [11] develop effective activity and fall detection systems (ARFDNet). At this point, the raw RGB video is passed to impose estimation system for extracting skeleton features. The skeleton coordinate is later preprocessed and inputted in a sliding window manner to develop especially convolution neural network (CNN) and gated recurrent unit (GRU), for learning the Spatio-temporal dynamics existing in the information. In [12], the authors proposed Wi-Sense—a human activity recognition (HAR) scheme which utilizes a CNN to identify human activity of the human on the basis of the environment-independent fingerprint derived from the Wi-Fi channel state information (CSI). Firstly, Wi-Sense captures the CSI through a typical Wi-Fi network interface card. Wi-Sense employs the CSI ratio model for reducing the noise and the effect of the phase offset. Additionally, it employs the principal component analysis (PCA) for removing data redundancy. Basly et al. [13] developed a deep temporal residual system for HAR that aim is to improve spatiotemporal features for enhancing performance of system. Eventually, we adopted a deep residual convolutional neural network (RCN) to preserve discriminatory visual features relayed to appearance and long short-term memory neural network\(LSTM-NN) for capturing the long-term temporal evolution of action.

In [14], the different textural characteristics such as point feature, grey level co-occurrence matrix, and local binary pattern accelerated powerful feature are retrieved from video activity that is a proposed work and classifiers such as k-nearest neighbor (KNN), probabilistic neural network (PNN), support vector machine (SVM), as well as the presented classifiers are utilized for classifying the activity. In [15], a futuristic architecture has been experimented and proposed to construct a precision-centric HAR model by examining the information attained from Personalized Positions Detection Scheme (PPDS) and Environment Monitoring Scheme (EMS) using ML techniques namely AdaBoost, SVM, and PNN. Furthermore, the presented technique employs the Dempster-Shafer Theory (DST)-based complete sensor data fusion thus improving the performance of global HAR.

This study develops an improved pelican optimization with deep transfer learning enabled HAR (IPODTL-HAR) system for disabled persons. The major goal of the IPODTL-HAR method is to recognize the human activities for disabled person and improve the quality of living. The presented IPODTL-HAR model follows data pre-processing to improve the quality of the data. Besides, EfficientNet model can be applied to derive a useful set of feature vectors and the hyperparameters are adjusted by the use of Nadam optimizer. Finally, the IPO with deep belief network (DBN) technique is utilized for the recognition and classification of human activities. The utilization of Nadam optimizer and IPO algorithm helps in effectually tuning the hyperparameters related to the EfficientNet and DBN models respectively. The experimental validation of the IPODTL-HAR method can be tested using benchmark dataset.

The rest of the paper is given as follows. Section 2 introduces the proposed model and Section 3 discusses the performance validation. Finally, Section 4 concludes the study.

2  The Proposed HAR Model

In this study, a new IPODTL-HAR model was introduced to recognize the human activities of disabled persons and improve the quality of living. The presented IPODTL-HAR model follows data pre-processing to improvise the data quality. Besides, EfficientNet method can be applied to derive a useful set of feature vectors and the hyperparameters are adjusted by the use of Nadam optimizer. Finally, the IPO with DBN model is utilized for the recognition and classification of human activities. Fig. 2 demonstrates the block diagram of IPODTL-HAR approach.

Figure 2: Block diagram of IPODTL-HAR approach

2.1 Feature Extraction: EfficientNet Model

In this study, the EfficientNet model is utilized to produce a collection of feature vectors. The concept of transfer learning is faster than training a model from scratch, overcomes limitations of the data as in the healthcare fields, increases performance, achieves less training time, and is computationally inexpensive [16]. In this work, the pre-trained convolutional neural network (CNN) model finetuned is the EfficientNet. In 2019 Google proposed a multi-dimensional mixed model scaling mechanism (EfficientNet sequence network) that has received considerable interest in the academic field. For exploring a model scaling mechanism that consider accuracy and speed, the EfficientNet series network that instantaneously scale the three dimension of depth, width, and resolution of the network is initially presented. The input is a three-channel color images with a height H and a width W. After the layer-wise convolution, the network can able to learn respective attributes to increase network efficiency by varying the number of channels, increase the network from the perception of input images and improves the network learning efficacy by proportionally reducing or enlarging the size of input images to improve the network. Also, it aims at increasing or decreasing the amount of network layers such that the network could improve network efficiency and learn more explicit feature information. In the EfficientNet, the composite parameter is utilized for concurrently performing the scaling of the abovementioned three dimensions, thus enhancing the total network performance.

For hyperparameter tuning of the EfficientNet model, the Nadam optimizer is utilized. The Nadam optimizer try to incorporate Nesterov accelerated adaptive moment estimation to the Adam [17]. The great advantage of this presented technique is that they assist to implement highly accurate phase in the gradient direction by updating mechanism with the momentum phase before the gradient computation and this can be demonstrated in the following equation:

wt=wt−1−α×m¯tv^t+ε,(1)

where

m¯t=(1−β1,t)g^t+β1,t+1 m^t,

m^t=mt1−∏i=1t+1β1i,(2)

g^t=gt1−∏i=1t+1β1i.

2.2 Activity Classification: DBN Model

For the recognition and classification of human activities, the DBN technique is utilized. A DBN refers to a neural network mechanism comprising of a multiple restricted Boltzmann machine (RBM) [18], the essential RBM unit is an energy-based mechanism.

The energy of RBM is defined using state (v,h) is formulated by:

E(v,h|θ)=−∑i=1naivi−∑j=1mbjhj−∑i=1n∑j=1mviwijhj(3)

where, θ=(wij,ai,bi) refers to the RBM parameter, the neuron count in the visible and hidden layers is indicated by n & m, the input unit of visible layer is denoted by v, and the state of neuron i in the visible layer is signified as vi. The bias value is fixed to ai. The output unit of the hidden layer is represented as h, the state of the hidden layer neuron j is denoted by hj, and the bias value is bj. The connection weight of i and j neurons are determined by wij. The (v,h) combined likelihood distribution is attained by:

p(v,h|θ)=exp⁡ (−E(v,h|θ))/∑v,hexp⁡(−E(v,h|θ))(4)

The marginal likelihood distribution of (v,h) is attained as follows:

p(v|θ)=∑hexp⁡ (−E(v,h|θ))/∑v,hexp⁡(−E(v,h|θ))(5)

p(h|θ)=∑vexp⁡ (−E(v,h|θ))/∑v,hexp⁡(−E(v,h|θ))(6)

If vi=1 or hj=1, the conditional probability function is:

p(hj=1|v,θ)=sigm(bj+∑i=1nviwij)(7)

p(vi=1|h,θ)=sigm(ai+∑j=1mwijhj)(8)

From the above equation, the activation function is denoted by the term sigm.

DBN feature extraction is a layer-wise learning mechanism of multiple RBMs, involving reverse reconstruction and forward learning. DBN maps complicated signals to output and has better feature extraction capability.

DBN resolves the optimization issue of deep neural networks through layer-wise training mechanism and provides the network a good initial weight by layer-wise training, which means that the network could reach the optimum solution if it is finetuned.

2.3 Hyperparameter Tuning: IPO Algorithm

To tune the hyperparameters related to the DBN technique, the IPO algorithm is utilized. The PO algorithm is a new metaheuristic optimization technique motivated by the behavior of pelican hunting [19]. Fast convergence speed, simple calculation, and adjustment parameters are the benefits of the proposed algorithm. They are found in the warm water and primarily live in swamps, lakes, rivers, and coasts. Generally, pelicans live in flocks; they were good at swimming and flying. They have excellent observation skills, sharp eyesight in flight, and they mainly feed on fish. When pelican finds school of fish, they arrange itself in a U-shape or line to dive from the sky toward the prey and make use of its wings to flap the water, forcing fish to move upward, and collects the fish in its throat pouch. Afterward defining the position of the prey, the pelican rushes toward the prey from a 1020 m height and dives straight into the water for hunting. The mathematical model of the PO algorithm was constructed based on the abovementioned description.

(1) Initialization: Assume that there exist N pelicans in M dimension space, the location of i−th pelicans in the M dimension space is Pi=[pi1,pi2,…,pim,…,piM], the location P of N pelicans is formulated in the following:

P=[P1P2⋮Pi⋮PN]=[p11p12p1mp1Mp21p22p2mp2M⋮⋮⋮⋮pi1pi2pimpiM⋮⋮⋮⋮pN1pN2pNmpNM],i=1,2,……,N(9)

In Eq. (9), pim refers to the location of i-th pelicans in the m-th dimensions. Initially, they are distributed randomly in a specific range, also the updating location of the pelicans is defined by,

Pim=lowm+random⋅(upm−lowm),i=1,2,…,N,m=1,2,…,M,(10)

In Eq. (10), lowm, upm represent the search range of pelican; random denotes a random number lies within zero and one.

(2) Moving towards prey: Here, the pelican finds the prey position and rushes toward the prey from higher altitude. The random distribution of prey in the searching space surges the exploration capability, and the updating of pelicans’ position at every iteration can be determined as follows

Pimt+1={Pimt+rand⋅(Smt−λ⋅Pimt),F(PS)<F(Pi)Pimt+rand⋅(Pimt−Smt),F(PS)≥F(Pi)(11)

In Eq. (11), the current iteration number is represented as t; the location of i‐th pelicans in m‐th dimension can be signified by Pimt; the location of prey in m-th dimension can be denoted as Smt; λ is randomly equivalent to one or two; the objective function value is denoted as F(P); the fitness value of i-th pelicans in the m-th dimension is represented by F(P).

(3) Winging on the water surface: Afterward, the pelican reaches the surface of water, they spread its wings toward the water surface for moving the fish upward, collecting the prey in its throat pouch. The mathematical formulation of pelican during hunting is given in the following

Pimt+1=Pimt+γ⋅(T−tT)⋅(2⋅random−1)⋅Pimt(12)

In Eq. (12), existing amount of iterations is represented by t; maximal iteration count is denoted as T; the neighborhood radius of Pimt is signified as γ⋅(T−tT), and it signifies the radius of the neighborhood of population member to locally search near every member to converge to an effective solution; random refers to a random value lies within zero and one.

After enhancing the PO approach, the optimization efficacy is additionally enhanced. The specific improvement strategy is shown in the following.

(1) Initialization strategy: The Tent chaotic map can be utilized for replacing the randomly generated model to initialize the pelican afterward the Tent chaotic mapping is developed, and the Eq. (10) is rewritten by using:

pim=lowm+Tent⋅(upm−lowm),i=1,2,…,N,m=1,2,…,M,(13)

Tentt+1={Tenttz,Tentt∈[0,z](1−Tentt)(1−z),Tentt∈[z,1](14)

From the expression, t denotes a current amount of iterations; T indicates a maximal iteration number; where ∈(0,1), Tentt∈[0,1],t=1,2,…,T.

In this phase, the pelican position can be initialized by the Tent chaotic map that assists in improving the global search performance.

(2) Moving towards prey: In the phase, the dynamic weight factor θ assists the pelican to continually upgrade their location. Initially, at the iteration, θ contains a larger value, once the pelican was capable of performing a best global search, and lastly at the iteration θ adaptively reduces and now the pelican is capable of performing a best local search when improving the convergence rate. Eq. (11) is rewritten by:

Pimt+1={θ=e2(1−t/T)−e−2(1−t/T)e2(1−t/T)+e−2(1−t/T)Pimt+rand⋅(Smt−Pimt)⋅θ,F(PS)<F(Pi)Pimt+rand⋅(Pimt−Smt)⋅θ,F(PS)≥F(Pi)(15)

The IPO approach extracts a fitness function for gaining enhanced classifier outcomes. It determines a positive integer for denoting superior outcome of the candidate solutions. In this paper, the reduction of the classifier error rate was assumed as the fitness function, as shown in Eq. (16).

fitness(xi)=ClassifierErrorRate(xi)

=number of misclassified samplesTotal number of samples∗100(16)

3  Experimental Validation

This section inspects the activity recognition performance of the IPODTL-HAR model using a dataset comprising 10299 samples. The dataset includes samples under six class labels, as given in Table 1. The IPODTL-HAR method is simulated by utilizing Python 3.6.5 tool.

Fig. 3 demonstrates the confusion matrix created by the IPODTL-HAR model on 70% of TR data and 30% of TS data. On 70% of TR data, the IPODTL-HAR model has recognized 1174 samples under C-1 class, 1315 samples under C-2 class, 1348 samples under C-3 class, 1160 samples under C-4 class, 1050 samples under C-5 class, and 927 samples under C-6 class. Also, on 30% of TS data, the IPODTL-HAR method has recognized 542 samples under C-1 class, 556 samples under C-2 class, 559 samples under C-3 class, 509 samples under C-4 class, 448 samples under C-5 class, and 397 samples under C-6 class.

Figure 3: Confusion matrices of IPODTL-HAR approach (a) 70% of TR data and (b) 30% of TS data

Table 2 and Fig. 4 report the overall HAR outcomes of the IPODTL-HAR model on the applied data. The results inferred that the IPODTL-HAR method has shown enhanced results under all cases. For example, on 70% of TR data, the IPODTL-HAR model has resulted to average accuy of 98.91%, precn of 96.74%, specy of 99.35%, Fscore of 96.67%, and MCC of 96.02%. Moreover, on 30% of TS data, the IPODTL-HAR technique has resulted to average accuy of 99.15%, precn of 97.44%, specy of 99.49%, Fscore of 97.39%, and MCC of 96.88%.

Figure 4: Result analysis of IPODTL-HAR approach under 70% of TR and 30% of TS data

Fig. 5 illustrates the confusion matrix created by the IPODTL-HAR method on 80% of TR data and 20% of TS data. On 80% of TR data, the IPODTL-HAR approach has recognized 1363 samples under C-1 class, 1464 samples under C-2 class, 1523 samples under C-3 class, 1337 samples under C-4 class, 1192 samples under C-5 class, and 1089 samples under C-6 class. Additionally, on 20% of TS data, the IPODTL-HAR method has recognized 347 samples under C-1 class, 377 samples under C-2 class, 391 samples under C-3 class, 319 samples under C-4 class, 321 samples under C-5 class, and 248 samples under C-6 class.

Figure 5: Confusion matrices of IPODTL-HAR approach (a) 80% of TR data and (b) 20% of TS data

Table 3 and Fig. 6 signify the overall HAR outcomes of the IPODTL-HAR method on the applied data. The results indicated the IPODTL-HAR method has displayed improvised results under all cases. For instance, on 80% of TR data, the IPODTL-HAR algorithm has resulted to average accuy of 98.90%, precn of 96.70%, specy of 99.34%, Fscore of 96.67%, and MCC of 96.02%. Moreover, on 20% of TS data, the IPODTL-HAR method has resulted to average accuy of 99.08%, precn of 97.29%, specy of 99.44%, Fscore of 97.22%, and MCC of 96.67%.

Figure 6: Result analysis of IPODTL-HAR approach under 80% of TR and 20% of TS data

The training accuracy (TA) and validation accuracy (VA) gained by the IPODTL-HAR algorithm on test dataset is illustrated in Fig. 7. The experimental outcome denoted the IPODTL-HAR approach has reached maximal values of TA and VA. To be specific the VA is superior than TA.

Figure 7: TA and VA analysis of IPODTL-HAR methodology

The training loss (TL) and validation loss (VL) reached by the IPODTL-HAR method on test dataset are established in Fig. 8. The experimental outcome implied that the IPODTL-HAR technique has accomplished least values of TL and VL. Particularly, the VL is lesser than TL.

Figure 8: TL and VL analysis of IPODTL-HAR methodology

A clear precision-recall analysis of the IPODTL-HAR approach on test dataset is illustrated in Fig. 9. The figure represented the IPODTL-HAR technique has resulted to enhanced values of precision-recall values under all classes.

Figure 9: Precision-recall curve analysis of IPODTL-HAR methodology

A brief ROC inspection of the IPODTL-HAR method on test dataset is demonstrated in Fig. 10. The results indicated the IPODTL-HAR approach has displayed its ability in categorizing distinct classes on the test dataset.

Figure 10: ROC curve analysis of IPODTL-HAR methodology

To exhibit the better performance of the IPODTL-HAR model, a comparison study with recent models is made in Table 4 [20,21]. Fig. 11 provides a comprehensive accuy examination of the IPODTL-HAR model with recent models. The figure implied that the RF, NNN, and SVM models have demonstrated reduced accuy values of 86.18%, 87.50%, and 88.81% respectively. Along with that, the ANN and LSTM models have tried to exhibit somewhat reasonable accuy values of 91.83% and 93.97% respectively. However, the IPODTL-HAR model has demonstrated better performance over other models with maximum accuy of 99.15%.

Figure 11: Accuy analysis of IPODTL-HAR approach with existing methodologies

Fig. 12 presents a brief precnexamination of the IPODTL-HAR model with recent models. The figure implied that the RF, NNN, and SVM models have demonstrated reduced precn values of 82.70%, 85.86%, and 88.86% correspondingly. Also, the ANN and LSTM approaches have tried to exhibit somewhat reasonable precn values of 88.56% and 91.82% correspondingly. But, the IPODTL-HAR method has demonstrated superior performance over other models with maximal precn of 97.44%.

Figure 12: Precn analysis of IPODTL-HAR approach with existing methodologies

Fig. 13 provides a detailed specy investigation of the IPODTL-HAR technique with recent models. The figure denoted the RF, NNN, and SVM techniques have demonstrated reduced specy values of 80.96%, 82.76%, and 87.44% correspondingly. Additionally, the ANN and LSTM approaches have tried to display somewhat reasonable specy values of 92.20% and 94% correspondingly. But, the IPODTL-HAR method has illustrated better performance over other approaches with higher specy of 99.49%.

Figure 13: Specy analysis of IPODTL-HAR approach with existing methodologies

Fig. 14 offers a comprehensive Fscore inspection of the IPODTL-HAR method with recent models. The figure inferred the RF, NNN, and SVM methods have illustrated reduced Fscore values of 80.94%, 83.06%, and 88.80% correspondingly. Also, the ANN and LSTM techniques have tried to show somewhat reasonable Fscore values of 90.85% and 92.50% correspondingly. However, the IPODTL-HAR model has demonstrated better performance over other models with maximum Fscore of 97.39%.

Figure 14: Fscore analysis of IPODTL-HAR approach with existing methodologies

From the detailed results and discussion, it is assured that the IPODTL-HAR model has demonstrated superior performance over other models.

4  Conclusion

In this study, a new IPODTL-HAR model was introduced to recognize the human activities for disabled persons and improve the quality of living. The presented IPODTL-HAR model follows data pre-processing for enhancing the data quality. Besides, EfficientNet method was applied to derive a useful set of feature vectors and the hyperparameters are adjusted by the use of Nadam optimizer. Finally, the IPO with deep belief network (DBN) model is utilized for the recognition and classification of human activities. The utilization of Nadam optimizer and IPO algorithm helps in effectually tuning the hyperparameters related to the EfficientNet and DBN models respectively. The experimental validation of the IPODTL-HAR technique is tested using benchmark dataset. Extensive comparison study highlighted the betterment of the IPODTL-HAR method over recent state of art HAR approaches interms of different measures. In future, the proposed model can be employed in real time environment.

Funding Statement: The author received no specific funding for this study.

Conflicts of Interest: The author declares that there are no conflicts of interest to report regarding the present study.

References

1. A. B. Jayo, X. Cantero, A. Almeida, L. Fasano, T. Montanaro et al., “Location based indoor and outdoor lightweight activity recognition system,” Electronics, vol. 11, no. 3, pp. 360, 2022.

2. M. O. Raza, N. Pathan, A. Umar and R. Bux, “Activity recognition and creation of web service for activity recognition using mobile sensor data using azure machine learning studio,” Review of Computer Engineering Research, vol. 8, no. 1, pp. 1–7, 2021.

3. H. Raeis, M. Kazemi and S. Shirmohammadi, “Human activity recognition with device-free sensors for well-being assessment in smart homes,” IEEE Instrumentation & Measurement Magazine, vol. 24, no. 6, pp. 46–57, 2021.

4. S. K. Yadav, K. Tiwari, H. M. Pandey and S. A. Akbar, “A review of multimodal human activity recognition with special emphasis on classification, applications, challenges and future directions,” Knowledge-Based Systems, vol. 223, no. 8, pp. 106970, 2021.

5. M. Z. Uddin and A. Soylu, “Human activity recognition using wearable sensors, discriminant analysis, and long short-term memory-based neural structured learning,” Scientific Reports, vol. 11, no. 1, pp. 16455, 2021.

6. A. Vijayvargiya, V. Gupta, R. Kumar, N. Dey and J. M. R. S. Tavares, “A hybrid WD-EEMD sEMG feature extraction technique for lower limb activity recognition,” IEEE Sensors Journal, vol. 21, no. 18, pp. 20431–20439, 2021.

7. S. Mekruksavanich and A. Jitpattanakul, “Deep convolutional neural network with RNNs for complex activity recognition using wrist-worn wearable sensor data,” Electronics, vol. 10, no. 14, pp. 1685, 2021.

8. I. Bustoni, I. Hidayatulloh, A. Ningtyas, A. Purwaningsih and S. Azhari, “Classification methods performance on human activity recognition,” Journal of Physics: Conference Series, vol. 1456, no. 1, pp. 12027, 2020.

9. D. Sehrawat and N. S. Gill, “IoT based human activity recognition system using smart sensors,” Advances in Science, Technology and Engineering Systems Journal, vol. 5, no. 4, pp. 516–522, 2020.

10. A. S. A. Sukor, A. Zakaria, N. A. Rahim, L. M. Kamarudin, R. Setchi et al., “A hybrid approach of knowledge-driven and data-driven reasoning for activity recognition in smart homes,” Journal of Intelligent & Fuzzy Systems, vol. 36, no. 5, pp. 4177–4188, 2019.

11. S. K. Yadav, A. Luthra, K. Tiwari, H. M. Pandey and S. A. Akbar, “ARFDNet: An efficient activity recognition & fall detection system using latent feature pooling,” Knowledge-Based Systems, vol. 239, no. 7, pp. 107948, 2022.

12. M. Muaaz, A. Chelli, M. W. Gerdes and M. Pätzold, “Wi-Sense: A passive human activity recognition system using Wi-Fi and convolutional neural network and its integration in health information systems,” Annals of Telecommunications, vol. 77, no. 3–4, pp. 163–175, 2022.

13. H. Basly, W. Ouarda, F. E. Sayadi, B. Ouni and A. M. Alimi, “DTR-HAR: Deep temporal residual representation for human activity recognition,” The Visual Computer, vol. 38, no. 3, pp. 993–1013, 2022.

14. K. P. S. Kumar and R. Bhavani, “Human activity recognition in egocentric video using PNN, SVM, kNN and SVM+kNN classifiers,” Cluster Computing, vol. 22, no. S5, pp. 10577–10586, 2019.

15. V. Venkatesh, P. Raj, K. Kannan and P. Balakrishnan, “Precision centric framework for activity recognition using Dempster Shaffer theory and information fusion algorithm in smart environment,” Journal of Intelligent & Fuzzy Systems, vol. 36, no. 3, pp. 2117–2124, 2019.

16. J. Wang, Q. Liu, H. Xie, Z. Yang and H. Zhou, “Boosted EfficientNet: detection of lymph node metastases in breast cancer using convolutional neural networks,” Cancers, vol. 13, no. 4, pp. 661, 2021.

17. S. H. Haji and A. M. Abdulazeez, “Comparison of optimization techniques based on gradient descent algorithm: A review,” PalArch’s Journal of Archaeology of Egypt/Egyptology, vol. 18, no. 4, pp. 2715–2743, 2021.

18. X. Z. Wang, T. Zhang and R. Wang, “Noniterative deep learning: incorporating restricted Boltzmann machine into multilayer random weight neural networks,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 49, no. 7, pp. 1299–1308, 2019.

19. W. Tuerxun, C. Xu, M. Haderbieke, L. Guo and Z. Cheng, “A wind turbine fault classification model using broad learning system optimized by improved pelican optimization algorithm,” Machines, vol. 10, no. 5, pp. 407, 2022.

20. A. Hayat, M. Dias, B. Bhuyan and R. Tomar, “Human activity recognition for elderly people using machine and deep learning approaches,” Information, vol. 13, no. 6, pp. 275, 2022.

21. T. H. Tan, J. Y. Wu, S. H. Liu and M. Gochoo, “Human activity recognition using an ensemble learning algorithm with smartphone sensor data,” Electronics, vol. 11, no. 3, pp. 322, 2022.

[picture] | This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.