Brain-computer interfaces (BCIs) records brain activity using electroencephalogram (EEG) headsets in the form of EEG signals; these signals can be recorded, processed and classified into different hand movements, which can be used to control other IoT devices. Classification of hand movements will be one step closer to applying these algorithms in real-life situations using EEG headsets. This paper uses different feature extraction techniques and sophisticated machine learning algorithms to classify hand movements from EEG brain signals to control prosthetic hands for amputated persons. To achieve good classification accuracy, denoising and feature extraction of EEG signals is a significant step. We saw a considerable increase in all the machine learning models when the moving average filter was applied to the raw EEG data. Feature extraction techniques like a fast fourier transform (FFT) and continuous wave transform (CWT) were used in this study; three types of features were extracted, i.e., FFT Features, CWT Coefficients and CWT scalogram images. We trained and compared different machine learning (ML) models like logistic regression, random forest, k-nearest neighbors (KNN), light gradient boosting machine (GBM) and XG boost on FFT and CWT features and deep learning (DL) models like VGG-16, DenseNet201 and ResNet50 trained on CWT scalogram images. XG Boost with FFT features gave the maximum accuracy of 88%.
Amyotrophic lateral sclerosis (ALS) is a growing disease related to the nervous system that attacks nerve cells in the brain and disturbs muscle movement control. It is one of the rapidly spreading diseases as the symptoms of this disease get worse over time. Currently, medical science has no efficient treatment for this disease. Thus, it is highly desirable to detect it at an early stage. BCI offers patients with ALS and other neurological disorders to control prosthetic hands, wheelchairs, etc. [
BCIs measure brain activity using different techniques, analyze it, extract essential features and convert those features into commands that can control output devices like prosthetic hands, wheelchairs, IoT devices, etc. [
This study used a publicly available EEG dataset with events like hand motions and compared different ML models like logistic regression, random forest, KNN, light GBM and XG boost on FFT and CWT feature extraction methods. In addition, some deep learning (DL) approaches like VGG-16, DenseNet201 and ResNet50 are also used here. This study used different metrics like precision, recall, F1-score, support and accuracy to compare these ML and DL models. To progress this research, it is decided to use publicly available data from Kaggle, which have events like hand movements. This dataset was used in mind that the methods and machine learning algorithms we will be using can later be used for the wheelchair control dataset. The dataset used in this study was already epoched and pre-processed; we applied moving average filter as a processing technique and feature extraction methods like FFT and CWT. The main aim of this paper is to create brain-controlled interfaces for patients who have ALS. With the help of EEG headsets, patients will control IoT devices, wheelchairs, etc.
Alam et al. [
İşcan and Nikulin in 2018 used SSVEP-based BCI parallelly during the conversations as some subjects’ perturbations resulted even in better performance. For example, the decision tree gave excellent results (>95%) when compared with K-NN and naïve Bayes algorithms [
Al-Fahoum et al. [
Narayan et al. [
Lazarou et al. [
Chaurasiya et al. [
Zhang et al. [
Bilucaglia et al. [
This study uses different feature extraction methods and machine learning models to predict the probability of fist motion on EEG records. The dataset consists of already epoched EEG data for 19 electrodes which were then processed using moving average for noise removal; different feature extraction methods are applied like FFT [
The dataset consists of already epoched EEG data for 19 electrodes i.e., ‘C3’, ‘Cz’, ‘C4’, ‘Fp1’, ‘Fp2’, ‘F7’, ‘F3’, ‘Fz’, ‘F4’, ‘F8’, ‘T7’, ‘T8’, ‘P7’, ‘P3’, ‘Pz’, ‘P4’, ‘P8’, ‘O1’, ‘O2’. There are 180 epochs with three conditions i.e., resting state (1) and hand movements (2 and 3). This headset follows 10–20 system of electrode placement method which basically describes the location of scalp electrodes as shown in the
EEG data used in this research have opted from BCI EEG data analysis (NEUROML2020 class competition) [
Dataset snapshot of BCI EEG epoched signals is shown in
This data was already pre-processed, though it was not mentioned in the EEG data description. EEG raw data pre-processing includes removal of DC component and it is usually done before epoching. Specifically, we removed the DC component by calculating the mean and subtracting it from the EEG readings (data-mean), which gave us a negligible mean, implying that the data was already pre-processed.
Here, BCI EEG data was visualized for each electrode to differentiate between the conditions. The mean of all epochs for condition one was plotted for all the 19 electrodes during the epoch period of 80 s, similarly grouped for second and third conditions, representing hand motions as shown in
The moving average is one of the most common approaches used to capture significant trends in time series data. In addition, a finite impulse response (FIR) filter is used for a set of time-series data points by comparing different subsets of time-series data sets.
EEG brain signals can also be seen as a time series; therefore, moving average is used to remove the EEG signals’ artifacts and noise, as shown in
Frequency domain features were extracted using FFT, a widespread feature extraction method [
The considered dataset consists of EEG signals in the time domain and any time-dependent signal can be subdivided into a collection of sinusoids that can represent a single frequency. FFT converts the signal from the time domain to the frequency domain. Therefore, we can extract all the frequencies (sin waves) from which the signal is composed (ex: after performing the FFT, the raw EEG signal gave the frequencies: 2 Hz, 2.3 Hz, 13 Hz and 20 Hz). There are several types of brain waves, as shown in
After applying FFT on raw EEG, we can use frequencies to say which waves dominate a specific event. We compared the bands for both the conditions (i.e., steady-state and hand motions).
As shown in
Using the FFT feature extraction method, 180 epochs data were obtained; we calculated delta, theta, alpha and beta EEG band values for each channel. We took the standard deviation for each channel as an expected value. This makes a total of 19x4x2 = 152 Features, as shown in
CWT feature extraction method applies inner products to estimate the pattern match between morlet wavelet (ψ) and EEG signal. CWT analyzes the EEG signal to stretched and shifted versions of compressed morlet wavelets. For a scale parameter, a > 0 and position, b, the CWT given by
In this study, we trained our machine learning model in this CWT Coefficients. As a result, we obtained 128 Features from CWT coefficients for each epoch, as shown in
To apply CNN [
This study applied 5 different classification models on FFT features and CWT features, this includes logistic regression [
Light GBM trained on FFT features and CWT features have the following parameters: objective = binary, tree learner = data, number of leaves = 99, learning rate = 0.1, bagging fraction = 0.8, bagging freq = 1, feature fraction = 0.8, boosting type = gbdt and metric = binary logloss.
The random forest model, trained on FFT Features, has criterion=entropy, min samples leaf = 5, min samples split = 2 and several estimators = 700 and for CWT features, it is criterion=gini, min samples leaf = 5, min samples split = 2 and number of estimators = 400.
XG Boost, which is trained on FFT features, has an objective of binary logistic with the number of estimators = 10 and the model trained on CWT features has the same objective but with the number of estimators = 20.
KNN model have the following parameters neighbour = 2, leaf_size = 30, metric=‘minkowski’, p = 2, weights=‘uniform’ for FFT features and CWT features.
For classifying CWT scalogram images, pre-trained models like VGGnet-16, DenseNET201 and ResNet-50 were used with weights of the ImageNet. The model inputs a 32x81x3 input, where 32 are CWT features extracted and 81-time points. This image was provided to pre-trained models and subsequently passed through dense layers of 512 nodes for VGGnet and Resnet-50 and two 512 nodes layers in case of DenseNet201, Dropout of 0.5 was applied to avoid overfitting of data.
Model name | Total parameters | Trainable parameters | Non-trainable parameters | Epochs |
---|---|---|---|---|
DenseNET-201 | 20,551,745 | 2,229,761 | 18,321,984 | 200 |
VGGnet-16 | 15,240,001 | 525,313 | 14,714,688 | 100 |
ResNet-50 | 26,997,121 | 3,409,409 | 23,587,712 | 100 |
This study’s overall objective is to develop a robust and accurate workflow to predict hand motion and rest state. Our study presented the processing of EEG signals using the moving average method, two feature extraction techniques, i.e., FFT feature extraction and CWT feature extraction. Different machine learning models like the random forest, logistic regression, KNN, Light GBM and XG Boost for FFT features and CWT coefficients and VGG-16, DenseNet201 and ResNet-50 for CWT scalogram images.
Model | Class | Precision | Recall | F1-score | Support | Accuracy |
---|---|---|---|---|---|---|
Logistic regression | Steady state | 80% | 77% | 83% | 6 | 83% |
Hand motion | 85% | 92% | 88% | 12 | ||
Macro average | 82% | 79% | 80% | 18 | ||
Weighted average | 83% | 83% | 83% | 18 | ||
Random forest | Steady state | 50% | 67% | 57% | 6 | 67% |
Hand motion | 80% | 67% | 73% | 12 | ||
Macro average | 65% | 67% | 65% | 18 | ||
Weighted average | 70% | 67% | 68% | 18 | ||
KNN | Steady State | 71% | 73% | 83% | 84 | 78% |
Hand motion | 74% | 70% | 71% | 78 | ||
Macro average | 85% | 78% | 77% | 162 | ||
Weighted average | 85% | 78% | 77% | 162 | ||
Light GBM | Steady state | 70% | 76% | 73% | 9 | 70% |
Hand motion | 70% | 64% | 67% | 9 | ||
Macro average | 70% | 70% | 70% | 18 | ||
Weighted average | 70% | 70% | 70% | 18 | ||
XG Boost | Steady state | 88% | 89% | 88% | 84 | 88% |
Hand motion | 89% | 87% | 88% | 78 | ||
Macro average | 88% | 88% | 88% | 162 | ||
Weighted average | 88% | 88% | 88% | 162 |
Model | Class | Precision | Recall | F1-score | Support | Accuracy |
---|---|---|---|---|---|---|
Logistic regression | Steady state | 60% | 73% | 66% | 84 | 61.05% |
Hand motion | 62% | 49% | 55% | 78 | ||
Macro average | 61% | 61% | 60% | 162 | ||
Weighted average | 61% | 61% | 61% | 162 | ||
Random forest | Steady state | 56% | 83% | 67% | 6 | 72% |
Hand motion | 89% | 67% | 76% | 12 | ||
Macro average | 72% | 75% | 71% | 18 | ||
Weighted average | 78% | 72% | 73% | 18 | ||
KNN | Steady state | 67% | 100% | 80% | 84 | 74.85% |
Hand motion | 100% | 46% | 63% | 78 | ||
Macro average | 83% | 73% | 72% | 162 | ||
Weighted average | 83% | 74% | 72% | 162 | ||
Light GBM | Steady state | 60% | 67% | 63% | 9 | 61.12% |
Hand motion | 62% | 56% | 59% | 9 | ||
Macro average | 61% | 61% | 61% | 18 | ||
Weighted average | 61% | 61% | 61% | 18 | ||
XG boost | Steady state | 65% | 100% | 79% | 81 | 73.34% |
Hand motion | 100% | 45% | 63% | 81 | ||
Macro average | 82% | 73% | 71% | 162 | ||
Weighted average | 82% | 73% | 71% | 162 |
Model | Class | Precision | Recall | F1-score | Support | Accuracy |
---|---|---|---|---|---|---|
VGG-16 | Steady state | 76% | 77% | 76% | 338 | 76% |
Hand motion | 77% | 76% | 76% | 346 | ||
Macro average | 76% | 76% | 76% | 684 | ||
Weighted average | 76% | 76% | 76% | 684 | ||
DenseNet201 | Steady state | 85% | 75% | 82% | 338 | 84% |
Hand motion | 80% | 88% | 86% | 346 | ||
Macro average | 84% | 84% | 84% | 684 | ||
Weighted average | 84% | 84% | 84% | 684 | ||
ResNet-50 | Steady state | 84% | 84% | 84% | 338 | 85% |
Hand motion | 85% | 85% | 85% | 346 | ||
Macro average | 85% | 85% | 85% | 684 | ||
Weighted average | 85% | 85% | 85% | 684 |
In this study, different feature extraction methods and ML models have been used to predict the probability of fist motion on EEG records. Various feature extraction methods are applied, like FFT and CWT, on the dataset. On these features, different machine learning models were trained and compared. XG Boost and logistic regression models performed well in FFT features and achieved 88% and 83% accuracy, while XG Boost and KNN performed equally for CWT features with 74.85% and 73.34%. For CWT scalogram images, ResNet50 performance is better than the VGG-16 as it gave an accuracy of 85%. This study shows that XG Boost trained on FFT Feature Extraction with Moving Average Filter as the signal processing technique gave the highest accuracy for the dataset of about 88%. To create BCI for ALS patients, we need a large EEG dataset. This dataset can also be created on our own using EEG headsets. It can be further extended towards brain-controlled wheelchairs for patients who have ALS or other BCI applications. Once brain commands have been detected, it will direct it towards the IoT devices such as prosthetic hands and this is how the patient will be able to use BCI.
The authors extend their appreciation to the Deanship of Scientific Research at King Saud University for funding this research.