Hand Gestures Recognition Based on One-Channel Surface EMG Signal

Junyi Cao^1*, Zhongming Tian^2*, Zhengtao Wang^3*

¹Xi’an Jiao Tong University, Xi’an, China.
²Chongqing University, Chongqing, China.
³Sun Yat-sen University, Guangzhou, China.

DOI: 10.4236/jsea.2019.129023                

Abstract

This paper presents an experiment using OPENBCI to collect data of two hand gestures and decoding the signal to distinguish gestures. The signal was extracted with three electrodes on the subject’s forearm and transferred in one channel. After utilizing a Butterworth bandpass filter, we chose a novel way to detect gesture action segment. Instead of using moving average algorithm, which is based on the calculation of energy, We developed an algorithm based on the Hilbert transform to find a dynamic threshold and identified the action segment. Four features have been extracted from each activity section, generating feature vectors for classification. During the process of classification, we made a comparison between K-nearest-neighbors (KNN) and support vector machine (SVM), based on a relatively small amount of samples. Most common experiments are based on a large quantity of data to pursue a highly fitted model. But there are certain circumstances where we cannot obtain enough training data, so it makes the exploration of best method to do classification under small sample data imperative. Though KNN is known for its simplicity and practicability, it is a relatively time-consuming method. On the other hand, SVM has a better performance in terms of time requirement and recognition accuracy, due to its application of different Risk Minimization Principle. Experimental results show an average recognition rate for the SVM algorithm that is 1.25% higher than for KNN while SVM is 2.031 s shorter than that KNN.

Keywords

Electromyography (EMG), Gesture Recognition, Hilbert Transform, K-Nearest-Neighbors (KNN), Support Vector Machine (SVM)

1. Introduction

Hand gestures are involved in many aspects of human life, including games, medical care, education and shopping. Hand gesture recognition refers to the process that a computer automatically detects and analyzes the measured bioelectrical signals in order to identify hand gestures of an individual. It enjoys great popularity worldwide and has been studied extensively. So far a new method for hand gesture recognition with Electromyography (EMG) has been put forward considering its convenience and financial benefits.

EMG is the superposition of the action potentials of the muscle tissue occurring during a voluntary contraction, providing information on the flexion and extension of the muscles as well as the shape and position of the limbs during the completion of the movement [1] . The hand gesture recognition based on EMG has several advantages compared with optical detection. The design of the EMG sensor is relatively simple while the sensor is not sensitive to the environment. In addition, the signal processing for EMG is not so complicated. However, the problem of using EMG to recognize hand gestures is the inaccuracy, so solving the problem is the motivation of the paper.

This paper focuses on the experiment where we plan to use EMG signals to recognize 2 hand gestures. And based on that, we are going to improve the accuracy of recognition by optimizing the algorithm. We plan to use the surface electromyography sensor (OpenBCI Ganglion) to capture muscle activity information on the skin surface of the corresponding muscle group. We are going to perform experiments to find the best placement of the electrodes and get the most reliable data. MATLAB is utilized for analyzing and processing the signals and for distinguishing the EMG generated by different gestures. The accuracy will be evaluated by performing additional tests.

Starting from the methods adopted during the whole process of the research including signal processing and classification, the rest of the paper explains the experiment in detail, analyzes and discusses the results and draws conclusions based on the findings. Several novel ideas are to be demonstrated in the following part as well.

2. Methodology

2.1. Data Acquisition

Only one channel was chosen to collect data from the OpenBCI device, and three electrodes were placed on the right arm of the subject. Two of them were attached to 1+/1− pin and the other one was attached to bottom pin, working as a common ground. During the process, we found that the results varied according to the placement of electrodes. The EMG is produced by the electric potential change, which is generated by muscle cells. As a result, gestures produced by different parts of muscle will generate distinguished potentials. The optimal position of the electrodes is determined by performing several trials of the acquisition experiment (shown in Figure 1). In addition, we found out the testing environment is another parameter that needs to be controlled to get a high-quality raw data.

2.2. sEMG Signal Processing

The recognition of the gesture SEMG signals consists of three parts: data pre-processing, feature extraction and classification. For data pre-processing, a Butterworth filter and a segmentation algorithm are used for getting a cleaner data and for reducing the data amount. The feature extraction algorithm compresses the sEMG signal segments into feature vectors. The features are designed so that they emphasize the gesture class specific characteristics of the sEMG signal. The classifier is trained with the feature vectors to distinguish the different gestures from each other with high accuracy.

1) Data Pre-Processing

a) Filtering

We use Butterworth filter to process the original signal. Butterworth filter is a signal processing filter with flat frequency response curve in passband (shown in Figure 2) [2] . In this experiment, the passband was chosen from 20 Hz to 90 Hz, with a 3 dB passband ripple and 40 dB stopband attenuation.

As we can see in the amplitude-frequency using FFT (see in Figure 3 and Figure 4), the low frequency, which is of no relation to the EMG signal, has been filtered.

Figure 1. The placement of the four channel-surface EMG electrodes is shown in the on the bottom. The gestures used for detection are shown on the top.

Figure 2. The amplitude-frequency figure of a low-pass Butterworth filter.

Figure 3. The amplitude-frequency figure of the original signal.

Figure 4. The amplitude-frequency figure of the filtered signal.

b) Detection of Gesture Action Segment

In the beginning, we planned to use the sliding window method based on the calculation of energy in order to identify the start and the end of the segments.

We have:

$Q_i={\displaystyle {\int }_{t_i-\Delta t}^{t_i+\Delta t}x{\left(t\right)}^2\text{d}t}$ (1)

where $\Delta t$ is half of the window length, $x\left(t\right)$ refers to the amplitude of the EMG signal at time t, and $Q_i$ represents the energy value of signal at time $t_i$ . From the moment, if a series of $Q_k\left(k=i,i+1,i+2,\cdots \right)$ is consistently larger than A (a certain threshold value), in that case, the $t_i$ moment is considered to be the start of an action segment. Using the same method the end of the action can be recognized.

However, we found that it was difficult to determine the value of the threshold energy for each action segmentsince the signals of different actions vary widely. A specific threshold could not be found to recognize all pulses. Therefore, we utilize algorithms based on the toolbox function Hilbert in MATLAB to find a dynamic threshold in order to solve this problem.

The Hilbert transform is a specific linear operator that is given by the convolution with the function $\frac{1}{\pi t}$ (see in (2)). It is related to the actual data by a 90-degree phase shift.

$H\left(u\right)\left(t\right)=\frac{1}{\pi }{\displaystyle {\int }_{-\infty }^{\infty }\frac{u\left(\tau \right)}{t-\tau }\text{d}\tau }$ (2) [3]

The toolbox function hilbert in MATLAB computes the Hilbert transform for a real input sequence x and returns an analytical signal of the same length, y = hilbert(x), where the real part of y is the original real data and the imaginary part is the actual Hilbert transform [4] . The magnitude of the analytical signal is the complex envelope of the original signal.

However, after figuring out the dynamic threshold and the beginning and end of the actions, we discovered that due to the instability of some signals, one action can be recognized as two because of some sudden attenuation of the signal (see in Figure 5). Therefore, we have made some improvements for the envelope extraction with an algorithm of “dilation” and “erosion” (see in Figure 6).

Figure 5. The mistake in recognition when detecting the action segment

Figure 6. The detection of activity after the improvement with an algorithm of “dilation” and “erosion”.

2) Feature Extraction

Feature extraction is carried out on the SEMG signals to preserve the signal patterns in order to distinguish different gestures. In our experiment, the Mean Absolute Values (MAV), the Root Mean Square (RMS), the variance and the Auto-Regressive (AR) model coefficients are used to the modeling of different SEMG signals and then input to the network for classification. A short description of the calculation is given below.

Mean Absolute Value (MAV) is defined as:

$\text{MAV}=\frac{1}{N}{\displaystyle \underset{n=1}{\overset{N}{\sum }}\left|x_n\right|}$ (3)

where N denotes the length of the signal and $x_n$ represents the EMG signal in a segment [5] .

Variance: the general equation used to find variance by:

$\text{Var}=\sqrt{\frac{1}{N}{\displaystyle \underset{n=1}{\overset{N}{\sum }}{\left(x_n-\bar{x}\right)}^2}}$ (4)

where $\bar{x}$ is the mean value of the EMG signal, N denotes the length of the signal and $x_n$ represents the EMG signal in a segment.

Root Mean Square (RMS) can be expressed as:

$\text{RMS}=\sqrt{\frac{1}{N}{\displaystyle \underset{n+1}{\overset{N}{\sum }}{x}_n^2}}$ (5)

where N denotes the length of the signal and $x_n$ represents the EMG signal in a segment.

Auto Regressive (AR) model coefficients: the AR model of a signal can be expressed as:

$x\left(n\right)=-{\displaystyle \underset{k=1}{\overset{p}{\sum }}{a}_{k}x\left(n-k\right)+u\left(n\right)}$ (6)

where $a_k$ refers to the coefficient of the kth order of the signal, p refers to the order of the model, and $u\left(n\right)$ is the white noise.

In this experiment, we use the burg method to fit the AR model and get the coefficients of four orders as a feature of the signal.

3) Classification

In sEMG pattern recognition, different algorithms have been used to assign the feature vectors. Neural Network group of classifiers are widely used due to its expandability and ability of bearing both simple complex cases. However, the choice of features and the time-constraints make this classifier excessively complex. Others preferred to utilize the Fuzzy logic approach, which allows to insert user experience in the system, and to contradict itself for patterns changing [6] . In our experiment, after extracting a 7-order feature vector of the static hand gesture, we selected k-nearest neighbors (KNN) algorithm at first, because of its simplicity and practicability. The principle of KNN is classifying objects by a plurality vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors. However, the classification of objects usually requires a large amount of computation, because we need to calculate the distance between every testing point and the rest to get its k nearest neighbors. For that reason a Support Vector Machine (SVM) was chosen, which provides high accuracy in calibration and classification. SVM is a discriminative classifier formally defined by a separating hyperplane. It is widely used in binary classification, since it benefits from structural risk minimization principle and avoidance of overfitting. When there are limited training data available, SVM usually outperforms the traditional parameter estimation methods. It effectively avoids outfitting problem, especially when there is only small amount of data samples [7] .

The objective of SVM algorithm is to find a decision boundary in an N-dimensional space that distinctly classifies that data points (Figure 7). Maximizing the margin distance between two hyperplanes provides some reinforcement so that future data points can be classified with more confidence. So, the ultimate goal is to find a plane that has the maximum margin.

Suppose there are two gestures GTR1, GTR2 needed to be classified. We denote the training set with samples as:

$\left\{\left(x_i,g_i\right)\right\},i=1,\cdots ,n$ (7)

where $x_i\in R^d$ represents a feature vector of a gesture:

$g_i=\{\begin{array}{l}+1,\text{if}{x}_i\text{belongstoGTR1}\left(w{x}_i-b\ge 1\right)\\ -1,\text{if}{x}_i\text{belongstoGTR2}\left(w{x}_i-b\le -1\right)\end{array}$ (8)

A separating plane can be written as:

$w\cdot x-b=0$ (9)

Then the only problem is to find the maximum margin (find the minimum w), which can be solved by kernel function [8] .

The first hundred repetitions of each gesture were used as training samples to train the classifier and the rest of the gesture samples were used for testing. Recognition results for the two subjects are given in Table 1 and Table 2. In the first column, the figures in parenthesis are the total numbers of test samples and figures on their left are the numbers of correct recognitions. Accuracy is defined as

Table 1. Results of using SVM algorithm in classification.

Table 2. Results of using KNN algorithm in classification.

Figure 7. Illustration of SVM algorithm. Solid line is the hyperplane to distinguish two groups.

$\text{ACC}=\frac{T_p}{T_p+F_p}$

where was the number of correct recognitions, was the number of wrong recognitions. Time refers to the total time used to train the model.

Through the chart, we can see by using SVM the average recognition rate higher than using KNN algorithm. Also, the time consumed to train the model in SVM is short than KNN. Our result showed that, based on a relatively [5] small data amount, SVM is the better choice for classification algorithm.

3. Experiment

1) Experiment Subject

One of our group members vonlunteers to be a subject in the experiment. He is 20-year-old healthy male, height of 188 cm and weight of 90 kg. He has no history of neuromuscular or joint disease and he is right-handed.

2) Experiment Device

OpenBCI is an open-source brain-computer interface platform. OpenBCI boards have 4 channels and can be used to measure and record electrical activity produced by the brain (EEG), muscles (EMG), and heart (EKG).

3) Task

The subject sat comfortably on a chair and was relaxed. Then, he executed 2 hand gestures and each gesture was repeated more than 60 times. Every gesture action lasted about 1 second with an interval of 1 to 2 seconds.

4. Conclusions

We have carried out this experiment in order to explore plausible ways to identify hand gestures with sEMG signals and to compare different algorithms for classification. After using one channel from the OpenBCI board to collect sEMG signals of two different hand gestures, we processed the signal and extracted four features to generate feature vectors for classification. Then we classified the signals with two different algorithms, KNN and SVM. The overall result of gesture recognition has achieved our expectations, reaching an accuracy of 97.5% for one gesture and 100% for another with a testing sample of 40 for each gesture. Meanwhile, we compare the two algorithms we have selected. From the result, it can be concluded that in such case SVN achieves a higher accuracy for recognition than KNN, along with a shorter time.

However, our experiment still needs improving in some aspects. To begin with, since we did not have a deep understanding of the classification methods using KNN and SVM, we have not optimized the algorithm by adjusting the parameters. It can be further improved in the future by optimizing the two algorithms.

In addition, we did not design the hardware. The whole device is not wearable and is inconvenient to move around, which is a limitation for people in need of help.

On all accounts, recognition of hand gesture based on SEMG signals is, though complicated and challenging, an efficient way to identify gestures and help people in need. Further study should be carried out including the optimization of the algorithm for classifying by adjusting the parameters and the development of hardware.

Acknowledgements

We would like to thank Professor Jan Van der Spiegel and our teaching assistant Susan Chen for their guidance with our sincere gratitude. Also, many thanks are expressed to Yubin Qin and his group for their kind help during the experiment.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Zhang, X.U. (2010) Body Gesture Recognition and Interaction Based on Surface Electromyogram. Thesis, University of Science and Technology of China, Hefei.
[2]	https://en.wikipedia.org/wiki/Butterworth_filter
[3]	https://en.wikipedia.org/wiki/Hilbert_transform
[4]	https://ww2.mathworks.cn/help/signal/ug/hilbert-transform.html?requestedDomain=cn
[5]	Ahsan, M.R., Ibrahimy, M.I. and Khalifa, O.O. (2011) Electromygraphy (EMG) Signal Based Hand Gesture Recognition Using Artificial Neural Network (ANN). 2011 4th International Conference on Mechatronics (ICOM), Kuala Lumpur, 17-19 May 2011, 1-6. https://doi.org/10.1109/ICOM.2011.5937135
[6]	Mahmoud, T., Carlo, B., Pedro, A.L., Bica, O.L. and De, A.A.T. (2018) Robust Hand Gesture Recognition with a Double Channel Surface EMG Wearable Armband and SVM Classifier. Biomedical Signal Processing and Control, 46, 121-130. https://doi.org/10.1016/j.bspc.2018.07.010
[7]	Chen, X., Zhang, X., Zhao, Z., Yang, J. and Wang, K. (2007) Multiple Hand Gesture Recognition Based on Surface EMG Signal. 2007 1st International Conference on Bioinformatics and Biomedical Engineering, Wuhan, 6-8 July 2007, 506-509. https://doi.org/10.1109/ICBBE.2007.133
[8]	Wu, J., Pan, G., Zhang, D., Qi, G. and Li, S. (2009) Gesture Recognition with a 3-D Accelerometer. In: Zhang, D., Portmann, M., Tan, A.H. and Indulska, J., Eds., Ubiquitous Intelligence and Computing. UIC 2009: Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 25-38. https://doi.org/10.1007/978-3-642-02830-4_4