Mental State Detection in Classroom Based on EEG Brain Signals

Abstract

The goal of this work is to identify human brain waves in different states non-invasively, and to distinguish them into different levels of mental states in order to provide immediate mental state feedback to a classroom instructor and maximize learning outcomes. In order to apply such knowledge, this project utilizes a commercial NeuroSky Mindwave Mobile EEG to collect brain signals, MATLAB to filter data, voltage thresholds to detect blinks, which are used in tandem with power spectral density (PSD) analysis in order to classify mental states. This knowledge can then be provided to a class instructor who can use it to maximize the learning experience for students.

Share and Cite:

Gong, Y. and Xu, S. (2019) Mental State Detection in Classroom Based on EEG Brain Signals. Natural Science, 11, 315-322. doi: 10.4236/ns.2019.1111034.

1. Introduction

Electroencephalography (EEG) is a method of recording brain activity from electrophysiological indicators. These indicators are evident from the measurable postsynaptic potentials generated by neurons firing. An EEG device will then record these changes in the electrical waves generated by brain activity at the cerebral cortex or scalp surface. Through observation, it quickly becomes evident that different physiological states and facial movements will generate signals with distinct electrical attributes.

Owing to its non-invasive nature, EEG detection has been widely used in many fields, such as neurophysiology, psychology, pathophysiology, cognitive neuroscience, neuroengineering, and even social psychology, etc. [1]. A further logical application of this technology is in identifying the mental state of its users, which has significant potential to guide interactions through a variety of applications, whether it is from consultation, the classroom, or for monitoring worker productivity [1]. With these potential applications, it naturally follows that there have been numerous previous attempts to apply observed electrophysiological phenomena readable by the brain. For example, an early attempt in early 2010 with researchers affiliated with the National Chiao Tung University in Taiwan, which monitored a test subject’s physiological state by EEG. [2] In response to drowsiness or alertness, the temperature of a miniature room’s air conditioning and the light intensity would automatically adjust. A further application that was explored in 2016 was in the use of an EEG to assess stress levels in an individual, funded by the Ministry of Education in Malaysia [3]. By synthesizing the readout information from EEG and functional near-infrared spectroscopy (fNIRS), the researchers classified brain activities with a support vector machine (SVM) in order to attempt to improve upon previous methods of stress detection through non-invasive brain machine interfaces. In addition, multiple studies have investigated the relationship between certain mental states and what is interpretable from an EEG. One example includes an IEEE published paper for a 2017 conference in Tiruneveli, India, that not only reports on existing databases on emotional detection and EEGs, but also on methods of processing signals, and analyzing them.

Therefore, it follows that a classroom environment is a use case for applying mental state detection as a supplementary aid to enhance the learning experience. As EEG technology has advanced, there now exist EEGs that do not require a conductive gel to function, and unit cost continues to drop [4]. For example, a class of students equipped with EEG headsets could be monitored by detecting, processing, and interpreting a postsynaptic potentials and frequencies to inform the instructor of the mental state of each individual student. This immediate feedback will also allow the instructor to understand the level of engagement students having with the lecture, and respond with a suitable course of action up to his or her discretion to improve student engagement.

2. MATERIALS

The tools utilized in this work include a NeuroSky Mindwave Mobile (Figure 1), a commercial EEG headset, and MATLAB (Matrix Laboratory) for reading and processing postsynaptic potential. The headset features a single electrode sensor contacting the forehead, and does not require conduction gel or paste to function [4]. In addition, the headset includes a clip-on sensor to attach onto the earlobe. It represents an example of a reasonably accessible product that is comparatively less complex to configure for a test subject than earlier EEG designs. These are desirable traits for a device that is intended to be worn in the classroom for the sake of convenience, and hence, it is preferable for data recorded in experiments to reflect the capabilities of technology accessible to an educational institution.

In addition, a MATLAB script, “readRAW_dft.m” was utilized for allowing a Bluetooth connected device with an operating system to interpret electrophysiological signals from the test subject. This was utilized, along with other MATLAB plotting tools in order to collect data. Some modifications and additional features were implemented due to some of the initial shortcomings of the script, which will be described in further detail in the next section.

Figure 1. The NeuroSky Mindwave Mobile 2 EEG unit utilized for this project.

3. METHODS

In order to optimize the data gathered from the output of the NeuroSky EEG headset, several processing steps were necessary (see Figure 2 for steps necessary for blink-detection) Though blinks and the level of concentration of a test subject produce distinct electrophysiological signals, the testing environment could not have been perfectly controlled for signal noise. For example, the lights in the experimental area were powered by 220 V AC at 50 Hz, which overlaps with the upper range of the human brain’s frequency range when the test subject is concentrating [1]. This source and similar sources of noise needed to be eliminated from the raw signal read by the EEG.

To this end, filtering was done on the signal for the method of eyeblink detection in this project. Each trial lasted for 20 seconds, with a sampling rate of 512 per second, during which the EEG headset measures the postsynaptic potentials at its sensor region as a function of time. The units are microvolts and seconds respectively, though a raw signal waveform contains numerous artifacts of external noise (see Figure 3).

This signal is first post-processed with a band-pass filter function built into MATLAB [5]. The band-pass frequencies have been set between 4 Hz and 30 Hz. The range below 4 Hz was excluded as it denotes an already unconscious individual [1], and the upper range was made large enough to include some states of mental concentration [1] while excluding noise from the surrounding environments, such as the 50 Hz AC lighting. It is additionally filtered through a convolution process: ideal for retaining the higher-level features of the signal while eliminating remaining irregularities in the data (see Figure 4). The combination of these functions is also overall quicker to process than MATLAB’s built-in “filter” function.

This signal is then interpreted by a blink-detection mechanism, which functions based on voltage thresholds. The set value for the threshold is 250 microvolts, which was qualitatively chosen from initial trials to avoid false positives while still detecting most blinks. This blink recognition method is also used for detecting two mental states out of the four tested for: a distracted and tired state. By detecting the number of times the test subject has blinked within 20 seconds, a total of seven blinks or greater will register as “tired”, and a total of two or fewer blinks will register as “distracted”.

The timespan measured in each trial is 20 seconds, and the sampling rate is set to 512 times/second. As a result, each data sample will have 20 × 512 = 10,240 sample points, corresponding to each different time. This will generate a row vector of a magnitude up to 10,240, which results in a data matrix for analysis.

The method of analyzing the voltage amplitude in the time domain relies on an inequality function for detecting a voltage spike in excess of the voltage threshold. First, upon inspecting the matrix generated by the filtered signal, and finding the corresponding value for a blink of an eye, it is marked (Figure 5). This detection mechanism identifies a blink by finding an individual voltage peak each time the threshold is crossed: a data point which is greater in voltage than the previous point and the following point.

In order to be able to utilize an EEG to determine the mental state of an individual, experimentation was conducted to identify electrophysiological indicators. The majority of the trials conducted follow a

Figure 2. Eyeblink detection mechanism flowchart.

Figure 3. A raw EEG signal from a test subject: note the high frequencies. The larger pulses indicate eyeblinks.

Figure 4. Filtered signal, note the absence of relatively higher frequencies.

Figure 5. Eyeblink detection mechanism at work (red triangle marks).

structure of analyzing frequency in relation to some other variable. For example, an analysis of power spectral density (PSD) in relation to the frequency of postsynaptic potential oscillations. It is not a new concept to analyze PSD in the context of analyzing mental states with an EEG [4], and the insights gleaned can often differentiate between the types of waves in ascending order of frequency: delta, theta, alpha, and beta waves (Table 1). Delta waves are most prominent when the brain is unconscious, theta waves are generally associated with a drowsy state, alpha waves with a relaxed but wakeful state, and beta waves with a normal state up to being relatively alert. In general, the lower ranges of these frequencies are associated with a higher PSD [6].

As the default graphical output of the NeuroSky EEG headset related voltage over time, a process had to be implemented for converting such a graph to power spectral density over frequency. In order to perform this function, the mathematical mechanism utilizes a fast Fourier transformation (FFT) to convert the domain of the initial graph from time in seconds to frequency in hertz, the reciprocal unit (4 to 25 Hz, with 3 to 25 μV/sqrt (Hz) as the new y-axis). The FFT was favored over a discrete Fourier transformation for processing speed, which is highly relevant to this classroom application [7].

P ( f ) = lim T 1 T | S T ( f ) | 2

The FFT decomposes the original N-point sequence into a series of short sequences and makes full use of the symmetry and periodicity of the exponential factors in the FFT formula. The corresponding FFTs of these short sequences are then appropriately combined with the corresponding FFTs of these short sequences to achieve the purpose of deduplication calculations, reducing the number of multiplication operations and thereby simplifying the structure.

Then, due to the conjugate law of the imaginary portion of the function, the only useful information for the final result is stored in the subscript “1 to 1 + N/2”. The negative phase frequency of the signal converted to the spectrum is eliminated and the above calculation is applied after this elimination. The power spectral density formula finds the normalized power for each frequency. Here, every 256 sample points are used for the above operation, that is, the power density spectrum is calculated every 0.5 seconds. After removing the sample point of the negative frequency, it is 256/2 + 1 = 129 available sample points, and one record will produce 10,240/256 = 40 line vectors, which will be stored in a matrix of 40 rows and 129 columns. Next, the matrix is summed by columns and divided by 40. The result is a row vector of length 129. The average calculation increases the stability of the data to reduce the interference generated by the outside world and the device itself.

Then, according to the actual application, the power sample points corresponding to the signal with the frequency of 4 - 25 Hz are extracted, and the data analysis table is imported. For classification, the mental states of “tired”, “normal”, and “vigorous”, are divided into detecting A (theta waves (4 - 8 Hz)), B (alpha waves to low beta waves (8 - 16 Hz)), and C (mid-to-high beta waves respectively (16 - 30 Hz)), in other words, splitting a row vector into three row vectors, the data in different frequency domain regions can be more conveniently compared. Then, all the power data for different frequencies in each area is superimposed (because they have the same dimensions), so that the power in different mental state areas can

Table 1. A table based on a study exploring the relationship between the presence of certain brain wave frequencies and mental states [1].

be expressed by a number, and they are represented by the letters A, B, and C. In order to determine the proportion of the frequencies in these regions to each other, a simple formula is used to calculate the ratio of each number and the remaining number sum: RATIO A = A/(B + C). The denominator excludes the value of the proportion of what is being calculated in particular so that each data point will be used only once in this calculation. The calculation shows the contrast of power in different mental states, and the overall process is described in Figure 6.

The above graphical representations (Figure 6) show the proportion of each of the three categories of frequencies relative to each other. Note the graph (Figure 7) with an especially high proportion of theta waves (type “A”), which denotes a tired state [6].

Figure 6. Graphical representation of the power spectral density at different mental states.

Figure 7. Summary of the PSD graph generation process.

4. DISCUSSION

In testing a commercially available EEG headset, it has become apparent that it can detect differentiable signals for varying mental states and motor movements, and this can be applied to detect concentration to a usable resolution in a classroom setting.

This is evident from corroboration with separate research efforts on electrophysiological analysis, which denote a range of around 8 Hz and below as an unfocused, drowsy state. Previous work has also attempted to utilize power spectral density as an analysis method, and despite disruption in the data from noise at relatively higher frequencies (50+ Hz), there is a clear spike at the lowest range of frequencies. These insights are sufficient for a basic classroom implementation of EEG headsets.

Some of the main limitations to the research conducted concern a small sample size: the number of trials conducted remains in the double digits. First of all, this has some errors compared to the larger data set nature. Furthermore, it imposed limitations on the type of analysis that could be carried out, excluding techniques such machine learning [8]. This limits the quality of the research conducted here, even if the results are usable on a basic level.

In order to conclude, the significance of this work must be reiterated on. It does not represent any notably new or groundbreaking developments, but verifies previous studies and is focused on assessing the viability of its specific application. If a relatively simplistic analysis of the brain’s electrophysiological outputs is capable of generating enough useful results to be of use in a classroom environment, the technical challenges of implementing EEG headsets into the classroom are not particularly imposing. Though the cost may be prohibitive for some public institutions at the moment, advancements will likely continue to progress in improving accessibility.

Acknowledgements

We would like to express our sincere appreciation to Professor Jan Van der Spiegel of the University of Pennsylvania for his valuable suggestions and guidance during this research program.

We would also like to thank the Cathaypath International Summer (CIS) program for funding and facilities, and it has been our pleasure to participate and engage in the program.

NOTES

*These authors are co-first authors, sorted by alphabetical order of last name.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Al-Shargie, F., Kiguchi, M., Badruddin, N., Dass, S.C., Hani, A.F.M. and Tang, T.B. (2016) Mental Stress Assessment Using Simultaneous Measurement of EEG and fNIRS. Biomedical Optics Express, 7, 3882.
https://doi.org/10.1364/BOE.7.003882
[2] Lin, C.T., Lin, F.C., Chen, S.A., Lu, S.W., Chen, T.C. and Ko, L.W. (2010) EEG-Based Brain-Computer Interface for Smart Living Environmental Auto-Adjustment. Journal of Medical and Biological Engineering, 30, 237-245.
https://doi.org/10.5405/jmbe.30.4.07
[3] Kimmatkar, N.V. and Babu, B.V. (2017) Initial Analysis of Brain EEG Signal for Mental State Detection of Human Being. In 2017 International Conference on Trends in Electronics and Informatics (ICEI), Tirunelveli, 11-12 May 2017, 287-295.
https://doi.org/10.1109/ICOEI.2017.8300934
[4] Bashivan, P., Rish, I. and Heisig, S. (2015) Mental State Recognition via Wearable EEG. Proceedings of 5th NIPS Workshop on Machine Learning and Interpretation in Neuroimaging (MLINI15) (2015) 5-1.
[5] “Bandpass.” MathWorks.
https://www.mathworks.com/help/signal/ref/bandpass.html
[6] Kumar Ahirwal, M. and Dlondhe, N. (2012) Power Spectrum Analysis of EEG Signals for Estimating Visual Attention. International Journal of Computer Applications, 42, 34-40.
https://doi.org/10.5120/5769-7993
[7] Weisstein, E.W. “Fast Fourier Transform.” From MathWorld—A Wolfram Web Resource.
http://mathworld.wolfram.com/FastFourierTransform.html
[8] Chollet, F. (2017) Deep Learning with Python. In Deep Learning with Python.

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.