EEG Features in Mental Tasks Recognition and Neurofeedbackimi.ntu.edu.sg/NewsEvents/Events/PastSeminars/Documents/Wang_… · EEG Signal Processing 9 ... Background & Motivation &

EEG Features in Mental Tasks Recognition and Neurofeedback

Ph.D. Candidate: Wang Qiang

Supervisor: Asst. Prof. Olga Sourina

Co-Supervisor: Assoc. Prof. Vladimir V. Kulish

Division of Information Engineering

School of Electrical and Electronic Engineering

Nanyang Technological University

Institute for Media Innovation

Nanyang Technological University

1

Outline

Background & Motivation & Objective

Demos & Publication List

Proposed Algorithms

2

Conclusion & Future Works

EEG: EEG provides wonderful tools for brain state monitoring.

– High temporal resolution.

– Tremendous algorithms are available for time series.

– Successful medical applications.

Neurofeedback: Neurofeedback systems provide visual/audio feedback according to

EEG signal. It is useful for brain training. – Use neurofeedback to enhance the work performance.

– Use neurofeedback to treat ADHD patients.

Motivation

3

This project is inter-disciplinary:

biosignals medical application serious game

pattern recognition cognitive informatics psychology

Research Objectives:

• Design an experiment protocol for mental tasks recognition.

• Study nonlinear model and propose effective EEG features for mental tasks recognition.

• Propose faster, more accurate algorithms with less EEG channels for mental tasks recognition.

• Propose neurofeedback strategies.

• Design and implement 2D and 3D neurofeedback games.

• Develop a protocol to use neurofeedback game for psychological disorder treatment and optimum concentration level searching.

• Use proposed concentration level recognition techniques to provide a feedback loop in e-learning system.

Research Objective

4

Outline



Proposed Algorithms

5


Relative power training in EEG based neurofeedback.

• Theta/Beta training1. Increase theta band power.

Decrease beta band power.

• Active alpha training2. Increase alpha band power.

Decrease EMG power.

1. T. M. Sokhadze, et al., "EEG biofeedback as a treatment for substance use disorders: Review,

rating of efficacy, and recommendations for further research," Applied Psychophysiology

Biofeedback, vol. 33, pp. 1-28, 2008.

2. S. Hanslmayr, et al., "Increasing individual upper alpha power by neurofeedback improves

cognitive performance in human subjects," Applied Psychophysiology Biofeedback, vol. 30, pp. 1-

10, 2005.

Related Works

6

A well-known EEG database for mental tasks classification

recorded by Zak Keirn1 is available.

• Seven subjects participated the experiment for two session.

• In each session, subjects performed 5 different mental tasks

for 5 trials.

Relax Counting Letter composition Multiplication Rotation

1. Z. Keirn,“ Alternative modes of communication between man and machine,” Master’s thesis,

Electrical Engineering Department, Purdue University,USA,1998.

EEG Database for Mental Tasks

7

N. Liang et. al.1 processed the mental tasks EEG database in 2006.

• Autoregressive features were used.

• Different classifiers were compared, multi-class SVM classifier

can achieve the best accuracy.

• With multi-class SVM classifier, 52.07% accuracy were reported

for multi-class classification.

1. N. Liang, P. Saratchandran, G. Huang, and N. Sundararajan, “Classification of mental tasks from eeg signals using

extreme learning machine,” International Journal of Neural Systems, vol. 16, no. 1, pp. 29–38, 2006.

Related Works

8

EEG data were processed according to the following procedure.

EEG Signal Processing

9

EEG Signal Segmentation

Ocular Artifact Removal

Feature Extraction

Feature Selection

Classification

EEG signals were divided into segments with 512 samples

(overlapping with 480 samples).

EEG Signal Segmentation

10

128

512 samples

Segment 1 Segment 2 ….

32

Ocular artifacts were detected by applying with a fixed-weight

leakage normalized stochastic least mean fourth algorithm1 on

EOG channel.

Segments contains OAs were discarded.

1. P.Celka, B.Boashash, and P.Colditz, “Preprocessing and time-frequency analysis of new born eeg

seizures,” IEEE Engineering in Medicine and Biology Magazine,vol.20, no.5, pp.30–39, 2001.

Ocular Artifact Removal

11

Six group of features were extracted from each clean segment.

Feature Extraction

12

Feature Type No. of feature Time cost (ms)

Relative Power (PSD) 5 1

Autoregressive (AR) coefficient 6 70

Higher Order Crossing (HOC) 16 240

Generalized Higuchi Fractal

Dimension Spectrum (GHFDS)

2 620

Entropy 10 1500

Statistical 6 1

Generalized fractal dimension spectrum.

Feature Extraction

13

To speed up multi-class svm evaluation, we applied feature

selection method before classification. Following features

selection schemes were considerate and compared.

Random Forests (RF) scheme could achieve the best performance.

Feature Selection

14

Multi-class SVMs were used as classifier.

RBF kernel was applied and C-gamma parameters were selected

with grid search procedure.

Classification

15

Mental tasks classification results when different features were

used.

Statistical features could achieve better accuracy than AR features which were

used in N. Zhang’s research1. In their paper, the accuracy is 52.07%.

Combine all features could enhance the performance.

1. N. Liang, P. Saratchandran, G. Huang, and N. Sundararajan, “Classification of mental tasks from eeg signals using

extreme learning machine,” International Journal of Neural Systems, vol. 16, no. 1, pp. 29–38, 2006.

Classification Result

16

Benefits of feature selection.


17

Benefits of feature selection.


18

Experiment Setup:

• EEG recording device

– 14-channels,

– Sampling frequency: 128 Hz,

– A/D resolution: 16-bit.

• PC for processing data

– CPU: Intel Core 2 Quad Q9400 (2.66 Hz * 4),

– RAM: DDR3 3.25 GB,

• EEG processing software

– EEG recording : Emotiv Testbench,

– EEG processing: Numpy.

• Subjects

– 10 subjects.

Arithmetic Task Experiment

19

Data Acquisition Protocol:

• Session 1

– Relaxation Session (Relax, no task to fulfill)

• Session 2

– Arithmetic Session (Working on 3-digit arithmetic problems)

Arithmetic Task Experiment

20

Comparison between different type of EEG features.


21

Comparison between different type of EEG features.


22

EEG channel rank.


23

EEG channel rank.


24

Outline



Proposed Algorithms

25


A well-known EEG database for mental tasks recognition was also used.

Arithmetic task experiment was also designed and carried out to collect the

labeled EEG data.

Proposed and implemented Fractal Dimension Model Study. Generalized

Higuchi Fractal Dimension Spectrum.

Proposed and implemented Mental tasks recognition algorithms.

Statistical features could achieve the best accuracy (55.23% ).

Combine all features could enhance the accuracy (59.82%).

With random forests feature selection method, the no. of features used in

classification can be reduced to 77 and the classification can be maintained

(60.41%).

(F8, F3, AF3, O2) channels are important for arithmetic task classification.

Proposed and implemented neurofeedback games based on novel EEG

features.

Conclusion

26

• Parallelize the feature extraction step with MapReduce

Model.

• Develop real-time mental tasks recognition application

based on Hadoop framework.

• Design neurofeedback novel algorithm and compare the

working performance enhancement with alpha train

neurofeedback.

Future Works

27

Outline



Proposed Algorithms

28


Real-time EEG monitoring tool.

Blooby Demo

29

Demonstrate EEG Properties on 3D models.

Support real-time mode and playback mode.

Support interactive operation.

3 type of indicators.

For Presentation/bloby/Blobby-Emotiv7.wmv

Neurofeedback games.

Neurofeedback Demos

30

Book Section: O. Sourina, Q. Wang, Y. Liu, M. K. Nguyen, EEG-enabled Human-Computer Interaction and

Applications, in Towards Practical Brain-Computer Interfaces, B. Allison, etc., Springer, in

press, 2011

Journal Papers:

Sourina, O., Wang, Q., Liu, Y., , Nguyen, M. K., Fractal-based Brain State Recognition from

EEG in Human Computer Interaction, Communications in Computer and Information Science,

In Press

Wang, Q., Sourina, O., Nguyen, M. K., Fractal dimension based neurofeedback in serious

games, Visual Computer, Vol.27, No. 4, pp. 299-309

Sourina, O., Wang, Q., Nguyen, M. K., EEG-based "Serious" games and monitoring tools for

pain management, Studies in Health Technology and Informatics, Vol.163, pp. 606-610

Sourina, O., Liu, Y., Wang, Q., Nguyen, M. K., EEG-based personalized digital experience,

Lecture Notes in Computer Science , Vol.6766, pp. 591-599

Conference Papers: Wang, Q., Sourina, O., Nguyen, M. K., EEG-based "Serious" Games Design for Medical

Applications, Proc. 2010 Int. Conf. on Cyberworlds, 2010, pp. 270-276

Sourina, O., Wang, Q., Liu, Y., , Nguyen, M. K., A real-time fractal-based brain state

recognition from EEG and its applicationse, Proc. 2011 Biosignals , 2011, pp. 82-90

Publication List

31

Q & A

32

Generalized fractal dimension spectrum.

Feature Extraction

33

Statistical features1.

Feature Extraction

34

1. R. Picard, E. Vyzas, and J. Healey, “Toward machine emotional intelligence: Analysis of affective

physiological state,” IEEE Transactionson Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp.

1175–1191, 2001.

Relative Power features1.

Feature Extraction

35

1. S. Sanei and J. A. Chambers, EEG Signal Processing. San Francisco: WILEY, 2007.

Autoregressive coefficients.

AR(6) model is used to model EEG segments.

Feature Extraction

36

1. N. Liang, P. Saratchandran, G. Huang, and N. Sundararajan, “Classification of mental tasks from

eeg signals using extreme learning machine,” International Journal of Neural Systems, vol. 16, no.

1, pp. 29–38, 2006.

The q order difference operator is defined as:

The crossing number is summarize as follow:

Higher order crossing.

Difference operator is defined as:

Feature Extraction

37

1. S. He and B. Kedem, “Higher order crossings spectral analysis of an almost periodic random sequence in

noise,” IEEE Transactionson Information Theory, vol. 35, no. 2, pp. 360–370, 1989.

Entropy.

Entropy could be used as another important quantification feature in nonlinear

dynamical analysis of time series which is related to the rate of information

production.

We calculated three types of entropy which could be applied to short and noisy

time series:

approximate entropy1

sample entropy1

SVD entropy2

Feature Extraction

38

1. J. Richman and J. Moorman, “Physiological time-series analysis using approximate and sample entropy,”

American Journal of Physiology Heart and Circulatory Physiology, vol.278, no.647-6, pp.H2039–H2049,

2000.

2. S. Faul, G. Boylan, S. Connolly, W. Marnane, and G. Lightbody, “Chaos theory analysis of the new born

eeg-is it worth the wait?”, pp. 381–386, 2005.

Random forests.

Random Forests (RF) method proposed by Breiman1 was used as the supervised feature

selection scheme.

This method could deal with the situation when there are many more features than

observations. This method also reduces the risk of overfitting2.

Feature Selection

39

1. L. Breiman, “Random forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.

2. S. Diaz-Uriarte and R. A. deAndres, “Gene selection and classification of microarray data using random

forest,” BMC Bioinformatics, vol. 7, no. 3, 2006.

Other features selection schemes.

LASSO1

Stability selection2

F-score3

All these scheme is implemented by scikit-learning python library4.

Feature Selection

40

1. R. Tibshirani, “Regression shrinkage and selection via the lasso: A retrospective,” Journal of the Royal

Statistical Society. SeriesB:Statistical Methodology, vol. 73, no.3, pp.273–282, 2011.

2. N. Meinshausen and P. Buhlmann, “Stability selection,” Journal of the Royal Statistical Society.

SeriesB:Statistical Methodology, vol.72, no.4, pp.417–473, 2010.

3. Y. Chen and C.Lin, “Combining svms with various feature selection strategies,” Studies in Fuzziness and

Soft Computing, vol. 207, pp.315–324,2006.

4. F. Pedregosa, G. Varoquaux, A. Gramfort, V.Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R.

Weiss, V. Dubourg, J.Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, “Scikit-learn:

Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.