FractalDimensionBasedNeurofeedback_final.pdf

8/11/2019 FractalDimensionBasedNeurofeedback_final.pdf

1/11

Noname manuscript No.(will be inserted by the editor)

Fractal Dimension Based Neurofeedback in Serious Games

Qiang Wang Olga Sourina Minh Khoa Nguyen

Received: date / Accepted: date

Abstract EEG-based technology is widely used in serious

game design since more wireless headsets that meet con-

sumer criteria for wearability, price, portability and ease-of-

use are coming to the market. Originally, such technologies

were mostly used in different medical applications, Brain

Computer Interfaces (BCI) and neurofeedback games. The

algorithms adopted in such applications are mainly based on

power spectrum analysis, which may not be fully revealing

the nonlinear complexity of the brain activities. In this pa-

per, we first review neurofeedback games, EEG processing

methods and algorithms, and then propose a new nonlinear

fractal dimension based approach to neurofeedback imple-

mentation targeting EEG-based serious games design. Only

one channel is used in the proposed concentration quan-

tification algorithm. The developed method was compared

with other methods used for the concentration level recog-

nition in neurofeedback games. The result analysis demon-

strated an efficiency of the proposed approach. We designed

and implemented new EEG-based 2D and 3D neurofeed-

back games that make the process of brain training more

enjoyable.

Keywords EEG HCI BCI neurofeedback fractal

dimension game design medical application

1 Introduction

Electroencephalogram (EEG) is a noninvasive technique

that allows recording the electrical potentials over the scalp

which are produced by activities of brain cortex and re-

flect the state of the brain [31]. Nowadays, EEG-based tech-

niques have been widely used in BCI applications that help

Q. WangO. SourinaM.K. NguyenInstitute for Media Innovation and School of Electrical & Electronic

Engineering, Nanyang Technological University, Singapore, 637553

E-mail: [email protected]

disabled people to communicate with machines [33,34], in

video games as game controllers [25], and in neurofeedback

games [42]. With the availability of portable wireless EEG

devices, EEG-based applications are no longer confined to

the lab environment

Neurofeedback is a technique that presents the real-time

feedback to the user in the form of video display and/or

sound based on the processing results of EEG signals taken

from the scalp of the user [16]. Many researches reveal that

EEG and Event Related Potential (ERP) distortions always

reflect psychological disorders such as Attention Deficit Hy-

peractivity Disorder (ADHD) [28,13], Autistic Spectrum

Disorders (ASD) [9, 39,22, 10], Substance Use Disorders

(SUD) including alcohol and drug abuse [35,36], etc. Simi-

lar to other parts of the body, brain functions can be trained

as well. Neurofeedback (NF) is a technique for training

brain functions and it is an alternative choice for the dis-

orders treatment besides traditional medical treatments.

Currently, EEG signal processing algorithms embedded

in the neurofeedback games extract a power spectrum or an

amplitude feature of the users EEG signals. These features

may not fully reveal the whole complexity of the brain pro-

cess. More advanced models are needed to analyze nonlin-

ear properties of EEG signals. In this paper, we proposed

a new nonlinear fractal dimension approach to neurofeed-

back implementation. We use fractal dimension algorithms

to analyze the complexity of EEG signals. Fractal dimen-

sion values are calculated and used to quantify the level of

the user concentration. Our hypothesis is that changes in the

subjects concentration level could be noticed as changes in

fractal dimension values of the EEG signal. The experiment

results show that the fractal dimension feature can repre-

sent the brain states better than the power spectrum density.

2D and 3D neurofeedback games are implemented based on

the fractal dimension model to help the user to improve the

player concentration ability.


2/11

2 Qiang Wang et al.

In Sect. 2, neurofeedback games, EEG processing meth-

ods and algorithms, and game engines are reviewed. In

Sect. 3, a new fractal dimension based method for neuro-

feedback implementation is introduced. The experiment on

concentration level detection and comparison of the pro-posed algorithm with other methods are described in Sect. 4.

The implementation of the proposed fractal dimension based

neurofeedback games is introduced in Sect. 5. Finally, con-

clusion and discussion of future work are given in Sect. 6.

2 Related work

2.1 Neurofeedback game applications

Many neurofeedback games were assessed by the healing

effect of the ADHD, one of the most known psychologi-

cal disorders with significant EEG distortion. The patients

with ADHD have problems to concentrate. The abnormal

behavior in / ratio was reported in [8]. Besides the ra-tio, the distortion in Slow Cortical Potential (SCP) was also

notified by [14]. Both frequency neurofeedback training and

SCP neurofeedback training can achieve good healing ef-

fects for ADHD patients [14,15].

ASD is another psychological disorder associated with

abnormalities of social interactions and communications as

well as seriously restricted interests and highly repeated ac-

tions [9]. In work [10], EEG analysis during resting condi-

tion with open eyes was done for an eight-year-old girl with

ASD patterns. Theband andband of EEG signal acted

abnormally, and the corresponding neurofeedback scheme

was designed to rectify the abnormalities. After 21 sessions

of the treatment, the sustained attention was enhanced and

the ASD symptoms were decreased. Another research group

also achieved good results in neurofeedback treatment with

standard Quantitative EEG (QEEG) protocol where the aim

is to decrease theta band power at the central and frontal

brain areas [22]. Similar to ASD, General Anxiety Disorder

(GAD) could cause unacceptable social behaviors as well.

GAD can also be treated with EEG band suppression and

symmetry training [20].

SUD including drug or alcohol abuse always leads to

changes in social behaviors. Neurofeedback was proved to

be an affordable alternative treatment for SUD. Chronic al-

coholics show significant diminution inband of EEG sig-

nals. The corresponding neurofeedback treatment could de-

crease the brain waves in this band that was effective in al-

coholic patients treatment [35]. For drug abuse, a decreased

band power and an excess of fast beta band activities

were detected. In addition, a subject with drug addiction had

lower amplitude in P300 ERP component compared to the

controlled subject. The addiction was proved to be relieved

with the long term neurofeedback treatment [36].

Besides medical applications, neurofeedback could also

help a healthy person to enhance his/her brain functions. Re-

searchers indicated that cognitive performances, e.g. cued

recall performance, can be enhanced if a healthy person

learns how to increase special components of EEG signals

with neurofeedback [40,17].

2.2 EEG signal processing methods

The brain state recognition algorithms embedded in BCI

systems especially neurofeedback systems are mainly based

on the power spectrum analysis. EEG signal can be divided

into several different frequency bands, i.e.band (30Hz). The Sensorimotor rhythm activity (12-15Hz) is also used in several neurofeedback systems. Each

frequency band is related to the specific brain functions.

Generally,band is prevalent in infants EEG or EEG when

the subject is sleeping; band is prevalent in EEG when

the subject feels drowsiness; band is significant when the

subject is relaxed; band is associated with fast activities

andband is related to problem solving and memory work

[11]. In the feature extraction step, the power over different

bands are assessed and extracted from the users EEG sig-

nals and then rectified with corresponding therapy, e.g. /ratio therapy, or compared with a standard QEEG database

(QEEG protocol) to generate the adaptive recovery therapy.

In work [29], the / ratio in C4 position according to10-20 International System [19] was successfully used in

neurofeedback to improve attention. The frequency training

method is the most prevalent method used in the neurofeed-

back training systems and other EEG-based games because

the frequency band power is easy to be obtained and an-

alyzed with the existing signal processing tools. Besides,

in work [32], the EEG spectrum weighted frequency also

showed the ability in attention level recognition.

ERP analysis is the process to analyze the EEG signal

synchronized with an event. Slow Cortical Potential (SCP)

and P300 are important ERP approaches applied in neuro-

feedback treatments. SCP reflects the changes in cortical

polarization, i.e. negative and positive trends, of EEG sig-

nals which last from 300ms to several seconds after an event

stimulus [6]. Abnormalities in SCP of ADHD patients were

studied in [14], and the corresponding neurofeedback pro-

tocol could enhance the continuous performance. The P300

component of ERP occurs during 300ms - 600 ms after an

event stimulus which is obtained by oddball paradigm in

which low probability target items are inter-mixed with high

probability non-target items. Researches indicated that the

amplitude of P300 component is related to the process of

allocation of attention resources and its latency reflects the

stimulus evaluation and classification time. The pathology

of P300 component in drug abuse patients was reported in

[36], and neurofeedback based on P300 component training


3/11

Fractal Dimension Based Neurofeedback in Serious Games 3

was proposed.

Although the signal processing algorithms embedded in

neurofeedback games are well applied in clinical treatments,

the linear features, e.g. power spectral density and ampli-

tude, extracted from EEG cannot represent the brain activ-ities perfectly due to the nonlinearity of EEG signal. Non-

linear methods, e.g. entropy analysis and fractal dimension

analysis, have become popular in EEG processing for med-

ical applications [23,24, 38,37] and been applied to neuro-

feedback systems [41] to model brain activities and EEG

based emotion recognition system [27]. With effective non-

linear EEG features, the accuracy of brain state recognition

could be improved, thus the treatment performance of the

neurofeedback games would be enhanced.

2.3 Game Engines

EEG-based games consist of two parts: signal processing

and game implementation. Game implementation can be ef-

fectively done with the help of game engines. Game en-

gines are tools that programmers use to design and imple-

ment games. They provide ready-made utilities or tools to

develop a game. According to Jeff Ward [43], three types of

game engines are frequently seen: roll-your-own game en-

gines, mostly-ready game engines, and point-and-click en-

gines.

Roll-your-own game engines, including OpenGL and

DirectX, require the game makers to be well-versed in pro-

gramming, and it takes a lot of time to design a game.

However, they give the game makers flexibility and more

freedom in designing their own components for the game.

Mostly-ready game engine is most popular in the mar-

ket. Renderer, physics engine, collision detection, graphic,

sound system, etc. are usually available in these game en-

gines. OGRE, Panda3D, Unreal, etc. belong to this kind

of game engine. A point-and-click engine is the highest

level game engine that requires least programming knowl-

edge. However, they are quite limited in the number of the

provided readymade functions. These engines include Al-

ice, Game Maker, etc. As EEG-based games include sig-

nal processing, a game engine should support programming

language C++, Python or any other scripting environments

which allow EEG recognition and interpretation.

In our neurofeedback games, SDL [4] game engine and

Flash CS3 were adopted for 2D games; Panda3D [2] and

Unreal Engine [5] were applied in 3D games. Most of them

are open source and can be easily integrated with our EEG

signal processing methods.

3 Methodology

3.1 EEG-based Game Design

In the past, traditional neurofeedback games were imple-mented for clinical applications with complex EEG devices

which were hard to be set up. Recently, more and more

portable and wireless EEG devices became available. More

effective EEG processing algorithms which could be used

with fewer electrodes in real-time applications are on de-

mand. In this paper, wwe proposed a design of neurofeed-

back concentration games based on fractal dimension model

with one-channel EEG signal. Different from traditional

neurofeedback games, we focus on the monitoring of the

brain state recognized from the EEG using fractal dimension

model. The main idea of concentration games is using the

neurofeedback to encourage the player to improve his/herlevel of concentration. The proposed EEG analysis method

is based on fractal dimension model which could capture

any changes in the brain state.

In order to develop a real-time application, fewer chan-

nels and faster signal processing algorithms should be used.

In our implementation, only one channel located in occipital

lobe is selected as the occipital lobe is responsible for visual

perception and visual attention [26,30]. Entropy based frac-

tal dimension model [21] is used to distinguish brain states

such as relaxed and concentrated.

3.2 Previous Approaches

The main task in our research is brain state recognition, es-

pecially concentration level recognition. There are methods

for attention level recognition. In work [29], / ratio isused for attention level estimation. In work [32], so-called

brain rate is used as a reliable attention level indicator. Both

methods are based on the analysis of power spectrum den-

sity of EEG signals.

In the /ratio method, a power spectrum density ofEEG signals could be estimated with periodogram method.

Discrete Time Fourier Transform (DTFT) with fixed num-

ber of samples (N) is applied to estimate the power densityspectrum (PSD) as shown in (1) and (2):

DT FT{x(n)}=X(ej) =

n=

x(n)ejn (1)

PSD{x(n)}=S() = 1

NX(ej)2 (2)

where x(n) is the input EEG signal, X(ej) is the DTFTof EEG signal, andS() is the power spectrum density of

EEG signal. Theandband power could be estimated by

integrating the power spectrum within the band (4-8Hz)

and band (12-30Hz) respectively. Then, the ratio of the


4/11

4 Qiang Wang et al.

(a) (b)

Fig. 1: Mono-fractal Weierstrass signal (a) fractal dimension value is 1.1 (b) fractal dimension value is 1.7

band power to band power that indicates the attentionlevel can be obtained as follows:

/ ratio =band power

band power=

28/Fs24/Fs S()d

230/Fs212/Fs S()d

(3)

whereF sis the sampling frequency of the EEG device.

In the brain rate method, the PSD is also estimated first

with periodogram method as illustrated before. The brain

rate can be calculated as spectrum weighted frequency as

follows,

fb(brain rate)= 2Fs

S()d

S()d (4)

whereF sis the sampling frequency of the EEG device.

3.3 Fractal Dimension Model

We proposed to use fractal dimension as a feature instead

of the power of EEG signals in the implementation of

neurofeedback in serious games. Fractal dimension could

be used as a measurement of complexity and irregularity

of a signal. In signal processing, Higher fractal dimension

value means that the signal is more complex, while lowerfractal dimension (FD) value means that the signal is more

regular. In Fig.1a and Fig. 1b, examples of mono-fractal

Weierstrass signals with low FD value 1.1 and high FD

value 1.7 are shown. In our real-time implementation,

Higuchi [18] and box-counting [7] algorithms were chosen

for FD calculation.

Let us briefly describe the algorithms.

In Higuchi method, the samples are first clustered into

several sub-sequences according to the poly-phase structure

as follows:

Xmk :x(m),x(m + k),x(m + 2k),...,x(m + (N mk)k) (5)

The length of the sequences L(k)is calculated according to(6) and (7):

Lk(m) =

1

k[(

(Nmk )

i=1

|x(m + ik) x(m + (i 1)k)|) N 1

(Nmk

)k]

(6)

L(k) =1

k

k

m=1

Lk(m) (7)

wherekdenotes the number of the sub-sequences and Lk(m)denotes the length of the m-th sub-sequence. The total length

L(k)is proportional tokFD :

L(k) kFD (8)

where FD is the fractal dimension value and k is the time-

delay information. The fractal dimension value could be cal-

culated as follows,

FD=logL(k)

log k(9)

The algorithm is shown in Alg. 1.

Box-counting method can calculate the fractal dimen-

sion values of the signal in time domain without any sub-

sequence extraction steps. The main step of box-counting

method is box construction. Unified and normalized boxes

are constructed in 2D space (time-amplitude) which can

cover one segment of the signal. Finally, the number of oc-

cupied boxes is counted. The boxes constructing and count-

ing processes are shown in Fig. 2. The number of counted

boxesN(d)is proportional todFD :

N(d) dFD (10)

where d denotes the length of the side of the boxes. The

fractal dimension value could be calculated as follows:

FD= logN(d)log d

(11)


5/11


Algorithm 1Higuchi Method

kMax2log2(length(x))4

xListem ptyList(){list for storing data in x direction for calculat-ing the slope}

yListem ptyList(){list for storing data in y direction for calculat-ing the slope}fork=1 to kMaxdo

sumlk0form=1 to kdo

Xmk extractSubSequence(X)lmkSumSubSequence(Xmk )sumlksumlk+ lmkmm + 1

end for

lksumlk/kxListAdd El ement(log(1/k))yListAdd El ement(log(lk))kk+ 1

end for

f dgetSlope(xList,yList)

Fig. 2: Boxes construction in box-counting method, the dark

boxes are counted

Algorithm 2Box-counting Method

kMax log2(length(x)) 1xListem ptyList(){list for storing data in x direction for calculat-ing the slope}

yListem ptyList(){list for storing data in y direction for calculat-ing the slope}fork=1 to kMaxdo

d2k

ConstructNormalizedBoxes{X Y directions have the same num-ber of sides

}Nd= countBox(){Count the no. of boxes occupied by the EEGsignal}

xListAdd El ement(log(1/d))yListAdd El ement(log(Nd))kk+ 1

end for

f dgetSlope(xList,yList)

The algorithm is shown in Alg. 2.

Higuchi and box-counting methods were compared in

both computation complexity and accuracy. Brownian Mo-

tion and Weierstrass mono-fractal signals whose theoretical

fractal dimension values are known were used for the com-parison. The results are shown in Fig. 3. Although Higuchi

method is slower than box-counting method, the accuracy

of Higuchi method is better than box-counting method in

FD value evaluation for both Brownian Motion and Weier-

strass signals. In our work, both algorithms were used for

neurofeedback implementation.

3.4 Classifier Comparison and Threshold Evaluation

Receiver Operating Characteristics (ROC) graph is a tech-

nique for comparing the performance of classifiers or

method of threshold selection, which is especially useful intwo classes case [12]. In two classes training situation, a

score is calculated for all samples with labels. A threshold is

needed to classify all these labeled samples into two classes

according to the score calculated in the feature extraction

process. For each specific threshold, a confusion matrix can

be constructed (shown in Fig. 4a) and the true positive (t p)

rate and false positive (f p) rate can be evaluated as follows:

t p=Positives correctly classified

Total positives (12)

f p= Negatives incorrectly classifiedTotal negatives (13)

The ROC space can be constructed with the f prate asxaxis

and t prate asyaxis. The performance of the threshold could

be drawn as a point in ROC space (shown in Fig. 4b, the

threshold corresponding to point A has best performance).

If the point is closer to the northwest corner in ROC space,

the accuracy is higher. A ROC curve can be generalized

when different thresholds are used. The best threshold

is chosen according to the distance of the corresponding

ROC point to the northwest corner. The performance of the

classifier could be evaluated by the area of the southeast

ROC plane separated by the ROC curve (shown in Fig. 4c).The bigger area means that the average performance of the

classifier is better. In Fig. 4c, the classifier with ROC curve

in red has better average performance even though the best

accuracy is worse.

4 Experiments and results

Because there is no standard database and benchmark avail-

able for the concentration (or attention) level recognition,

an experiment on brain state classification was set up to dis-

tinguish relaxation and concentration states. Five subjects


6/11

6 Qiang Wang et al.

(a) (b)

Fig. 3: The comparison of Higuchi and box-counting algorithms for FD evaluation over (a) Brownian Motion signals and (b)

Weierstrass signals

(a)

0 0.5 10

0.2

0.4

0.6

0.8

1

False Positive Rate

TruePositiveRate

ROC Space

A

B

C

(b) (c)

Fig. 4: Illustration of ROC (a) Confusion Matrix (b) ROC space (c) comparison of two classifiers

Table 1: Comparison of brain state recognition methods

Higuchi Box-Counting Brain Rate / ratio

Accuracy mean 0.8811 0.8654 0.8493 0.8026

var 0.0077 0.0140 0.0078 0.0190

Best Threshold mean 1.9331 1.6048 15.8071 1.6701

var 0.0025 0.0022 35.05 0.2557

Time Consuming (s) mean 0.0172 0.0050 0.0035 0.0043

aged from 22 to 30 were invited to participate in the exper-

iment. In the first session, in order to induce the relaxation

state, a comfortable environment was set up to help the sub-

ject relax. In the second session, in order to induce the con-

centration state, the subjects were required to complete sev-

eral math problems. Only one electrode was used and placed

in O1 position according to the 10-20 international system

[19] in occipital lobe which is associated with visual percep-

tion. EEG signals were recorded by the Emotiv [35] device

with sampling frequency of 128Hz and 16-bit A/D resolu-

tion. The EEG data of two sessions were divided into thou-

sands of pieces. Each piece of data had 1024 samples and

was labeled according to the induced brain states. Four dif-

ferent methods, i.e. Higuchi method, box-counting method,

/ratio method and brain rate, were applied for the EEGsignal processing and evaluated with ROC curve. The data

were processed with Matlab on an Intel Core 2 Quad CPU

Q9400 (2.66GHz) with a 3.25GB main memory.

The results of the brain state recognition experiment are

shown in Fig. 5 and Table 1. Fig. 5 gives an intuitive com-

parison of four methods we evaluated with ROC. In most

cases, the curves for Higuchi and box-counting methods

are closer to the northwest corner and cover more area in

ROC domain. This implies that Higuchi and box-counting


7/11


0 0.5 10

0.2

0.4

0.6

0.8

1ROC curve of brain state recognition

False Positive Rate

TruePositive

Rate

HiguchiBoxCounting

/ ratio

Brain rate

(a) Subject 1

0 0.5 10

0.2

0.4

0.6

0.8


False Positive Rate

TruePositive

Rate

HiguchiBoxCounting

/ ratio

Brain rate

(b) Subject 2

0 0.5 10

0.2

0.4

0.6

0.8


False Positive Rate

TruePositive

Rate

HiguchiBoxCounting

/ ratio

Brain rate

(c) Subject 3

0 0.5 10

0.2

0.4

0.6

0.8


False Positive Rate

TruePositiveRate

HiguchiBoxCounting

/ ratio

Brain rate

(d) Subject 4

0 0.5 10

0.2

0.4

0.6

0.8


False Positive Rate

TruePositiveRate

HiguchiBoxCounting

/ ratio

Brain rate

(e) Subject 5

Fig. 5: ROC curve of brain state recognition methods for 5 subjects

Concentration State Relax State

1.8

1.85

1.9

1.95

2

Boxplot for Higuchi Method

(a)

Concentration State Relax State

1.5

1.55

1.6

1.65

Boxplot for Boxcounting Method

(b)

Fig. 6: The comparison of (a) Higuchi method and (b) Box-counting method in FD evaluation of the EEG signals in different

brain states for all subjects


8/11

8 Qiang Wang et al.

Fig. 7: Concentration based neurofeedback flowchart

methods could achieve a better result than the / ratioand brain rate methods. Table 1 gives the mean and vari-

ance of the accuracy and best threshold for 5 subjects infour different methods. The accuracy of Higuchi method is

88.11% with the variance of 0.0077 and the accuracy of

box-counting method is 86.54% with variance of 0.0014.

Both of them are better than / ratio method of whichthe accuracy is 80.26% with variance of 0.019 and brain

state method of which the accuracy is 84.93% with variance

of 0.0078. The variance in threshold also shows the box-

counting and Higuchi methods have relatively smaller dif-

ference in thresholds for different subjects. This result im-

plies that, for different users, the changes in threshold might

be insignificant when fractal dimension model is adopted

in neurofeedback games. The average time for processingof 1024 samples was also given in Table 1. The Higuchi

method is slower than the other methods. However, its speed

is fast enough for real-time processing when the sampling

frequency is 128Hz.

Both Higuchi and box-counting methods were imple-

mented as fractal dimension calculation methods and then

were applied in the neurofeedback games. The efficiency of

these two methods is shown in Fig. 6 with boxplot. The

states of relaxation and concentration can be separated by

FD values for all five subjects. In both Higuchi and box-

counting algorithms, the experiment results show that con-

centration level can be distinguished for 80% of the sub-

jects when a default threshold is set to 1.93 in Higuchi

method and 1.60 in box-counting method. For 100% of the

subjects, the concentration level can be recognized with atrained threshold. It is clear that fractal dimension model

can be used to distinguish the relaxation and concentration

states with a simple threshold even though there are over-

laps which may be due to individual differences in EEG. A

short training session can be applied to determine the default

threshold to minimize the individual effects.

5 Neurofeedback Games Implementation

2D neurofeedback games such as Brain Chi and Pipe,

and 3D neurofeedback games such as Dancing Robot,

Escape were proposed and developed for concentration

level control. Brain Chi was developed with SDL game

engine [4], Pipe- with Flash CS3, Dancing Robot

- with Panda3D [2] and Escape was developed with

Unreal Engine [5]. All games were scripted and compiled

under Microsoft Visual C++ environment except Escape

and Pipe which were scripted with Unreal Script and

ActionScript respectively.


9/11


5.1 Neurofeedback Hardware Setting

EEG data could be acquired by Emotiv [1] or PET 2 EEG

devices. Only O1 electrode (according to the 10-20 inter-

national system) in Occipital lobe is active, and the EEGsigmal is transmitted to computer with Bluetooth. The al-

gorithm uses 2-42Hz band-pass filter first, then it calculates

fractal dimension values of the input EEG signals in real-

time with Higuchi method or box-counting method and la-

bels them with different brain states according to the adap-

tive thresholds. The default threshold used to distinguish

the concentration state and relaxation state is set up to 1.93

in Higuchi based method and 1.6 in box-counting based

method. The threshold is adaptive to the users current state.

The data acquisition and processing algorithm flowchart is

shown in Fig. 7, and hardware setup of the neurofeedback

games is shown in Fig. 8a. The user playing neurofeed-back game Dancing Robot with Emotiv device is shown

in Fig. 8b.

5.2 Game Strategy

The proposed games have the following general game strat-

egy. If the concentration level has been achieved by the

user, points are rewarded to encourage the player. With de-

crease of the concentration level the player could lose the

reward points. The concentration based game strategy is pre-

sented in the flowchart in Fig. 7. The game strategy could be

changed to relaxation game if the player wanted to be trained

how to relax.

Dancing Robot is a simple 3D single-player game. In

this game, the player is required to control the speed of the

robot while the robot is dancing. A screenshot of the Danc-

ing Robot game is shown in Fig. 9a. The speed of the robot

dance depends on the concentration level of the player. The

player concentrates to make the robot dancing faster. The

robot dances sluggishly if the player is distracted. An adap-

tive threshold is used for training purpose.

Brain Chi is a 2D single-player EEG-based game. A

screenshot of Brain Chi is shown in Fig. 9b. The player

controls the game simply by using his/her brain power of

concentration. His/her task is to help a little boy hero to fight

against evil bats using a protection ball. The size of the pro-

tection ball is controlled by the concentration level of the

player. To win the game, the player needs to increase the

ball by concentration to eliminate all the bats.

Pipe is a classic pipe game where the player tries to

arrange pieces of pipes on a board so the water can be de-

livered between two stations without leaking. This game is

slightly different from the above two games. The recognized

brain state could influence on the games flow. For example,

distraction could cause the time allocated to the player to be

reduced and hence, upon recognizing this, the player is ex-

pected to put more effort in concentrating. In Fig. 9c, water

level (green bar) serves as a time indicator and concentration

level (blue bar) serves as a concentration level. The blue bar

has effect on the rate of change of the green bar.Escape is also a 3D single-player game. However, this

game has an educational purpose. The story in the game re-

quires the player to solve the educational puzzles so he /she

could get the passwords to unlock the doors. In this game,

EEG could be used as an alternative way to get the pass-

words when the player cant figure out how to solve those

puzzles. The player has to stay concentrated for a specified

time, and the password will be given to him. If the player

uses brain power help, the overall game time allocated for

the player to escape is reduced. The screenshot of the game

is shown in Fig. 9d.

Examples of the games are given in [3].

6 Conclusion and Future Work

In this paper, we reviewed EEG-based neurofeedback

games and algorithms embedded in the neurofeedback

games. We proposed a novel fractal dimension based

method to quantify concentration level of the brain state.

The proposed method was compared with other methods

used for concentration level recognition in neurofeedback

games. The comparison results showed that both Higuchi

and box-counting algorithms could be effectively used forfeature extraction in the proposed method. The brain states

were recognizable with the difference in fractal dimension

values. Original 2D and 3D EEG-based games such as

Brain Chi, Dancing Robot, Pipe, and Escape

were designed and implemented for concentration training.

The fractal dimension algorithms were embedded in the

neurofeedback games to enhance the efficiency of the

neurofeedback.

More research is planned in the future to compare the

efficiency of the proposed fractal dimension approach and

traditional signal processing approaches in neurofeedback

games. The possibility of applying the fractal dimensionbased neurofeedback in pain management would be studied

as well.

Acknowledgements This project is supported by grant

NRF2008IDM-IDM004-020 of National Research Fund of Sin-

gapore and by Institute for Media Innovation.

References

1. Emotiv. URL http://www.emotiv.com2. Panda3d. URL http://www.panda3d.org.


10/11

10 Qiang Wang et al.

(a) (b)

Fig. 8: Hardware setup of the game (a) neurofeedback system hardware (b) equipped user

(a) Dancing Robot screenshot (b) Brain Chi screenshot

(c) Pipe screenshot (d) Escape screenshot

Fig. 9: Screenshots of neurofeedback 2D and 3D games (a)Dancing Robot (b)Brain Chi (c)Pipe (d)Escape


11/11


3. Project personalized digital media experience. URL

http://www3.ntu.edu.sg/home/EOSourina/projects.html

4. Simple directmedia layer. URL http://www.libsdl.org

5. Unreal engine. URL http://www.udk.com

6. Birbaumer, N.: Slow cortical potentials: Plasticity, operant control,

and behavioral effects. Neuroscientist5(2), 7478 (1999)7. Block, A., Von Bloh, W., Schellnhuber, H.J.: Efficient box-

counting determination of generalized fractal dimensions. Physi-

cal Review A42(4), 18691874 (1990)

8. Clarke, A.R., Barry, R.J., McCarthy, R., Selikowitz, M.: Elec-

troencephalogram differences in two subtypes of attention-

deficit/hyperactivity disorder. Psychophysiology38(2), 212221

(2001)

9. Coben, R., Linden, M., Myers, T.E.: Neurofeedback for autis-

tic spectrum disorder: A review of the literature. Applied Psy-

chophysiology Biofeedback35(1), 83105 (2010)

10. Cowan, J.D., Markham, L.: Eeg biofeedback for the attention

problems of autism - a case study. Biofeedback and Self-

Regulation 19(3), 287287 (1994)

11. Demos, J.N.: Getting Started with Neurofeedback. WW Norton &

Company, New York (2005)12. Fawcett, T.: An introduction to roc analysis. Pattern Recognition

Letters27(8), 861874 (2006)

13. Fuchs, T., Birbaumer, N., Lutzenberger, W., Gruzelier,

J.H., Kaiser, J.: Neurofeedback treatment for attention-

deficit/hyperactivity disorder in children: A comparison with

methylphenidate. Applied Psychophysiology Biofeedback28(1),

112 (2003)

14. Gevensleben, H., Holl, B., Albrecht, B., Schlamp, D., Kratz, O.,

Studer, P., Wangler, S., Rothenberger, A., Moll, G.H., Heinrich,

H.: Distinct eeg effects related to neurofeedback training in chil-

dren with adhd: A randomized controlled trial. International Jour-

nal of Psychophysiology74(2), 149157 (2009)

15. Gevensleben, H., Holl, B., Albrecht, B., Vogel, C., Schlamp, D.,

Kratz, O., Studer, P., Rothenberger, A., Moll, G.H., Heinrich, H.:

Is neurofeedback an efficacious treatment for adhd? a randomisedcontrolled clinical trial. Journal of Child Psychology and Psychi-

atry and Allied Disciplines50(7), 780789 (2009)

16. Hammond, D.C.: What is neurofeedback? Journal of Neurother-

apy10(4), 2536 (2006)

17. Hanslmayr, S., Sauseng, P., Doppelmayr, M., Schabus, M.,

Klimesch, W.: Increasing individual upper alpha power by neu-

rofeedback improves cognitive performance in human subjects.

Applied Psychophysiology Biofeedback30(1), 110 (2005)

18. Higuchi, T.: Approach to an irregular time series on the basis of

the fractal theory. Physica D: Nonlinear Phenomena31(2), 277

283 (1988)

19. Homan, R.W., Herman, J., Purdy, P.: Cerebral location of interna-

tional 10-20 system electrode placement. Electroencephalography

and Clinical Neurophysiology66(4), 376382 (1987)

20. Kerson, C., Sherman, R.A., Kozlowski, G.P.: Alpha suppressionand symmetry training for generalized anxiety symptoms. Journal

of Neurotherapy13(3), 146155 (2009)

21. Kinsner, W.: A unified approach to fractal dimensions. In: Proc.

ICCI 2005: Fourth IEEE International Conference on Cognitive

Informatics, pp. 5872 (2005)

22. Kouijzer, M.E.J., vanSchie, H.T., de Moor, J.M.H., Gerrits, B.J.L.,

Buitelaar, J.K.: Neurofeedback treatment in autism. preliminary

findings in behavioral, cognitive, and neurophysiological func-

tioning. Research in Autism Spectrum Disorders4(3), 386399

(2010)

23. Kulish, V., Sourin, A., Sourina, O.: Analysis and visualization of

human electroencephalograms seen as fractal time series. Journal

of Mechanics in Medicine and Biology 6(2), 175188 (2006)

24. Kulish, V., Sourin, A., Sourina, O.: Human electroencephalograms

seen as fractal time series: Mathematical analysis and visualiza-tion. Computers in Biology and Medicine 36(3), 291302 (2006)

25. Lecuyer, A., Lotte, F., Reilly, R.B., Leeb, R., Hirose, M., Slater,

M.: Brain-computer interfaces, virtual reality, and videogames.

Computer 41(10), 6672 (2008)

26. Li, Z.H., Coles, C.D., Lynch, M.E., Ma, X.Y., Peltier, S., Hu, X.P.:

Occipital-temporal reduction and sustained visual attention deficit

in prenatal alcohol exposed adults. Brain Imaging and Behavior2(1), 3948 (2008)

27. Liu, Y., Sourina, O., Nguyen, M.K.: Real-time eeg-based human

emotion recognition and visualization. In: Proc. 2010 Int. Conf.

on Cyberworlds, pp. 262269 (2010)

28. Lubar, J.F., Swartwood, M.O., Swartwood, J.N., ODonnell, P.H.:

Evaluation of the effectiveness of eeg neurofeedback training for

adhd in a clinical setting as measured by changes in t.o.v.a. scores,

behavioral ratings, and wisc-r performance. Biofeedback and Self-

Regulation 20(1), 8399 (1995)

29. Lutsyuk, N.V., Eismont, E.V., Pavlenko, V.B.: Modulation of at-

tention in healthy children using a course of eeg-feedback ses-

sions. Neurophysiology38(5-6), 389395 (2006)

30. Murray, S.O., Wojciulik, E.: Attention increases neural selectivity

in the human lateral occipital complex. Nature Neuroscience7(1),

7074 (2004)31. Nunez, P.L., Srinivasan, R.: Electric Fields of the Brain. Oxford

University Press, New York (2006)

32. Pop-Jordanov, J., Pop-Jordanova, N.: Neurophysical substrates of

arousal and attention. Cognitive Processing10(1), 7179 (2009)

33. Rebsamen, B., Burdet, E., Guan, C., Zhang, H., Teo, C.L., Zeng,

Q., Ang, M., Laugier, C.: A brain-controlled wheelchair based on

p300 and path guidance. In: Proc.1st IEEE/RAS-EMBS Interna-

tional Conference on Biomedical Robotics and Biomechatronics,

pp. 11011106 (2006)

34. Rebsamen, B., Teo, C.L., Zeng, Q., Ang Jr, M.H., Burdet, E.,

Guan, C., Zhang, H., Laugier, C.: Controlling a wheelchair in-

doors using thought. IEEE Intelligent Systems 22(2), 1824

(2007)

35. Saxby, E., Peniston, E.G.: Alpha-theta brainwave neurofeedback

training: An effective treatment for male and female alcoholicswith depressive symptoms. Journal of Clinical Psychology51(5),

685693 (1995)

36. Sokhadze, T.M., Cannon, R.L., Trudeau, D.L.: Eeg biofeedback as

a treatment for substance use disorders: Review, rating of efficacy,

and recommendations for further research. Applied Psychophysi-

ology Biofeedback33(1), 128 (2008)

37. Sourina, O., Kulish, V., Sourin, A.: Novel tools for quantification

of brain responses to music stimuli. In: Proc. 13th International

Conference on Biomedical Engineering, pp. 411414 (2009)

38. Sourina, O., Sourin, A., Kulish, V., Gagalowicz, A., Philips, W.:

Eeg data driven animation and its application. In: Proc. Com-

puter Vision/Computer Graphics Collaboration Techniques, pp.

380388 (2009)

39. Thompson, L., Thompson, M., Reid, A.: Neurofeedback outcomes

in clients with aspergers syndrome. Applied PsychophysiologyBiofeedback35(1), 6381 (2010)

40. Vernon, D., Egner, T., Cooper, N., Compton, T., Neilands, C.,

Sheri, A., Gruzelier, J.: The effect of training distinct neurofeed-

back protocols on aspects of cognitive performance. International

Journal of Psychophysiology47(1), 7585 (2003)

41. Wang, Q., Sourina, O., Nguyen, M.K.: Eeg-based serious games

design for medical applications. In: Proc. 2010 Int. Conf. on Cy-

berworlds, pp. 270276 (2010)

42. Wang, Q., Sourina, O., Nguyen, M.K.: Fractal dimension based

algorithm for neurofeedback games. In: Proc. CGI 2010, p. SP25

(2010)

43. Ward, J.: What is a game engine? URL http:// www .gamecareer-

guide .com/features/529/what is a game.php

Documents

FractalDimensionBasedNeurofeedback_final.pdf