View
225
Download
0
Category
Preview:
Citation preview
8/13/2019 IT10 0560 80
1/41
Smart SMS-SMS Application Management Platform
BRAIN COMPUTER INTERFACE APPLICATION
FRAMEWORK
T.N. Malalasekera
(IT 10 0560 80)
Degree of Bachelor of Science in Information Technology
Department of Information Technology
Sri Lanka Institute of Information Technology
October 2013
H. H. Rajamanthrie IT 10 0296 64
8/13/2019 IT10 0560 80
2/41
Smart SMS-SMS Application Management Platform
BRAIN COMPUTER INTERFACE APPLICATION
FRAMEWORK
T.N. Malalasekera
(IT 10 0560 80)
Dissertation submitted in partial fulfillment of the requirements for the degree
of Science
Department of Information Technology
Sri Lanka Institute of Information Technology
October 2013
H. H. Rajamanthrie IT 10 0296 64
8/13/2019 IT10 0560 80
3/41
DECLARATION
I declare that this is my own work and this dissertation does not incorporate without
acknowledgment any material previously submitted for a Degree or Diploma in any Other
University or Institute of higher learning and to the best of my knowledge and belief it does
not contain any material previously published or written by another person except where the
acknowledgment is made in the text.
Also, I hereby grant to Sri Lanka Institute of Information Technology the nonexclusive right
to reproduce and distribute my dissertation, in whole or in part in print, electronic or other
medium. I retain the right to use this content in whole or part in future works (such as articles
or books)
Signature: Date: 23.10.2013
The above candidate has carried out research for the B.Sc. Dissertation under my supervision.
Signature of the supervisor: Date: 23.10.2013
8/13/2019 IT10 0560 80
4/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
ACKNOWLEDGEMENT
We take this opportunity to express our deep sense of gratitude to those who contributed to
either our collective or individual efforts. Particularly, our special thanks go to Sri Lanka
Institute of Information Technology (SLIIT) for providing necessary resources to complete
the project successfully. Special thanks are extended to the lecture in-charge, Mr. Jayantha
Amararachchi who provided the required lecture materials and the necessary guidance to
complete the project successfully. Our heartfelt thanks go out to the supervisor of our project,
Dr. Rohana Priyantha Thilakumara and the co-supervisor, Mr. Darshika Koggalahewa for the
kind patience, guidance and constant support rendered to us at all the stages of the project,
right from the very beginning, till the final presentation of our completed project. The team
extends sincere gratitude to all colleagues, who forwarded their enthusiastic ideas during the
requirements gathering phase of the project. The teammates would also like to thank all the
staff members of the SLIIT Malabe campus for their valuable suggestions and opinions that
appear in the final product. Finally, we thank all who lend their kind support; friends and
families who continued to give their insights, patience, support and co-operation which
motivated us in reaching greater heights.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY I
8/13/2019 IT10 0560 80
5/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
ABSTRACT
Brain-Computer Interface (BCI) technology is a potentially powerful communication and
control option in the interaction between Human Brain and Computer systems. A BrainComputer Interface (BCI) is a direct communication pathway between the brain and an
external device. As many companies have introduced many BCI devices in to the market
developers have focused on developing BCI based applications. Even though there are many
devices available, they are not capable enough for focusing on developing advanced
application. Neurosky is one of the leading company which producing BCI devices to the
industry. Neurosky mind wave device is capable of giving brain signals according to the
attention and meditation level as well as the eye blink strength. In the proposed project the
team intended to develop a framework which will be utilized every functions needed by game
developers. But according to the data has collected using various users it clarified that the
device is not capable of developing an advance application as it was failed to give a stable
value which will be helped in developing advanced applications. Therefore within the project,
the team is introduced some applications which will be useful for disable people and which
will be giving entertainment for the users.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY II
8/13/2019 IT10 0560 80
6/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
ContentsDECLARATION.................................................................................................................................... 1
ACKNOWLEDGEMENT....................................................................................................................... i
ABSTRACT............................................................................................................................................ ii
List of figures......................................................................................................................................... iv
List of Abbreviations.............................................................................................................................. v
1 INTRODUCTION.......................................................................................................................... 1
1.1 Background Context............................................................................................................... 1
1.2 Research Problem to be addressed.......................................................................................... 3
1.3 Research Questions........................................................................................................................... 5
2 CONTENT...................................................................................................................................... 6
2.1 Addressing the Literature........................................................................................................ 6
2.2 Methodology........................................................................................................................... 9
2.2.1 Overview......................................................................................................................... 9
2.2.2 Overview of the System Design.................................................................................... 10
2.2.3 User Characteristics...................................................................................................... 11
2.2.4 Product functions.......................................................................................................... 12
2.2.5 Tools and Technologies................................................................................................ 13
2.2.6 Product Constraints....................................................................................................... 14
2.2.7 Assumptions and Dependencies.................................................................................... 15
2.3 Research Findings................................................................................................................. 16
3 RESULT AND DISCUSSION..................................................................................................... 22
4 CONCLUSION............................................................................................................................. 31
References ............................................................................................................................................ 32
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY III
8/13/2019 IT10 0560 80
7/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
List of figures
Figure 1................................................................................................................................................... 8
Figure 2................................................................................................................................................. 10
Figure 3................................................................................................................................................. 18
Figure 4................................................................................................................................................. 18
Figure 5................................................................................................................................................. 19
Figure 6................................................................................................................................................. 19
Figure 7................................................................................................................................................. 20
Figure 8................................................................................................................................................. 20
Figure 9................................................................................................................................................. 21
Figure 10............................................................................................................................................... 22Figure 11............................................................................................................................................... 23
Figure 12............................................................................................................................................... 24
Figure 13............................................................................................................................................... 25
Figure 14............................................................................................................................................... 26
Figure 15............................................................................................................................................... 27
Figure 16............................................................................................................................................... 28
Figure 17............................................................................................................................................... 29
Figure 18............................................................................................................................................... 30
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY IV
8/13/2019 IT10 0560 80
8/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
List of Abbreviations
EEG Electro - Electroencephalogram
BCI Brain Computer InterfacefMRI functional Magnetic Resonance Imaging
EEC Encephalogram
SSVEP Steady State Visual Evoked Potential
ALS Amyotrophic Lateral Sclerosis
API Application Programming Interface
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY V
8/13/2019 IT10 0560 80
9/41
1 INTRODUCTION1.1 Background ContextAlthough it seems a science fiction, there are currently brain-computer interfaces, and
innovative research are rapidly expanding the level of control that can be achieved. The
researchers, psychologists, artists, and others have been experimenting with brain-computer
interfaces that read noninvasive brain signals with an electroencephalogram (EEG).
Computer Interfaces based on EEG of the brain by sensors placed on the head to detect brain
waves and feed them as input to a computer.
BCI has always been a topic of interest in the field of Rehabilitation and Assistive
Technology. BCI The potential is limitless when it comes to improving the lives of people
with disabilities. For example, BCI-based systems could be used for people who are severely
restricted or cannot move their hands (Eg:- spinal cord injury) to conduct an electric
wheelchair, operate appliances and so on. Or even to help someone in a vegetative state to
communicate by speaking the words that the individual would like to say.
There are two main categories of BCI systems - "invasive" and "noninvasive". Invasive
systems interact directly with the brain via electrodes / sensors are implanted in the brain or
on its surface. While non-invasive brain interact indirectly via electrodes / sensors on the
surface of the head that detects emissions brain signals (eg electroencephalography (EEG),
magnetic resonance imaging (fMRI), and magnetic sensor systems).
Noninvasive BCI systems usually consist of a head cover (aka EEG cap) with multiple holes /
slots to put the electrodes in the relevant areas of the surface of the head to detect and recordelectrical signals from the brain emitter. Electro gel is used in the electrodes to improve the
contact between the scalp and the electrode. Due to the requirement of a gel such electrodes
are also known as wet electrodes. Existing systems can be from a few to more than 100
electrodes. The Practical using wet electrodes, as an example of gel drying, repeated cleaning
of the electrodes and the skin head to configure EEG Cap, sensitive skin irritation due to the
application of gel, etc. - does not make them suitable for a quick setup and everyday use. Due
to the above disadvantages that prompted the development of dry electrodes. Unlike wet
electrode, dry electrodes do not require the use of gel and can be configured directly direct
8/13/2019 IT10 0560 80
10/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
into the EEG Cap. Although still in the infancy of its developmental cycle, dry electrodes are
quickly catching up in terms of detecting high quality brain signals as their wet counterpart.
Regular comparison studies are being carried out to evaluate the performance of Wet vs. Dry
Electrodes within the context of EEG based noninvasive BCI.
With the advancement of technology, several companies have been motivated in producing
BCI devices to the industry. Most commercially available devices are produced using dry
electrodes. NeuroSky wave mobile headsets mind and brain waves are one of the most
popular devices among developers and new users. In application development, developers are
driven in using these devices. However, some devices are not able to meet the requirements
of the developer in the development of advanced applications. When collecting data
regarding the level of attention and meditation levels of users can vary from one to another.
And when comparing the final results that you can make the output given by the device is not
much more able to use in developing applications. Because it is a mandatory to achieve a
stable average value of meditation and the level of care.
But most devices on the market are not good enough for the development of advanced
applications. Instead, they are suitable for use in applications simple BCI joined Simple
assistive technology and entertainment purposes such as developing improved BCI games
etc.
Neurosky has developed a sensor, noninvasive bio dry reading of electrical activity in the
brain to determine states of relaxation. NeuroSky is a low cost easy to use Encephalogram
(EEC) developed for leisure. Capture neuronal activity with three dry electrodes placed
below the ears, forehead and decoded by algorithms that apply. NeuroSky provides user
information on Delta, Theta, Alpha, Beta and power levels of gamma brain waves band.
NeuroSky detects attention, meditation and eye blinking levels based directly on the brain
activity of the user, and outputs a number per second on a numerical scale every emotion
captured. Using these figures, users can be grouped in different care, meditation and eye
blink categories and looked at the possibility that the timestamp precise moments when the
user makes a mistake.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 2
8/13/2019 IT10 0560 80
11/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
1.2 Research Problem to be addressedBCI serious games measure brainwaves using BCI functions provided by a diverse range of
third parties, and game developers may spend a lot of time supporting diverse BCI devices.
Thus, various devices should be controlled by a unified interface to reduce development time.
When considering the development of Brain Computer Interface serious games following
issues should be taken into consideration. First, the necessary awareness of characteristics of
brainwaves and brain function should be minimized, as this can be a considerable burden for
development. It is difficult for game developers to effectively develop BCI entertainment
serious games to its users. Therefore, a number of projects that cannot be started due to lack
of qualified experts or due to cost issues. To minimize the knowledge required by the game
developers, expert knowledge must be described in such a way that the entertainment game
developers can understand and apply to BCI serious games easily.
It is very difficult to use brain waves to control BCI serious games, as each user will have a
different level of brain wave emission. For example, some users brainwaves have measured
amplitude is larger than the other. Therefore, it requires a process that normalizes individual
characteristics, and then transfers the brain waves of the signal for the BCI serious game.
The development process requires systematic BCI serious game experts and game developers
dealing with the functions of the brain and brain waves. This will allow different developers
to quickly and easily produce games together.
Existing research on the development of BCI applications has been generally approaches to
extract features of brain waves, instead of the functions necessary for the development of the
game. Some researchers have integrated 3D engine for developing 3D games. However, these
studies have not provided a solution to the following requirements for BCI various serious
game development.
It is difficult for game developers to apply brainwave entertainment to games because usually
do not have enough knowledge of the characteristics of brainwaves and brain functions.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 3
8/13/2019 IT10 0560 80
12/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Consequently, the development process related BCI must be separated from BCI
development of serious games and managed by experts in the field of brain waves.
Existing frameworks (with the exception of Open-ViBE) usually are not compatible with
economic BCI devices suitable for BCI serious games. In general, the devices used for BCI
BCI applications are expensive multi-channel accurately analyze a user's brain waves.
However, BCI serious games can use low-cost devices BCI to allow more users to access
these devices include EPOC Emotiv and NeuroSky Mindset, but its accuracy is relatively low
compared to more expensive devices. In some cases, a number of functions cannot be used
because the devices used in BCI serious games are different from those used in BCI
applications.
The above researches have not provided a transform function to allow the measured
brainwaves to be used as the game control signals
Therefore, in the proposed project, is supposed to implement a framework that is suited for
the use of all the needs of developers. Although the requirements are met in order to
overcome past conflicts by developers, the application of the framework is a much more
complex when considering the procedure has to be followed. Also the use of the head
assembly provides mind wave signals generated from the user's brain and provides the
resulting signals as level classified and extracted. Therefore, the application of the framework
has been changed from the device used by the team is able to deal with signals extracted and
classified as well as the strength to open and close his eyes.
Then the problem to be solved is how to use properly signals in order to reach applications,
including games and assistive technology. Through the device that indicates the row data for
care, meditation level user to open and close his eyes. Next, you should check with many
users comparing the results to see if it is suitable to develop advanced applications.
Many developers use BCI devices for their development purposes. But the problem is if you
are capable enough to meet your requirements or not. Until the user to carry the device and
use it, they cannot predict whether that device is best suited for your development purposes.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 4
8/13/2019 IT10 0560 80
13/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
1.3 Research Questions
How are brain waves captured?
What is the age range and duration of brains waves taken from a person?
How are brain waves collected accurately without any error occurred?
How is the accuracy of collected brain waves tested?
How is BCI Device used in suitable Application?
How BCI application is develop to suit anyone at different age ranges and gender?
How is BCI device utilize efficiently?
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 5
8/13/2019 IT10 0560 80
14/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2 CONTENT2.1 Addressing the Literature
In what research has been done so far, the field information technology & BCI technologyhas develop a wealth of computer technology to support the various aspect of BCI
technology, software applications and Hardware applications. Primarily there are two
categories of BCI systems invasive and noninvasive. Invasive systems interact with the
brain directly via electrodes / sensors that are implanted into the brain or its surface. While
noninvasive systems interact with the brain indirectly via electrodes / sensors placed on the
surface of the head that detect brain signal emissions (e.g. Electro-Encephalography (EEG),
functional Magnetic Resonance Imaging (fMRI), and Magnetic Sensor Systems).
Researchers, psychologists, artists, and others have been experimenting with non-invasive
brain-computer interfaces that read brain signals with an electroencephalogram (EEG). EEG
based brain computer interfaces use sensors placed on the head to detect brainwaves and feed
them into a computer as input.
The three major Technologies used in BCIs.
BCIs are categorized according to the users mental activity that is performed to send
commands and messages. There are three main types of mental activities.
- Motor Imagery
- P300
- Steady State Visual Evoked Potential (SSVEP)
Motor Imagery
In Motor Imagery, an individual rehearses or simulates a given action mentally.
An example for the Motor Imagery - A person imagines performing an action, like squeezing
a ball. The EEG data are classified online and the result is graphically presented to the
subject as a horizontal bar on the screen that moves according to the hand movement of the
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 6
8/13/2019 IT10 0560 80
15/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
subject. The bar goes to the right if the right hand was moved or to the left if the left hand
was moved. To optimize the feature, the offline analysis of the data can be supported
P300
The P300 (P3) wave is a measured brain response that is evoked in the process of decision
making.
When recorded by electroencephalography (EEG), it is shown as a latency (a delay between
stimulus and response).
The signal is normally measured most strongly by the electrodes covering the parietal lobe
area of the head.
Steady State Visually Evoked Potentials (SSVEP)
In neurology, Steady State Visually Evoked Potentials (SSVEP) are the signals that are
responses made by brain to visual stimulation at specific frequencies.
As an example when a person gazes at a light or focuses his/hers attention it, the EEG activity
over Occipital Lobe area of the brain will show an increase in power at the corresponding
frequency.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 7
8/13/2019 IT10 0560 80
16/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Comparison of consumer Brain Computer
Figure 1
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 8
8/13/2019 IT10 0560 80
17/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2.2 Methodology2.2.1 OverviewThe brain computer interface technology, not only as a technical aid can be used to bring the
technology to a new level by using the concept of a way to entertain users. As can be used to
develop games and allow users to play the game using their attention and meditation levels or
eyes blink without being restricted to just mouse click. The proposed method previously tried
to transfer some knowledge of the game and BCI communities shared a preliminary
framework to be aware of the investigation of each. Since the team has used the wave head
NeuroSky mind and according to the results found in the testing phase, it could be noted that
the device is not long enough to be able to develop an early implementation. Therefore, the
team decided to implement applications based on assistive technology and entertainment
purpose as the development of games based on the concept of BCI and satisfied with the
device's capabilities.
The overall system objectives are common to all members and we are forced to merge our
individual components at the end to form the final system as a single module. The following
are the main expected outcomes of the research, considering the project as a module. Since
the team has intended to implement a framework that can bridge the gap between the gaming
community and BCI technology were expected results,
A BCI framework that can work on many kinds of BCI devices without any
incompatibility issues
A BCI framework that can be used by developers to make applications
A more accurate and flexible brain wave Feature Extraction technology comparing to
existing BCI Framework
A Game that can be controlled by mind for our demonstration purposes
But since the proposed framework has been changed based on limitations of the device and
conflicts of the developing procedure the team has developed applications as BCI enhanced
Simple Key Board, Keyboard, Snake and Ladders game which are working using the users
eye blink and the Ludo game and a sound generator based on the users attention and the
focus level.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 9
8/13/2019 IT10 0560 80
18/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
In the testing phase of the calculator and the key board it was found that eye blink strength
varies from person to person. Age gender also remarkable factor of the different of eye blink
strength. So the software had to be develop to adjust itself to suit different people According
to their eye blink strength. After testing different people eye blink strength average (mean) of
their strength had to be calculated and using the application.
In the development application the comfort of the eye also much consider because of that
time gap between two eye blink was carefully considered after analyzing previous gather
data. The key board and Simple key Board control by thread. This thread decided the timing
between two eye blink. The user should have enough training to use this application properly.
A Train user can easily operate and save time.
This Simple Key Board and Key Board operate only using eye blink signal. So the functions
of this applications are limited.
2.2.2 Overview of the System Design
Figure 2
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 10
8/13/2019 IT10 0560 80
19/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2.2.3 User CharacteristicsThe users of the system can be categorized under two user groups as follows
Normal user
Disabled user
The characteristics of those identified user groups are as follow
Normal user - Physical fit person
Both Ludo game and Snake & Ladders games are targeted to this use group. Neurosky BCI is
used to get the focus level and eye blink strength of the users. This games are meant to get
new experience and leisure time activities and it also can be used to improve their attention
and focus level.
Disabled user who have no limbs and unable to speak
Both Mind Simple Key Board and Advanced Blinking Key Board targeted to this user group.
Neurosky BCI is used to get only eye blink strength of the users. User can fully control
calculator & Key Board using Neurosky BCI so they can use this application solve
mathematical equation & communicate with other people without any barriers.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 11
8/13/2019 IT10 0560 80
20/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2.2.4 Product functionsThinkGear API
This API is the bridge between the NeuroSky BCI device and the developed
applications. It provides the functions to deal with the row data collected from the users based
on the attention, meditation level and the eye blink strength
BCI enhanced Advanced Blink Keyboard
This application is a keyboard that makes it easy for disable users to control and use the
keyboard to the device based on the blinking eyes instead of using hand. There are some
functions that gives users press the corresponding key board button using only the blinking
eyes with his hand and automatic search function will reduce the time taken to type a word.
BCI enhanced Simple Blink Key Board
This application also is same as the above mentioned keyboard. It also provides the same
functionality to the users with symbols which are represent the day today activities for easy
usage of the users as it allows users to control the options using the eye blink. For calculating
purposes user have to train his/her way of blinking in order to get the correct options.
BCI enhanced Ludo Game
This implementation of the game is for the entertainment of users that provide the
opportunity to play the game with the attention and the focus level. Depending on the user-
level approach is to roll the dice and win the game. Therefore, providing users a great
opportunity to train their level of attention and the focus level, and to keep them at a good
level.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 12
8/13/2019 IT10 0560 80
21/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
BCI enhanced Snake and Ladders Game
This implementation of the game is for the entertainment of users that provide the
opportunity to play the game with his blinking care use and the level of focus. According to
the user to open and close your eyes have to roll the dice and win the game. Therefore,
providing users a great opportunity to play the game in a new way, apart from the traditional
way.
2.2.5 Tools and Technologies
The latest technologies deployed to develop this application the following
Neurosky mindwave BCI Device
Microsoft Visual Studio 2010
.Net Framework 3.5
Neurosky mindwave is used take meditation, attention & eye blinking brain signals
separately from the user.
.Net Framework needs to operate ThinkGearNET API properly. ThinkGearNET API is a C
sharp application. This application classify raw data signal in to meditation, attention & eye
blinking. Efficiency of this application is little bit slowly because it has been written in C
Sharp Programming Language.
C sharp programming language is used to develop Ludo game, Snake and Ladders game,
Mind Controlling Calculator and Mind Controlling Key Board
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 13
8/13/2019 IT10 0560 80
22/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2.2.6 Product ConstraintsNeurosky mindwave detected only limited range of signals because it has only one dry
electrode (sensor)
The output of Neurosky mindwave is not much accurate and its varies from person to person
without any combination.
Neurosky mindwave API is written in C Sharp programming language so the efficiency of
code is not up to the required level and its takes considerable time to process the data of
brain signals.
Sometime BCI device driver conflict with windows default driver if it happen so BCI Driver
should to be manually installed.
Application are little bit slow due to the capabilities of BCI device and its ThinkGearNET
API
Our development environment is Microsoft visual Studio 2010. C Sharp Programming
Language is used to develop Mind controlled Games, Mind controlled Key Board and Mind
Controlling Calculator.
Developers may not be always capable of providing an extreme user friendly Control
functions since that may affect the efficiency of the BCI Device.
This BCI Application Framework only support Windows base platforms
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 14
8/13/2019 IT10 0560 80
23/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2.2.7 Assumptions and DependenciesIt assumed that user should have Neurosky mindwave BCI devise and the computer which
runs BCI Application Framework on the windows XP or higher version of windows
operating system.
The user must have Neurosky mindwave BCI with a fresh battery and device must be turn on.
Then should be connected to ThinkGearNET API. In order to run BCI Application
Framework successfully, the environment should be satisfied below mentioned conditions.
Microsoft Windows based supported architecture should be available and implemented
x86 x64 USB 2.0 or higher version port should be available Windows XP service pack 2 or higher version .Net Framework 3.5 or higher version available.
Neurosky mindwave drivers should be installed in the computer
Neurosky mindwave Bluetooth dongle should be connected to the computer
Above mentioned hardware and software requirements should be satisfied and integrated
within computer system to perform BCI Application Framework.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 15
8/13/2019 IT10 0560 80
24/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
2.3 Research FindingsSince the proposed framework has been changed by the implementation owing to some
conflicts the team was decided to develop applications which are compatible with the device.
In the framework the proposed functions are to be developed has been already facilitate by
the device. Therefore the signal classification is already done by the device and it generates
the outputs and gives the signals as alpha, beta and gamma ranges. So being struggle with
implementing the framework the team decided to implement the applications which are
simply compatible with the BCI mindwave headset.
In the next step the biggest problem was for the team is whether the given values of the
device as output is reliable or is to the point of accuracy or not. Therefore the team conductedmany tastings using many various users (at the initial stage the students of the SLIIT
including the team members). For the each and every user we have given the same time of
period as the testing time and first we collected the raw data which are generating by the
users brain according to the given behavior. By giving a paragraph of a book which is not
read by the users previously we have counted the attention level of the each and every user.
Then according to the raw data counted the mean value of the data for and each and every
user and generated the graphs. When comparing the final results the fact could be understand
is the variation of the data is not in a considerable level as the attention level of some users
are in a high level of value and some are in low level.
When it starts from a high level of value then it spreads up and down in a range and it became
for an average level. But the values are not stable and could not find any certain pattern
which can be generic for all users. If the users lose their focus level owing to any kind of
disturbance then the kept level of the attention level rapidly changed for a low value and we
could notice a wide difference between two values. Therefore the achieved values cannot be
guaranteed whether they would be usable for an advance application development.
Next aim was to collect the raw data regarding the attention level using background music.
For that purpose even we followed the same method as mentioned earlier. We played
background music for a certain time of period and let users to listen for the playing music and
collected the data. Finally when analyzing the results the attention level of some users was in
a considerably good level and some users were the worst. Also one point we could notice is
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 16
8/13/2019 IT10 0560 80
25/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
the meditation level of the users when listening to the music were at a good level of value and
spread to a certain range of value.
In that experiment even the team has to decide that the given values by the device are not
much reliable and in an accurate level as depending on the way of responses by the users for
an activity and also the device is not much compatible in giving wide range of brain signal
values since it has only one sensor which can be capture only from the users forehead.
When considering about the users desires on playing games also the team has collected the
data regarding the attention and the focus level. In that situation even some of the users
performed well and responded to the experiment. But some users who are novel for playing
games were not highly responsive for the brain activity.
Another function of the device is generates signal regarding the eye blink of the users. In that
situation even the strength of the blinking speed is vary from person to person. Therefore
when testing that value even w could realize that the generating values are changing time to
time and we have to define an average value for the blink when using it for controlling an
application as a game. Because when users get tired the strength and the speed of the blinking
can be degraded to a low level. Therefore relying on the eye blink even cannot go for
developing an advance application as well as the capabilities of the device.
Therefore when conducting the research the team could realize that the currently using BCI
mindwave headset is not much reliable in given signal values which can be usable in
implementing advance applications. The capabilities of the device as it does not contain many
sensors which can be absorbable in brain signals by many areas of the head restricted the
developers in developing advance applications. As a major fact by using this device the
developers cannot implement fully functional advance games or any other application.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 17
8/13/2019 IT10 0560 80
26/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 3
Figure 4
0
20
40
60
80
100
120
128
55
82
109
136
163
190
217
244
271
298
325
352
379
406
433
460
487
514
541
568
595
622
649
676
703
730
757
784
811
838
865
892
919
946
Attention Level Values
0
10
20
30
40
50
60
70
80
90
127
53
79
105
131
157
183
209
235
261
287
313
339
365
391
417
443
469
495
521
547
573
599
625
651
677
703
729
755
781
807
833
859
885
911
937
Running Average of Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 18
8/13/2019 IT10 0560 80
27/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 5
Figure 6
0
5
10
15
20
25
30
127
53
79
105
131
157
183
209
235
261
287
313
339
365
391
417
443
469
495
521
547
573
599
625
651
677
703
729
755
781
807
833
859
885
911
Variance Between Attention Level Values
0
10
20
30
40
50
6070
80
90
110
19
28
37
46
55
64
73
82
91
100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
316
325
Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 19
8/13/2019 IT10 0560 80
28/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 7
Figure 8
0
10
20
30
40
50
60
70
80
110
19
28
37
46
55
64
73
82
91
100
109
118
127
136
145
154
163
172
181
190
199
208
217
226
235
244
253
262
271
280
289
298
307
Running Average of Attention Level Values
0
5
10
15
20
25
1 917
25
33
41
49
57
65
73
81
89
97
105
113
121
129
137
145
153
161
169
177
185
193
201
209
217
225
233
241
249
257
265
273
281
289
Variance Between Attention Level Values
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 20
8/13/2019 IT10 0560 80
29/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 9
0
50
100
150
200
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65
Eye Blink Strength
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 21
8/13/2019 IT10 0560 80
30/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
3 RESULT AND DISCUSSION
Figure 10
Main interfaces of the system
This interface contain two options
Simple Mode Blink advanced mode
Using this interface user can select Simple Blink Key Board and the Advanced Blink Key
Board
Two button of the main interface highlight one after the other then the user can select the
relevant button by blinking his eyes when the relevant button is highlighted. The main
fracture of this applications are can be used by any person who have different level of eye
blink strength without any hesitation.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 22
8/13/2019 IT10 0560 80
31/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Interface of Simple Blink Keyboard
This application has been design for disabled users to communicate very easily without
bothering English language and the users with symbols which are represent the day today
activities for easy usage of the users as it allows users to control the options using the eye
blink. The NeuroSky device detect the brain signal of eye blink. These eye blink is used to
control button of the Simple Blink Keyboard.
First user has to select the relevant row by blinking user eye when its highlighted. (Wen the
Simple Blink Keyboard is open button row of it are automatically highlighted from bottom to
top)
After that the button of the selected row highlighted from the left right
Blinking eyes user has to select the correct button when it is highlighted.
Figure 11
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 23
8/13/2019 IT10 0560 80
32/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
As explain above disabled use can use the Simple Blink Keyboard by selecting rows from left
Eg :- l2- First user has to select relevant row of buttons by blinking users eye
Figure 12
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 24
8/13/2019 IT10 0560 80
33/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Then user select relevant button of the selected row by blinking users eye
Figure 13
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 25
8/13/2019 IT10 0560 80
34/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Then user select relevant button of the selected row by blinking users eye. After selecting the
particular symbol it will pronounce the idea relevant to for the symbol.
Figure 14
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 26
8/13/2019 IT10 0560 80
35/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 15
Interface of Advance Blink Key Board
This application has been design for disabled users to communicate with the world without
any communication barriers. The Neurosky device detect the brain signal of eye blink. These
eye blink is used to control button of the Blink Key Board.
First user has to select the relevant row by blinking user eye when its highlighted. (Wen the
Key Board is open button row of it are automatically highlighted from bottom to top)
After that the button of the selected row highlighted from the left right
Blinking eyes user has to select the correct button when it is highlighted.
After typing all the word of the text SAY IT button should be selected then what you have
type is pronounced.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 27
8/13/2019 IT10 0560 80
36/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 16
First type the text that are to be pronounced
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 28
8/13/2019 IT10 0560 80
37/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Figure 17
After selecting the relevant button it will automatically search for the matching word
beginning with the sleeted letter
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 29
8/13/2019 IT10 0560 80
38/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
Then select SAY IT button and it will pronounce the text.
Figure 18
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 30
8/13/2019 IT10 0560 80
39/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
4 CONCLUSIONFor a long time, researchers have been working on a marriage of human and machine that
sounds like something out of science fiction: a brain computer interface. The technologyholds great promise for people who cant use their arms or hands normally because they have
had spinal cord injuries or suffer from conditions such as amyotrophic lateral sclerosis (ALS)
or cerebral palsy. BCI could help them control computers, wheelchairs, televisions, or other
devices with brain activity.
To success with the projects like above mentioned, the capabilities of the used BCI devices
should be in a high standard level with the fully functional options. Also there should be
many sensors which are capable of capturing brain signal within vast areas of the brain.
Since we are using the NeuroSky mindwave head set which is contain only one sensor is not
much capable of developing advance systems and applications. And the generated raw data
also not much accurate and reliable as it can be used for a considerable application since it
does not gives a stable value and cannot find a certain pattern in order to use as an average
value for controlling an application
Therefore the team decided to implement applications depending on the capabilities of the
using BCI mindwave headset. So the implemented applications are Mind controlling
Calculator and a Keyboard which is working based on the users eye blink. Also a Ludo Game
controlling based on the users attention level and the focus level as well as a Snakes and
Ladders game based on the eye blink. The sound generator is an application working based
on the alpha, beta and gamma brain signals.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 31
8/13/2019 IT10 0560 80
40/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
References
[1] "Brain-Computer Interfaces: Beyond Medical Applications," 2013. [Online]. Available:
http://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-
applications.
[2] E. Kader, B. P. B. P. Emilie Belley, J.-A. Filion, A. Nutter, M. Parent-Vachon, M.
Saulnier, B. P. Stephanie Shedleur, B. P. Tsz Ting Wan, B. B. Elissa Sitcoff and P. O. Nicol
Korner-Bitensky, "MOTOR IMAGERY - Information for Patients and Families," 2010.
[Online]. Available:http://strokengine.ca/intervention/admin/patient/Motor%20Imagery-
Family%20InformationDec2010.pdf.
[3] "P300 (neuroscience)," 2012. [Online]. Available:
http://en.wikipedia.org/wiki/P300_(neuroscience). [Accessed 2013].
[4] K. Nakayama and M. Mackeben, "Steady State Visual Evoked Potentials In the Alert
Primate," 1982. [Online]. Available:
http://visionlab.harvard.edu/members/ken/Ken%20papers%20for%20web%20page/027Visio
nRes82.pdf.
[5] "Event-related Potential: An overview," 2011. [Online]. Available:
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3016705/. [Accessed 2013].
[6] "Electroencephalogram (EEG)," WebMD, LLC., 2010. [Online]. Available:
http://www.webmd.com/epilepsy/electroencephalogram-eeg-21508.
[7] "What Is A Framework?," 2003. [Online]. Available:
http://www.codeproject.com/Articles/5381/What-Is-A-Framework.
SRI LANKA INSTITUTE OF INFORMATION TECHNOLOGY 32
http://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applicationshttp://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applicationshttp://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applicationshttp://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdfhttp://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdfhttp://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdfhttp://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdfhttp://www.codeproject.com/Articles/5381/What-Is-A-Frameworkhttp://www.codeproject.com/Articles/5381/What-Is-A-Frameworkhttp://www.codeproject.com/Articles/5381/What-Is-A-Frameworkhttp://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdfhttp://strokengine.ca/intervention/admin/patient/Motor%20Imagery-Family%20InformationDec2010.pdfhttp://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applicationshttp://lifesciences.ieee.org/articles/114-brain-computer-interfaces-beyond-medical-applications8/13/2019 IT10 0560 80
41/41
BRAIN COMPUTER INTERFACE APPLICATION FRAMEWORK
[8] "Neurosky MindWave Mobile," Neurosky, 2012. [Online]. Available:
http://neurosky.com/Products/MindWaveMobile.aspx.
[9] Microsoft, "Microsoft Visual Studio," Microsoft, 2013. [Online]. Available:
http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012.
[10] S. Du and M. Vuskovic, "Temporal vs. Spectral Approach to Feature Extraction,"
[Online]. Available:
http://medusa.sdsu.edu/Robotics/Neuromuscular/Our_Publications/FE_Sijiang_press.pdf.
[11] J. B. Ochoa, "EEG Signal Classification for Brain," 2002.
http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012http://www.microsoft.com/visualstudio/eng/products/visual-studio-ultimate-2012Recommended