BCI Systems
Brendan Allison, Ph.D.Institute for AutomationUniversity of Bremen29 January, 2008
2
A BCI enables communication without movement.
Some patients cannot use any interface requiring movement.
What is a BCI?
3
What is a BCI?
4
Most BCIs translate your brain’s electrical activity (EEGs) into messages or commands.
Performing mental tasks produces electrical activity detectable with electrode caps.
What is a BCI?
5
When a neuron is active, its voltage may change by 100 mV or more.
Electrical activity in a single neuron.
How do EEGs work?Neural communication
produces electrical activity.
6
This activity may be detectable to electrodes on the scalp.
How do EEGs work?
Conventional electrode caps from EGI, Neuroscan, and Electro-Cap.
7
Newer EEG recording systems:
• Require less or no prep time and skill• Require less or no gel• Require fewer electrodes• Are more portable• Handle artifacts better• Are wireless• Are cheaper
How do EEGs work?
Field recording systems from Quasar, Advanced Brain Monitoring, and Pineda et al (2003).
8
QUASAR Hybrid or eIBE sensors
Audio Headset EOG/EEG GlassesQUASAR IBE Electrodes
How do EEGs work?
9
Emotiv and NeuroSky systems
How do EEGs work?
10
Invasive BCIs record activity from electrodes under the scalp.
Pyramidal neurons (Kandel et al., 1988)Electrocorticogram (ECoG) recording montage in a human patient. (Leuthardt et al., 2004)
How do EEGs work?
11
Other functional imaging approaches:
• Functional Magnetic Resonance Imaging (fMRI)• Magnetoencephalography (MEG)• Positron Emission Tomography (PET)• Functional Near Infrared (fNIR)
How do EEGs work?
A functional MRI An MEG machine Functional Near Infrared
12
BCIs cannot read minds or literally interpret mental activity.
helloyes
pain
What isn’t a BCI?
13
BCIs are a popular topic in sci fi!
The Matrices Firefox Strange Days Neuromancer
JohnnyMnemonic
Star Trek: The Cage
Dreamscape The X – Mens
What isn’t a BCI?
14
More bci fi ….
Total Recall Minority ReportBrainstorm The Lawnmower Men
The Govenator!!
What isn’t a BCI?
15
BCIs cannot:
• Read your thoughts.
What isn’t a BCI?
16
BCIs cannot:
• Operate without your knowledge or free will.
What isn’t a BCI?
17
BCIs cannot:
• Write to the brain.
What isn’t a BCI?
18
BCIs cannot:
• Repair injured areas.
What isn’t a BCI?
19
These similar approaches are not BCIs:
• Biofeedback• Prosthetics• Retinal or cochlear implants• Medical EEGs• EEG or fMRI Lie Detection• Neuromarketing• Employee screening• Attention or fatigue monitors
What isn’t a BCI?
20
BCIs rely on voluntary mental activities such as:
Imagined movement
What is a BCI?
Torrey Pines golf course
21
BCIs rely on voluntary mental activities such as:
Emotional imagery
What is a BCI?
22
BCIs rely on voluntary mental activities such as:
Selective attention(SSVEP)
What is a BCI?
23
BCIs rely on voluntary mental activities such as:
Selective attention(P300)
What is a BCI?
24
BCIs rely on voluntary mental activities such as:
Certain mental tasks6
X 942
Object rotation Math Singing
What is a BCI?
25
What is a BCI?
26
How do BCIs work?
o General Schematico P300 BCIo Mu BCIo Other BCIs
Components
27
A BCI requires the following:
- At least 2 electrodes - An amplifier designed for EEGs- A mediocre personal computer- An A/D card in the computer- Software
Components
28
All BCIs have at least four components:
1) Signal Acquisition
2) Feature Extraction
3) Translation Algorithm
4) Operating Environment
Components
The Four BCI Components(Wolpaw et al., 2002; Allison et al., 2007)
29
Acquisition: The user performs a task that produces a distinct EEG signature for that BCI
Extraction: Salient features are extracted from the EEG
Translation: A pattern classification system uses these EEG features to determine which task the user performed
Environment: The BCI presents feedback to the user, and forms a message or command
Components
30
Motor imagery: ERD
31
Many sources create noise.
Electrode cap Raw EEGs from a field setting
Acquisition
32
Acquisition
A headband used for a 1D mu BCI (Pineda et al., 2003) Several brain areas responsible for movement.
33
EnvironmentUser task:
Imagine movement to move the cursor down.
Relax to move it up.
A mu BCI using BCI2000 (Schalk et al., 2004)
34
Extraction, Translation
Mu activity used for one dimensional control (Wolpaw et al., 1991)
35
Virtual Navigation
36
Recording from a neuron (Kandel et al., 1988) Top: Utah intracranial electrode arrayBottom: Cone electrode
Cortical Neurons
37
A severely disabled subject controls a switch (Kennedy et al., 2004)
A patient with an implanted BCI (Cyberkinetics, Inc.)
Cortical Neurons
38
Cortical Neurons
ECog activity in several human patients during hand or tongue movement (Miller, in press)
39
The P300 only occurs after target flashes (Allison and Pineda, 2003).
Selective attention: P300
EEGs from a P300 BCI
-4
-2
0
2
4
6
0 100 200 300 400 500 600 700
time after flash begins (ms)vo
ltage
(uV)
NontargetsTargets
40
EEGs from a P300 BCI
-4
-2
0
2
4
6
0 100 200 300 400 500 600 700
time after flash begins (ms)
volta
ge (u
V)
NontargetsTargets
The P300 only occurs after target flashes (Allison, 2003).It is often largest over site Cz or Pz (central or parietal areas).
Acquisition, Extraction
41
K
Translation
Site Fz
-4-20
2468
1012
141618
0 50 100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
Averaged data from a P300 BCI at the NextFest Expo (Allison, 2004)
42
Selective attention: SSVEP
Herrmann et al, Exp. Brain Research 2001
43
Steady state visual evoked potential (SSVEP) BCI (Kelly et al., 2005)
SSVEP
6 Hz 15 Hz
44
6 Hz
15 HzA different SSVEP BCI (Allison et al., 2006)
SSVEP
45
Data from an SSVEP BCI using 6 and 15 Hz checkerboxes (Allison et al., 2008)
SSVEP
46
SSVEP – GT BrainLab
47
SSVEP – IAT BremenSSVEP spelling system
13 14 15 16 17
48
SSVEP – IAT Bremen
1
2
3
4
5
1 2 3 4 5
MATRIX RHOMBUS
49
SSVEP – IAT Bremen
50
0 5 10 15 20 250
1
2
0
1
2
Spec
tral
den
sity
5 Hz
10 Hz
a) Stimulation frequency: 5 Hz
0 5 10 15 20 250
1
2
0
1
2
Spec
tral
den
sity
7 Hz14 Hz
b) Stimulation frequency: 7 Hz
0 5 10 15 20 250
1
2
0
1
2
Frequency (Hz)
Spec
tral
den
sity 9 Hz
18 Hz
c) Stimulation frequency: 9 Hz
SSVEP – IAT Bremen
51
Row-Column layout- 44 commands -
Rhombus layout- 58 commands -
Average Average
3:40 4:25
99 %
30.6
98 %
28.4
Best Best
Time [min] 2:27 3:01
100 %
43
Accuracy 100 %
Bits/min 40
Nine subjects spelled
‘BRAIN COMPUTER INTERFACE.’
SSVEP – IAT Bremen
52
SSVEP – IAT Bremen
53
SSVEP – IAT Bremen
54
SSVEP – IAT Bremen
55
•Information throughput •BCI record = 9 char/min•Typing record = 750 char/min
•User needs, preferences, and abilities•Vocabulary and customizability•Equipment required
•cost•portability•invasiveness•support availability•appearance
•Task demands, ease of use and “distraction quotient”•Training time and parameters•Noise resistance
Comparing BCIs
56
“Future progress hinges on … recognition that BCI development is an interdisciplinary problem, involving neurobiology, psychology, engineering, mathematics, computer science, and clinical rehabilitation…” – Wolpaw et al., 2002.
Communications, linguistics, HCI, and human factors are also important.
Future directions
57
Replacing conventional interfaces for conventional users in conventional settings.
Stupid future directions
The future of BCIs is not Keanu Reeves.
58
New user groups
Less disabled usersRehabilitation: stroke, autism, attentionHealthy users
GamersSurgeons, drivers, soldiers, mechanicsLazy people
Future directions
59
Replacing conventional interfaces for disabledusers in conventional settings.
Replacing conventional interfaces for conventional users in specific environments.
Supplementing conventional interfaces.
Future directions
60
Why supplement conventional interfaces?More bandwidthCognitive overloadPrivacyConvey otherwise unavailable informationNovelty, coolness, and public perception
Future directions
Plantronics gamer headset. Headset with dry electrodes.
61
Game applications
Future directions
62
Future directions
63
Thanks to Chris Agocs, Jacqueline Boccanfuso, Ben Chi, Adriane Davis, Umang Dua, Bernhard Graimann, Axel Graeser, David Leland, ThorstenLueth, Luke McCampbell, Dennis McFarland, Melody Moore Jackson, Jaime Pineda, John Polich, Samir Ramji, Gerv Schalk, and Diana Valbuena for help with work presented here.
Thanks to my colleagues in the BCI community!
Attendees of the BCI conference in New York
Acknowledgements
64
www.bci-info.org
http://www.cis.gsu.edu/brainlab/bci.ucsd.eduwww.cyberkineticsinc.comwww.neuralsignals.com
Additional videos are available from the Wolpaw lab: www.bciresearch.org
More videos from the Pfurtscheller lab:http://bci.tugraz.at/movies.html
Other sites are easy to find online. Please contact the author for specific references for articles or other materials:
Websites with more info
65
Recommended
