A N E N E M Y I N O U R M I D S T

UNIVERSITY OF CHICAGO SUMMER 2006 BIOLOGICAL SCIENCES DIVISION

MIND OVER MATTER: MOVING OBJECTS

WITH YOUR MIND HAS LONG BEEN

FODDER FOR SCIENCE FICTION STORIES.

NOW, RESEARCH IS TURNING IT

INTO REALITY.By Kelli Whitlock Burton

MiNdMatter

icholas Hatsopoulos adjusts

the volume on his computer and looks

around. “Hear that?” he asks. There’s

a steady crackling noise coming from

the speakers, like hail on a tin roof.

Hatsopoulos smiles. The crackling

chorus is as satisfying to his ear as a

good guitar riff by Jimi Hendrix, one

of his favorite musicians.

What seems like a hail storm actually

is the sound of neurons firing inside

the motor cortex, the part of the brain

responsible for movement—a symphony

that just 15 years ago Hatsopoulos

could only dream of hearing, much

less composing.

Listening to the brain is no small

feat, and if that were the brightest note

in this little concert, it would be worth

an ovation. But Hatsopoulos, an assistant

professor of organismal biology and

anatomy at the University of Chicago,

can do more than eavesdrop. Along

with colleagues at Brown University,

he has developed a way to record signals

sent out by large groups of neurons—

commands telling the body how and

where to move—and to translate the

orders into a language a computer

understands and acts on.

The technology is called a brain-

computer interface—BCI for short—and

it’s not a new phenomenon. Scientists

have tinkered with BCI since the 1970s,

but it’s only in the past decade that the

technology’s true potential has been

realized. The main thrust today is devel-

oping BCI systems to aid people who are

paralyzed by injury or illness. While these

patients’ limbs may be stilled, studies show

that the motor cortex is not. Hatsopoulos’

team is one of only about half a dozen

university research groups working on

the problem in the United States.

Ten years ago, Hatsopoulos and

John Donoghue, his former postdoctoral

advisor at Brown University, became the

first scientists to teach monkeys how to

move a computer cursor with their minds.

Two years ago, they taught a person to do

it—a quadriplegic who was able to turn

on a television, check e-mail and wiggle

the fingers of an artificial hand, all with

his thoughts alone. The patient is part

of an FDA clinical trial of the BrainGate™

system, the product of a company

Donoghue and Hatsopoulos launched

in 2000.

In a nutshell, the researchers have

found a way to turn thought into action—

without moving a muscle.

WHEN SCIENCE TICKLES

If his dark curly hair, deep brown eyes and

distinctive surname aren’t clue enough to

his ancestry, the strains of Mediterranean

music coming from his office are a dead

giveaway. Hatsopoulos’ parents grew up in

Athens, Greece, but didn’t meet until the

N

19

Pho

to b

y D

an D

ry

Chicago researcher Nicholas Hatsopoulos

Medic

ineo

nth

eM

idw

ay

Sum

mer

200

6

20

early 1950s after each traveled separately

to Boston to attend college, his father at

MIT and his mother at Wellesley. With a

mechanical engineer and physicist for a

dad as well as a mathematician for a

mother, Hatsopoulos was surrounded

by science while he grew up in Lincoln,

Mass., just outside Boston. He even chose

to major in physics at Williams College.

Still, when it came time to make plans

after graduation, he wasn’t quite ready

to launch a scientific career. Instead,

Hatsopoulos decided to teach physics

and computers for a year in Greece.

When he wasn’t in class, he dabbled

with the bouzouki, an instrument similar

to a mandolin. He even toyed with the

idea of staying in his parents’ homeland

to become a professional bouzouki player.

But something about science tickled his

subconscious, so he returned to Boston

and took a job as a research assistant to a

Harvard mathematical psychologist. Soon

he enrolled in the psychology master’s

program at Brown. By 1992 he’d earned

a PhD in cognitive science. Skilled in

physics, psychology, cognition and the

bouzouki, Hatsopoulos headed to Caltech

for a postdoc with a scientist who studied

insects’ brains.

There, Hatsopoulos made his first

recording of a neuron—a single cell in a

locust’s brain. “As soon as I recorded my

first cell and saw the electrical spikes on the

oscilloscope and listened to the crackle of a

neuron spiking, I was hooked,” he said.

Hatsopoulos’ appreciation for science

is much like his love of music. While the

individual instruments are beautiful to

hear, it’s the combination of their sounds

that Hatsopoulos enjoys most. Similarly,

the actions of one brain cell may be

interesting, but the possibilities that

abound when cells work together are

downright fascinating.

Take the bouzouki, for example. “I

have always been enamored by the finger

work these musicians do and how they

coordinate their fingers that way,”

Hatsopoulos mused. How do the millions

of neurons in the motor cortex work

together to allow someone to strum sweet

music on a bouzouki?

“As soon as I recorded

my first cell and saw

the electrical spikes on

the oscilloscope and

listened to the crackle

of a neuron spiking, I

was hooked.”

Pho

to c

ourt

esy

of N

icho

las

Hat

sopo

ulos

The silicone chip—part of the brain-computer interface—

that’s implanted in the brain’s motor cortex. Photocourtesy of Nicholas Hatsopoulos

“We were looking at a group of neurons that

make up the neural ensemble that is responsible

for everything that we do as thinking beings.”

21

“What really turns me on in this

research is understanding how these large

groups of cells result in motor behavior

or thought,” Hatsopoulos said. Studying

single cells, as he was doing at Caltech,

would not help him find the answer.

So in 1995, Hatsopoulos returned

to Brown for a second postdoc with

Donoghue. The neuroscientist was bent

on figuring out how to record as many

neurons in the motor cortex as possible.

Hatsopoulos was more than happy to help.

THE RECORDINGINDUSTRY

Scientists had developed a way to listen

in on single brain cells years before

Donoghue began his studies. Many tech-

niques were employed, but one of the

most successful was a mechanism created

by Philip Kennedy, a scientist at Georgia

State University, who launched the first

human clinical trial of an implanted BCI

device. Consisting of glass cones with

two microelectrodes designed to pick up

neural impulses in the brain, the device

was implanted in the motor cortex of a

paralyzed patient. The woman was asked

to think about moving a computer cursor.

The device captured the cell signals and,

using a radio transmitter under the scalp,

sent them to a computer that decoded

them and moved the cursor.

But Kennedy could only record data

from one or two neurons with his device;

Donoghue’s sights were set much higher.

He wanted to capture signals from dozens,

perhaps hundreds, of brain cells. Monitor-

ing more neurons would provide a clearer

picture of brain activity, Donoghue believed.

All this and the researchers weren’t

even sure the device would collect the

multi-cell data they needed. They trained

a monkey to move a computer cursor by

maneuvering a joystick with its arm, set

up their system, and watched.

The array not only worked, it func-

tioned better than the scientists had

hoped. In 1996, the team recorded signals

from multiple brain cells in a monkey—

the first time anyone had collected data

from the monkey using this array. “I

remember that moment as really exciting

from more of a scientific point of view,”

Hatsopoulos said. “We were looking at

the brain on a more global level, looking

at a group of neurons that make up the

neural ensemble that is responsible for

everything that we do as thinking beings.”

The next step was to identify patterns

in neuronal activity related to movement.

Hatsopoulos’ job was to create algorithms

that translated the chatter between the

Things started to come together when

Donoghue met Richard Normann from

the University of Utah. Normann had

developed a sensor array—a silicone chip

about the size of a breath mint with 100

tiny electrodes, each capable of recording

neuronal impulses.

Donoghue and Hatsopoulos modified

the array, which had only been used in

cats and Petri dishes, for their studies

with monkeys. Each electrode on the

chip, which is implanted on the surface

of the motor cortex, picks up signals

from nearby neurons. Those impulses are

carried by gold wires that connect the

chip to a titanium pedestal that protrudes

about an inch from the monkey’s scalp.

A cord extends from the top of the

pedestal to a device that amplifies the

signals. A fiber-optic cable hooks the

amplifier to an acquisition system, which

captures the neural impulses and sends

them to a computer.

A titanium pedestal connects to the implanted chip.

Pho

to c

ourt

esy

of N

icho

las

Hat

sopo

ulos

22

neurons in the motor cortex into a

language the computer could understand

—language that conveys information

about movement.

That required Hatsopoulos to study the

monkey’s brain activity on a monitor that

displayed graphs of the neurons in action.

When active, neurons fire between 20 and

200 times a second, creating the spikes

on his computer. Hatsopoulos matched

neuronal signal patterns with each arm

movement. A computer program outfitted

with his algorithms could recognize those

patterns and move the cursor before the

monkey moved a muscle.

Eventually, the monkey figured out that

it didn’t have to move the joystick to make

the cursor move. These were exciting

developments for the researchers, but

there were annoyances, too. Hatsopoulos

couldn’t ask a monkey questions about its

thoughts or actions. The monkey couldn’t

provide any feedback. The researchers’

excitement soon grew to frustration as

they realized they’d taken the technology

as far as they could in animals.

In spring of 2000, Hatsopoulos and

Donoghue discussed their work’s potential

to help people. And then, Hatsopoulos

offered an idea that he never thought

would interest him. “What if we form

a company to do this?”

A RELUCTANTENTREPRENEUR

His father’s dream had been to come to

America and launch his own business. In

1956, the elder Hatsopoulos did, creating

a company to market a device he’d devel-

oped at MIT that converted heat energy

into electricity with no moving parts.

Entrepreneurship was not a dream

Hatsopoulos shared with his father. He

just wanted to be a scientist. But the work

he’d done with Donoghue changed all

that. Cyberkinetics was incorporated in

2001. The following January, Hatsopoulos

joined the faculty at Chicago and later

that year, the company merged with

Bionics, a business started by the Utah

scientist who’d developed the array

Hatsopoulos’ group modified for primates.

In August 2002, the company received

$5 million from a venture capital firm in

Boston and applied for FDA approval to

conduct a small clinical trial of their BCI

technology, which they named BrainGate.

“People have always believed this would

be possible,” said Tim Surgenor, chief

executive officer for Cyberkinetics. “We’ve

been reading about it in comic books

and science fiction our whole lives and

suddenly, someone comes along and says

it’s possible.”

A computer program

outfitted with

Hatsopoulos’

algorithms could

recognize neuronal

signal patterns in the

monkey’s brain activity

and move the cursor

before the monkey

moved a muscle.

The plan, Hatsopoulos said, wasn’t

just to create something that allowed

paraplegics to control electrical devices

with their minds. To set themselves apart

from similar entrepreneurial efforts in the

BCI field, they needed another “killer app.”

“We eventually came up with an idea

that is implicitly the mission of our

company: We want to provide the basic

operating system by which any brain-

computer interface system can work,” he

said. “We want to be the Microsoft of

communications between the brain and

the outside world.”

With a basic operating platform, tech-

nology could be developed to help people

with paralysis answer a phone, type a

letter, turn off a coffee maker—all at the

speed of thought. And scientists could

build on Cyberkinetics’ operating system

to develop therapeutic devices for neuro-

logical disorders: For example, a chip

implanted in the brain of an epileptic

that senses when a seizure is likely and

emits an electric jolt to stop it.

It took a year and five cases of supportive

documents to prepare the FDA application,

but by the middle of 2004, Cyberkinetics

received the go-ahead. The plan was to

enroll up to five paraplegics for a one-year

trial, implant the device, and teach them

to move a computer cursor with their

minds, a small step toward giving those

without the power to move the ability to

turn thought into motion.

“This technology is a whole new way

of dealing with neural repair,” Donoghue

said. “Instead of finding a substitute signal

for movement—like using the eyes to move

a cursor instead of a hand—Cyberkinetics

is trying to develop a system that takes the

brain’s own movement commands to the

outside world.”

ON TRIAL

Cyberkinetics received its first volunteer

for the trial in June 2004: Matthew Nagle,

a 25-year-old from Weymouth, Mass.,

whose spine had been severed at the neck

in a knife attack five years earlier.

Nagle underwent a five-hour surgery, in

which physicians made an 8-inch incision

in the scalp and bone, exposing the motor

cortex. They placed the silicone chip on the

brain surface with the electrodes penetrating

the cortex about 1 millimeter down. The

pedestal was positioned and the scalp

closed around it, leaving about 1 inch

protruding from the top of Nagle’s head.

Three times a week, Nagle was visited

by Maryam Saleh, a technician who used

to work with Cyberkinetics and now is a

doctoral student in Hatsopoulos’ lab. Saleh’s

first task was to teach Nagle to think

about moving. Before he was paralyzed, if

Nagle wanted to move a computer cursor,

he reached out his arm, grabbed a mouse,

and moved it. Without use of his

limbs, however, he must rely on

his mind. But how do you teach

someone to “think” about moving?

23

THE BRAINS BEHIND BRAINGATE

1 The person thinks about moving thecomputer cursor. Electrodes on a siliconechip implanted into the person’s braindetect neural activity from an array of neural impulses in the brain’s motor cortex.

2 The impulses transfer from the chip to a pedestal protruding from the scalp through connection wires.

3 The pedestal filters out unwanted signals or noise, then transfers the signal to anamplifier.

4 The signal is captured by an acquisitionsystem and is sent through a fiber-opticcable to a computer. The computer thentranslates the signal into action, causing the cursor to move.

Cyberkinetics is trying

to develop a system

that takes the brain’s

own movement

commands to the

outside world.

COMPUTER SYSTEM

PEDESTAL

SILICONE CHIP

AMPLIFIER

FIBER-OPTIC CABLE

ACQUISITION SYSTEM

Illus

trat

ion

by C

hris

tina

Ullm

an

24

Medic

ineo

nth

eM

idw

ay

Sum

mer

200

6

“Training patients to move things

with their minds is difficult and different

with each patient,” Saleh said. She had

Nagle visualize himself moving the cursor,

a simple-sounding command that took

him months to master. Then, during one

of their all-day sessions, Nagle did it.

He thought about moving the cursor—

and it moved.

“He was so excited, he told everybody,”

said Saleh, who now is working with a

patient in Chicago who enrolled in the

trial last year. “Most people involved in

the study think of themselves as pioneers.

Even though they’re told that this is

specifically geared toward controlling a

computer cursor, they see the prospects for

future applications. That’s why they do it.”

Indeed, study participants are told at

the outset that the trial’s goal is to help

scientists learn how to improve the

BrainGate system and how to train future

patients to use it. The system won’t

operate without a technician’s assistance

and when the yearlong trial is over, the

entire apparatus—pedestal, chip and all—

is removed.

“This isn’t being done for the patient’s

benefit; it’s being done for mankind’s

benefit,” said Richard Penn, professor of

neurosurgery at Chicago who operated on

another study participant with whom

Saleh now works.

“What’s so intriguing about this is

that we’re really listening in on what

neurons are doing,” Penn said. “This is

the strangest, most interesting surgery

I’ve ever done in that sense. Not the

technical stuff, but the data that we get

from the neurons firing in different

patterns when you’re thinking in different

ways. And seeing it is only the beginning.”

Researchers presented findings from

Nagle’s trial, which ended in 2005, at the

Society for Neuroscience annual meeting

later that year. Although the results are

preliminary, they suggest the BrainGate

system works.

So far, four volunteers have joined the

trial: two with spinal cord injuries, one

with amyotrophic lateral sclerosis (ALS,

also called Lou Gehrig’s disease) and a

stroke patient.

“Depending on their specific situation

and their specific interest, [people with

disabilities] may be interested in computers

turning text into speech, dialing telephones,

playing games, controlling their environ-

ment, moving their wheelchairs and

moving their arms,” Surgenor said. “For

each of these four participants, I think

each one is interested in something

different. We have to develop something

that’s flexible enough to work with a

number of different devices.”

SET IT AND FORGET IT!

An unabashed fan of infomercials,

Hatsopoulos draws his philosophy about

this project from a jingle he heard on a

promotion for a rotisserie oven. Baking a

chicken or roast in a conventional oven is

hard, messy work that requires constant

oversight, the infomercial host proclaimed.

Study participants are told at the outset that the

trial’s goal is to help scientists learn how to improve

the BrainGate system and how to train future

patients to use it.

NORMAL NEURAL ACTIVITY

CELL BODY OF MOTORNEURONSignals sent throughdendrites cause chemicalchanges that result in an electrical signal in thecell body.

AXONSNerve impulses are carried through axons away from the neuron’s cell body.

NEUROMUSCULAR JUNCTIONThe signal is passed by neuro-transmitters from synaptic bulbs onthe neuron to muscle fibers. Themuscle fibers then react to the signal.

Illus

trat

ion

by C

hris

tina

Ullm

an

DENDRITES

MUSCLEFIBERS

But with the rotisserie, you can “set it

and forget it!”

“That’s exactly my philosophy on

electrophysiology. You put the device

in and you forget it,” he said. “That’s

different than the way we used to do

recordings, with a single electrode that

you had to move around and spend

hours collecting data. Here, you just

‘set it and forget it!’”

At least, that’s the hope. BrainGate

has a long way to go before it functions

well without oversight. For example,

the cables and components patients use

now are fairly onerous and impractical.

Hatsopoulos and Donoghue want to

make the system wireless.

There’s also work to be done on the

array’s ability to capture neuronal impulses.

Now, the quality and number of signals

recorded are inconsistent, and Hatsopoulos

doesn’t know why. The team also wants

to improve the decoding software that

translates the brain’s electrical chatter

into movement commands a machine

can follow. That requires faster and

more accurate algorithms, a task

assigned to Hatsopoulos’ group.

Just what is the brain-computer interface going

to interface with? A robotic arm? A computer?

A wheelchair?

CHIP SENSOR PROCESSSensors, or electrodes, on thesilicone chip detect signalsfrom surrounding neuronsin the brain’s motorcortex. This area is highlysaturated with neurons,but each sensor onlyneeds to detect signalsfrom 10 to 50 neuronsto trigger the BrainGatesystem to move thecomputer cursor. Thesensors act as facilitatorsfor the message, which iscarried out by the computer.

MOTOR CORTEXThis neuron-rich area of the braininitiates body movement.

Illus

trat

ion

by C

hris

tina

Ullm

an

Actual size

4 MM

1 MM

PEDESTAL

SILICONE CHIPThe chip is implanted in thebrain’s motor cortex.

sensor onsilicone chip

signals

neuronsreleasesignals

motor cortexsurface

From a physiological standpoint,

researchers need to find out just how

long the chip can remain in the brain

and continue to function. Some of the

monkeys they’ve studied received implants

three years ago that still function with no

changes in brain physiology. But it’s yet

to be seen how a long-term brain implant

will affect a human.

Finally, the researchers need to focus

on the end target: Just what is the brain-

computer interface going to interface

with? A robotic arm? A computer? A

wheelchair? That may be the most

difficult question of all—and one that

the scientists must answer if they are

to prove that BrainGate is better than

what’s already out there.

“The kind of control that can be

obtained, I believe, will surpass these

other current methods,” Hatsopoulos said.

“We’re just scratching the surface now.”