Towards Closed-Loop Stimulation To Improve Human Memory

by

Chaim Katz

A thesis submitted in conformity with the requirements

for the degree of Master of Health Science

Institute of Biomaterials and Biomedical Engineering

University of Toronto

© Copyright by Chaim Katz 2018

ii

Towards Closed-Loop Stimulation To Improve Human Memory

Chaim Katz

Master of Health Science

Institute of Biomaterials and Biomedical Engineering

University of Toronto

2018

Abstract

Age-related memory decline is a major health concern. Deep brain stimulation (DBS) without any

feedback (open-loop) has proven effective for neurodegenerative conditions like Parkinson’s

disease, but its effects on human memory have shown mixed results. This project’s objective was

to create a framework to test DBS with feedback (closed-loop) to augment human memory. A

memory task was developed based on our hypothesis relating eye movements, brain

electrophysiology, and hippocampal-dependent memory. The hippocampus is a key brain memory

structure and a prime target for DBS. Participants completed a visual search task featuring targets

embedded in realistic scenes. Results revealed above-chance corrected recognition for scenes and

associated targets. Closed-loop stimulation based on the recorded response during the task was

piloted in a single patient. This thesis will set the stage for further investigating DBS' functional

effects on the hippocampus and DBS’ utility in improving human memory.

iii

Acknowledgements

I would like to acknowledge my friends, lab mates and family who unconditionally supported me

through this process.

To Lauren who puts up with me and still smiles.

To my parents who always taught me to question and pursue knowledge as well as the rest of the

family who has listened to my complaints and advised me both personally and professionally.

To my roommates who put up with my late nights and quirks while I figured this out.

To those who assisted in the work, including Victoria Barkley and her interactions with patients

editing and support, Kramay Patel with analysis, Ryan Tian with the embedding algorithm, Dr.

David Groppe and his assistance with statistics, clinical staff and other lab mates from Neuron to

Brain Lab who have assisted on various aspects of the project.

To those who deal with me in extracurriculars and ensure I at least strive for a work life balance.

To Dr. Kari Hoffman whose work and support provides the foundation for this project and to Dr.

Talakoub whose consistent guidance and tips throughout the project has helped exponentially.

To Dr. Duncan and her Memory Lab for support in design of the task, stimuli selection and

validating embedding of the targets.

To my committee Dr. Katherine Duncan, Dr. Jose Zariffa, Dr. Jérémie Lefebvre, for their constant

guidance throughout the project and considered me worthy of the degree.

Finally, to my supervisor Dr. Taufik A. Valiante, who ensured, despite his incredibly busy

schedule, that I was constantly learning and growing in a unique and exciting opportunity, and

providing necessary context for the work with a balance of hands off and hands on guidance

approach to move this project forwards effectively.

iv

Table of Contents

Acknowledgements .............................................................................................................. iii

Table of Contents ................................................................................................................ iv

Lists of Tables .................................................................................................................... vii

Lists of Figures .................................................................................................................. viii

Lists of Appendices ............................................................................................................... x

1.0 Introduction/Background .......................................................................................... 1

1.1 Organization of Thesis ...................................................................................................... 1

1.2 Investigating Memory and Memory Systems .................................................................... 2

1.3 Electrophysiology Characterization of Hippocampal Activity .......................................... 8

1.4 Theta Rhythm and Phase Reset by Eye Movements as a Mechanism for Plasticity ........ 10

1.5 Event-Related Potentials (ERPs) .................................................................................... 16

1.6 Deep Brain Stimulation Studies ...................................................................................... 17

1.7 Project Rationale Summary ........................................................................................... 20

2.0 Aim and Hypothesis ................................................................................................ 21

2.1 Aim: Stimulation Modifies Memory Performance in a Phase-Dependent Matter ........... 21

3.0 Methods and Materials/Memory Task Iterations .................................................... 23

3.1 Common Elements ......................................................................................................... 23

3.1.1 Participants: ...........................................................................................................................23

3.1.2 Stimuli: ..................................................................................................................................24

3.1.3 Behavioural Procedure: .........................................................................................................25

3.1.4 Behavioural Data Analysis: ...................................................................................................26

3.1.5 Eye Tracking Data Analysis: .................................................................................................26

3.1.6 Intracranial Data Analysis: ....................................................................................................27

3.2 Optimizing Targets (Pilot Phase) Methods ..................................................................... 29

3.2.1 Participants ............................................................................................................................29

3.2.2 Stimuli ...................................................................................................................................29

3.2.3 Behavioural Procedure ..........................................................................................................32

3.3 Results for Optimizing Targets ....................................................................................... 32

v

3.3.1 Behavioural Results ...............................................................................................................32

3.3.2 Adjustments for Patient Testing ............................................................................................33

3.4 EMU Testing Methods .................................................................................................... 34

3.4.1 Participants ............................................................................................................................34

3.4.2 Stimuli ...................................................................................................................................34

3.4.3 EMU Testing Behavioural Procedure ...................................................................................35

3.4.4 Behavioural Data Analysis ....................................................................................................36

3.4.5 Eye Tracking Data Analysis ..................................................................................................36

3.4.6 Intracranial Data Analysis .....................................................................................................37

3.5 Results EMU Testing: ..................................................................................................... 42

3.5.1 Behavioural Results ...............................................................................................................42

3.5.2 Intracranial Results. ...............................................................................................................45

3.6 Feasibility of Closed Loop Stimulation Methods ............................................................. 53

3.6.1 Participant ..............................................................................................................................53

3.6.2 Stimuli ...................................................................................................................................53

3.6.3 Stimulation while performing behavioural task ....................................................................53

3.6.4 Behavioural Data ...................................................................................................................55

3.6.5 Eye Tracking Data .................................................................................................................55

3.6.6 Intracranial Data ....................................................................................................................55

3.7 Results Feasibility of Closed Loop Stimulation (n=1)...................................................... 56

3.7.1 Behavioural Results ...............................................................................................................56

3.7.2 Intracranial Results. ...............................................................................................................57

3.7.3 Adjustments for Final Task: ..................................................................................................60

3.8 Final Task Iteration Methods ......................................................................................... 61

3.8.1 Participants ............................................................................................................................61

3.8.2 Stimuli ...................................................................................................................................61

3.8.3 Behavioural Procedure ..........................................................................................................62

3.8.4 Behavioural Data ...................................................................................................................63

3.8.5 Intracranial Data ....................................................................................................................63

3.9 Results Final Iteration Multiple Blocked Targets ........................................................... 63

3.9.1 Behavioural Results ...............................................................................................................63

3.9.2 Intracranial Results. ...............................................................................................................65

vi

4.0 Discussion ................................................................................................................ 67

4.1 Behavioural-Recognition, Associative Memory and Eye Movements .............................. 67

4.2 Intracranial Response and Stimulation .......................................................................... 71

5.0 Limitations .............................................................................................................. 77

5.1 Patient Factors and Stay Within the EMU ..................................................................... 77

5.2 The testing environment ................................................................................................. 78

5.3 Technical/Experimental Complications .......................................................................... 78

6.0 Future Work ........................................................................................................... 80

7.0 Conclusion ............................................................................................................... 82

8.0 References ............................................................................................................... 84

9.0 Appendices ............................................................................................................ 103

9.1 Appendix A: Consent Form to Participate in Neuropsychological Testing ................... 103

9.2 Appendix B: Consent Form to Participate in Stimulation Study .................................. 109

vii

Lists of Tables

Table 1: Task iterations overview................................................................................................. 29

Table 2: Patient demographics single target ................................................................................. 34

Table 3: Stimulation permutations example with two patients .................................................... 54

Table 4: Patient demographics multiple targets............................................................................ 61

Table 5: Scalp electrode individuals and respective scene presentation in task. .......................... 63

viii

Lists of Figures

Figure 1 | Example taxonomy of brain system .............................................................................. 4

Figure 2 | MTL anatomy and model .............................................................................................. 7

Figure 3 | Phsiological relevance of brain sscillations ................................................................... 9

Figure 4 | Place cell firing field.................................................................................................... 10

Figure 5 | Theta reset and plasticity ............................................................................................. 12

Figure 6 | Theta phase relationship .............................................................................................. 14

Figure 7 | Theta gamma coordination. ......................................................................................... 15

Figure 8 | Visual demonstration of hypothesis ............................................................................ 21

Figure 9 | Example scene ............................................................................................................. 24

Figure 10 | Targets and scenes ..................................................................................................... 30

Figure 11 | Embedding target difficulties .................................................................................... 31

Figure 12 | Pilot results ................................................................................................................ 33

Figure 13 | Reconstructed electrode placements. ......................................................................... 37

Figure 14 | Saccade ERP comparing event total .......................................................................... 38

Figure 15 | Overview of task and extraction of ERP ................................................................... 39

Figure 16 | Corrected recognition EMU ...................................................................................... 42

Figure 17 | Online and offline fixation events ............................................................................. 43

Figure 18 | Main sequence saccades ............................................................................................ 44

Figure 19 | ERPs in iEEG participants......................................................................................... 46

Figure 20 | Overview of all electrodes of P3 ERPs ..................................................................... 47

Figure 21 | ICA component investigation .................................................................................... 49

Figure 22 | Individual ERP and overall average in single target ................................................. 51

Figure 23 | Theta power pre-and post-event onset ....................................................................... 52

Figure 24 | Closed-loop stimulation setup ................................................................................... 55

Figure 25 | Scene memory based on stimulation ......................................................................... 56

Figure 26 | Target memory results as a function of stimulation timing ....................................... 57

Figure 27 | Calibration hippocampal ERP demo ......................................................................... 58

Figure 28 | CCEP from stimulation ............................................................................................. 59

Figure 29 | Final task memory performance for scalp(n=2) and iEEG (n=2). ............................ 64

Figure 30 | Final task iteration ERPs. .......................................................................................... 65

ix

Figure 31 | Final task ITPC .......................................................................................................... 66

Figure 32 | Task for assessing memory ....................................................................................... 70

Figure 33 | Theta burst stimulation example: .............................................................................. 76

x

Lists of Appendices

Appendix A: Consent Form to Participate in Neuropsychological Testing .......................... 103

Appendix B: Consent Form to Participate in Stimulation Study........................................... 109

1

1.0 Introduction/Background

1.1 Organization of Thesis

This thesis is divided into six major sections: (1) Introduction/Background and Rationale, (2)

Hypothesis and Research Aims, (3) Methods and Memory Task Iterations, (4) Discussion, (5)

Limitations, (6) Future Work and (7) Conclusion.

Section 1, the introduction, describes the motivating problem to test contingent brain

stimulation to modify memory. This includes a literature overview of memory-related

demographics and methodologies covering memory and its neural correlates. Studies discussed

feature intracranial recordings during memory tasks in animals and humans while providing

context for a visual search and associative memory task to time stimulation effectively with a

specific brain oscillation associated with memory. Considering previously published research in

the area, the hypothesis, research aim, and objectives are defined in Section 2. Section 3, which

introduces the materials and memory task iterations/versions, is unique in its delivery: all memory

task iterations are discussed to demonstrate the project's incremental development. This section

includes each task’s methods and results, which informed each subsequent memory task iteration.

Common methods are discussed and each task iteration is then described in its unique setting.

From this point, the thesis is divided into two subsections of discussion in Section 4 based on the

results from all task iterations in Section 3. The first subsection primarily focuses on the

behavioural task and its place in the literature, while the other discusses the implications of the

intracranial-recorded neural activity and how the stimulation methodology may be implemented

regarding where, when, what, and how to stimulate. Sections 5, 6 and 7 moves into the study's

limitations and how some limitations will be mitigated moving forward. Further, these sections

demonstrate the promise of the proposed framework for testing closed-loop deep brain stimulation

effects in a novel paradigm. In Section 7, the conclusion,establishes that the experimental design

of the memory task, the corresponding intracranial response, and the closed-loop stimulation on

that response has been implemented in a hospital setting.

Two appendicies is included: the consent form for patients who participated in the University

Health Network Toronto Western Hospital REB-approved study.

2

1.2 Investigating Memory and Memory Systems

Memory impairment due to neurological disorders, structural damage, epilepsy, and

natural ageing represents a major health care issue. Several conditions can cause these decrements

in memory, attention, and cognition. For example, in Parkinson’s disease, cognitive decline is

apparent in attention and working memory impairments, as observed in planning and goal-oriented

behaviour [1]. Additionally, patients with epilepsy can exhibit degraded executive function and

performance on intelligence measures[2]. Specifically with temporal lobe epilepsy (TLE), where

seizures originate from temporal lobe structures, patients can present remarkable memory

deficits[3]. Fifty million people have dementia worldwide, and 10 million new cases are diagnosed

every year[4]. In Canada, 1.1 million dementia patients costs Canadians $10.4 billion annually; a

figure that is projected to double by 2031[5]. To reduce this escalating economic and social costs,

further research into how memory works and how to decrease the impact of memory impairment

is essential.

Among current projects investigating technology to reduce memory deficits is the

Restoring Active Memory (RAM) program under the Defense Advanced Research Projects

Agency (DARPA). The RAM program's focus is traumatic brain injury (TBI). TBI frequently

results in memory retrieval impairments. This can present as a loss of memories formed in years

leading up to the injury (retrograde amnesia) or as a reduced capacity to retain and/or retrieve

information acquired after the injury (anterograde amnesia). Few effective therapies currently exist

to mitigate the consequences of TBI on memory. The goal of RAM is to “accelerate the

technological developments that address this public health challenge” by developing

neuroprosthetics to repair and augment the injured brain [6].

To develop these prosthesis, prevent or reverse memory decline/dementia, it is important

to understand the underlying organization of memory systems and different types of memory

models. Declarative memory via language, which refers to the ability to recall or recognize facts

and events knowingly, is uniquely human to the best of our knowledge. Explanations for

declarative memory failures, specifically in ageing, fault inadequate encoding and retrieval[7].

Encoding is the process by which the brain takes new information and processes it. These memory

deficits affect those with TBI but also cause severe impairment of quality of life in individuals

with epilepsy.

3

Individuals with epilepsy are excellent research candidates for investigating memory for

several reasons, including:

1) The cognitive effects and impairments that already exist within individuals with

epilepsy[3];

2) A unique opportunity to investigate individuals with medial refractory epilepsy who as

possible candidates for surgery may be implanted with electrodes to record from deep brain

structures associated with memory, like the hippocampus and its critical role in both memory and

temporal lobe epilepsy[8].

Epilepsy is a chronic neurological disorder that affects approximately 0.6% of Canadians

and about 1% of the population worldwide[9],[10]. Despite efforts to control seizures with one or

more anti-epileptic drugs, one-third of individuals with epilepsy become drug resistant and require

alternative treatments [9]. It is widely acknowledged that memory is severely impaired in TLE

[11]. TLE is notable because of its high prevalence, its potential for drug resistance, and its

common disabling effects on memory functions [3]. Such individuals with medically intractable

epilepsy are likely candidates for diagnostic and subsequent therapeutic (resective) surgery to

eliminate seizures altogether or at least reduce seizure frequency. One diagnostic procedure to

determine resective surgical candidacy is to record intracranial electrical activity using intracranial

electroencephalography (iEEG). This involves implanting electrodes in and on the brain to record

and localize seizures. The patient is monitored 24/7 as an in-patient in the epilepsy monitoring unit

throughout the procedure. Patients with intracranial electrode implants provide a unique

opportunity to record electrical activity from deep brain structures, including those thought to be

involved with memory (e.g., hippocampus, entorhinal cortex, frontal lobes).

The first study to highlight the importance of the hippocampus and Medial Temporal Lobe

(MTL) structures to memory formation and retrieval was observed through H.M., a patient with

drug-resistant epilepsy who underwent resection of these structures[8]. Removing these MTL

structures resulted in the loss of much of H.M.'s explicit memory post surgery; specifically he

could not apparently encode specific types of new memories [8]. The case of H.M. inspired further

research and experimental work into these structures and eventually, the hippocampus' role in

explicit memory and the physiology of memory, which is discussed in the following section[12].

The type of memory that the hippocampus supports has been investigated extensively since this

first report. Despite a comprehensive body of research on memory, there is no consensus on the

precise functions of the hippocampus within the memory network.

4

To describe the hippocampus’ role multiple memory models have been proposed to

compartmentalize memory into subsystems that typically include the hippocampus[13]. One

such model divides memory into explicit/declarative (consciously aware) and non-declarative

(unaware) memory systems [14], [15]. Declarative subsystems include semantic memory

(general factual knowledge), episodic (autobiographical and personal memory) and verbal and

relational and visual-spatial memory [16], [17]. It is also argued that there are different processes

between explicitly recalling an item or being familiar with it [18]. An example of a taxonomy of

memory systems and associated brain structures adapted from Squire is presented in Figure

1[17].

Figure 1 | Example taxonomy of brain systems: This breakdown shows division of memory into

declarative and non-declarative forms. It also highlights that in Squire’s proposed framework

different brain structures are associated with specific forms of memory. Specifically, medial

temporal structures are under the declarative memory category. Adapted from Squire[17].

Based on work with H.M., it is believed that declarative memory is hippocampally

dependent. The hippocampus also contributes to, or is critical for, ability to understand relationship

between related and unrelated objects [19], [20]. In rodent models, it has been shown that the

hippocampus is crucial for spatial exploration that depends on memory [21] (further discussed in

5

section 1.4). Non-human primates, on the other hand, explore the environment differently, using

eye movements as an active way of exploring their surroundings. This parallel between human

and rodent navigation is strengthened by studies demonstrating hippocampal activation during eye

movement [22]–[24]. Visual exploration of a scene (a specific set of stimuli ) in healthy

individuals, including where one looks [25] and how often one fixates in a scene, can also indicate

potential memory strength [26]. The connection between eye movements and attention is also

evident as the planning of saccadic eye movement or saccade (quick simultaneous changes of both

eyes between two phases of the fixation) can also lend insight to a participant attending to a task

[27]. Additionally, hippocampal activation has been associated with a higher amount of eye

movements during a novel presentation [24], [28] and has been associated with implicit memory

formation [23]. In other words, “rodent-like” activation in the human hippocampus may occur

during visual exploration rather than a physical exploration of the environment. Thus, tracked eye

movements during a visual exploration task may help investigate hippocampal-dependent memory

in a lab environment.

As there are memory-associated anatomical structures, there are also theoretical constructs

regarding memory's different processes. Memory-related processes are thought to occur in three

stages: encoding, maintenance, and retrieval [29]. Encoding involves the transformation of current

perceptual representations into the cognitive domain through the strengthening of connections or

binding the representation with neocortical structures. Maintenance keeps and strengthens the

information into the mental focus. Retrieval is the process of bringing past information back into

focus [29].

The MTL and its substructures can be tested in these different subtypes and stages [30].

The human MTL consists of several functionally and structurally distinct areas including the

amygdala, parahippocampal gyrus/entorhinal cortex, perirhinal cortex, and hippocampus (Figures

2A and 2B). The hippocampus is divided into substructures and subfields: cornu ammonis 1

(CA1), cornu ammonis 3 (CA3) and dentate gyrus (DG). These hippocampal substructures seem

to be differentially involved during explicit memory testing [31]–[34] and in association with

spatial location [35]–[37]. These MTL structures and specifically the hippocampus are considered

crucial for memory's encoding stage and may also be relevant for retrieving items not being

attended to or not yet consolidated in the neocortex [29]. One proposed mechanism of how

hippocampus is involved with memory is that the hippocampus consolidates information with

neocortical representations through matched and mismatched predicted events from sensory input

6

[38]. Within the hippocampus, Hasselmo proposed a model whereby the anatomy of the

hippocampal formation and its connections receive input and encode information summarized in

Figure 2 C and D[39]. During the encoding process, it is believed information is stored using

hippocampal networks that bind together information between neocortical networks, and these

networks can strengthen over time [40], [41]. This idea is reinforced through the proposed

involvement of amygdala and hippocampus in sensory integration [42]. Specifically, when sensory

inputs diverge, it is believed CA1 is active as to compare past and present stimuli [38]. According

to Hasselmo’s model of hippocampal encoding and retrieval, CA1 is very active in encoding state

[39]. The hippocampus, and specifically CA1 and CA3, might govern comparisons between

internal representations from the hippocampus, neocortex and the incoming stimuli [43], [44].

These activities may also occur at opportune times for encoding and retrieval [45], [46]. In fact,

there is ongoing research in support of MTL structures in novelty detection [43], [47], with greater

MTL activation [48]–[50] and greater responses to remembered, compared to non-remembered,

items [51], [52]. This novelty may involve the perirhinal cortex, while contextual and associative

novelty are thought to engage the hippocampus selectively [49]. Novelty paradigms can use

stimuli like scenes to detect these activations, as human beings are presented with naturalistic

scenes every day[53].

The type of testing procedure for memory and hippocampal activation needs to be carefully

considered since the hippocampus may not only be important for explicitly remembered items but

also for associative memory such as stimuli or objects or targets paired with a scene [32]. A

participant’s search for these targets provides goal orientation in a memory task and may maintain

attention, which affects neuronal rhythms [28], [54], [55] and involve other memory subsystems

such as semantic guiding of search through scenes[56]. Even such task-dependent presentation at

retrieval, such as options for yes/no responses to indicate memory and association of target to the

scene rather than being given a forced choice with both targets, is thought to be more hippocampal-

reliant than forced-choice recognition [57]. Finally, a visual search incurs eye movements that

during scene viewing, can be considered an aspect of hippocampal function. These memory

models and concepts provide options to design memory tasks and investigate hippocampally-

dependent memory.

7

Figure 2 | MTL anatomy and model: A) MTL location and structures sagittal section from Stanilou

and Markowitsch [58] with B) a coronal subsection of the hippocampal formation [59]. C)

Hassemlo’s Model indicating hippocampal anatomical and functional connectivity1) Perforant

path to entorhinal cortex layers II and III with dentate gyrus, which projects via mossy fibres (2)

to CA3, which have pyramidal excitatory connections within CA3. CA3 and CA1 are connected

via Shaffer collaterals and CA1 has a perforant path from entorhinal cortex. D) Importantly,

according to this model, the entorhinal cortex provides input from neocortical structures

(feedforward/sensory information) and transmits output back (feedback) to neocortical structures.

CA1 undergoes self-organization and forms new representations of entorhinal cortex input for

C

D

A B

8

comparison with recall from CA3. Feedback from CA1 and a match or mismatch between stored

memories outputs from CA1 is and modulates acetylcholine changes set by the medial septum, a

structure also thought to induce the theta rhythm. See Section 1.4[39].

1.3 Electrophysiology Characterization of Hippocampal Activity

Intracranial electroencephalography (iEEG) is used for clinical isolation of seizure

pathology and for determining the onset of seizure. This is accomplished by placing electrodes in

regions of the brain suspected of seizure activity. The origin of recorded potentials occurs at the

cellular level where positive and negatively charged particles or ions travel in and out of the neuron

and across neurons terminal end or synapse to other connected neurons. This activity, synaptic

transmission, between neurons determine if the basic unit of information processing in the human

brain, an action potential, will occur. Connections between various brain structures form networks

of activity. Ions in these networks are detected as electrical currents, which also create measurable

magnetic fields. These cellular activities can be measured through the electrophysiological

recordings such as iEEG which provide high spatial precision and high temporal resolution.

Many epilepsy candidates for surgery are temporal lobe epilepsy patients which affords

the opportunity to record from the hippocampus directly. As part of the surgical plan to reduce

seizures, electrodes are implanted in MTL structures that may be associated with memory.

Simultaneously with clinical treatment, these electrodes and electrical recordings can also be

utilized in research to investigate cognition at an unparalleled spatiotemporal resolution [60]. Brain

activity from various sites can be measured to investigate what may be representative of, or

associated with memory function.

The first example of analysis identifying associations between oscillations and function is

alpha oscillations. This oscillation with a frequency of 8-12 Hz was discovered in 1929 by Hans

Berger as the increased amplitude of the activity when an individual closed their eyes [61]. This

finding provided insight into rhythms and different functional activities but provided no circuitry

mechanism for what causes alpha. Although iEEG recordings cannot identify the underlying

cellular mechanism, iEEG can provide insight into mechanism based on similar previous animal

studies.

Local field potentials (LFPs) measure the electrical activity of neurons in the surrounding

area. For a full review of perceived cellular contributions to LFPs see Buzsáki [62]. Briefly, there

is no single contribution to the electrically recorded activity. First and foremost, recorded cellular

9

activity decreases in magnitude inversely proportional to the magnitude of distance from the

source squared. Therefore, the surface recordings are a result a much larger summation of activity

and are believed to be a measure of synchronous activity of synaptic currents.

Through iEEG, one can localize activity to investigate memory and cognitive functions

and their possible dependence on the precise temporal pattern of activity in neural assemblies [63].

It is believed that inhibitory and excitatory neurons interact to create oscillatory behaviour/rhythms

within the brain and these synchronized activities are grouped by frequency bands. This periodicity

or oscillation refers to quality, state, or fact of being regularly recurrent: a repeating pattern or

structure in time or space and can be described by its components in patterns per cycle (frequency)

and the phase of oscillation within one cycle [64]. The phase of a neuronal oscillation is important

to timing and synchronization of oscillatory activity in the brain. Cycles per second is the

oscillatory frequency, and one measured type is the theta rhythms (4-7 Hz). Neuronal firing on the

other hand is thought to be correlated to high gamma(80-200Hz) activity in the recorded areas.

Using iEEG, one can investigate properties of specific oscillations such as their frequency, power

or synchrony using their phase alignment during relevant events (See Figure 3). Specifically,

analyzing these features in theta within the hippocampal regions in the animal literature suggest a

strong link between theta phase and behavioural performance.

Figure 3 | Phsiological relevance of brain sscillations: Physiological meaning of recorded brain

signals where phase plays an important role. Phase is important for coupling neuronal firing to

specific phases and coupling frequencies from different assemblies to specific phases. Different

10

frequencies of oscillation can be associated with different tasks and the amplitude or power of

activity within a frequency band can be related to task relevance. Taken from Klimesch et al. [65].

1.4 Theta Rhythm and Phase Reset by Eye Movements as a Mechanism for

Plasticity

In animals, the theta rhythm has been associated with memory and, in the case of rodents,

specifically with spatial memory [66]. Specifically, cells seem to fire at specific phases of

oscillations in the theta rhythm, which indicate an animal’s location within its spatial environment,

also known as place cells (see Figure 4).

Figure 4 | Place cell firing field: Example of place cells firing at specific phases of theta (highest

firing rate in the trough) and each neuron (N1-N5) moves along the theta oscillation as the animal

moves. Each neuron’s field is denoted along the track. Taken from Dragoi [67].

Impairment of theta activity, through cellular modifications, lesions, or stimulation, can

impair memory [68]–[71] suggesting a causal link between theta and memory. Theta “oscillations”

can be found in humans, but the frequency range is more variable [72]–[74]. Results of how theta

rhythm are modulated during tasks in humans have been inconsistent where some show that theta's

increase indicates improved memory performance while others show that theta's decrease indicates

memory effects (for full review see [75]). This variation may be due to the type of memory tested,

11

recording method used or even which frequencies were included in the analyses. The theta rhythm

in both animals and humans appears to modulate meaningfully with memory.

Mechanisms by which theta is generated remain unclear; however, one suggested

mechanisms of theta generation is through the medial septum afferent pathway, which if removed,

completely abates the theta oscillation [76]. However, theta can also be generated spontaneously

in the isolated hippocampus in-vitro suggesting endogenous theta generating mechanisms [77].

Although there is no definitive origin for theta, it in part relies on a combination of excitatory

inputs from entorhinal cortex and CA3, with interneuron activity mediated by hyperpolarization-

activated cyclic nucleotide channels from the medial septum [76], [78]. While the theta rhythms'

full mechanism is not understood, it seems to be associated and modulated with behaviour. Theta's

frequency also matches the frequency of whisking [79] and sniffing [80] in rats, plus saccadic eye

movements in humans [81]. Theta from the human hippocampus has also shown relation to human

working memory [82] and should be investigated further.

In cellular terms, memory can be defined using a Hebbian view of synaptic plasticity

summarized by the quote: “Neurons that fire together wire together” [83]. This principle is

predicated on the necessity of both pre- and postsynaptic activity occurring and the precise relative

timing, that then results in strengthening of the synaptic connection between the cells. Such

‘synaptic plasticity’ is believed to be one of the ways in which modifications to neuronal firing

patterns are achieved across distributed brain networks giving rise to the elusive ‘engram’. This

engram, represents a population of neurons that are activated by learning, have enduring cellular

changes and are reactivated by a part of the original stimulus for recall [84]. One putative

mechanism for synaptic plasticity includes increasing of N-methyl-D-aspartate receptors, which

can change the strength of activation at a synapse. This stronger synaptic connectivity known as

long-term potentiation (LTP) between cells will then require less presynaptic activity to drive the

post-synaptic cells to threshold.

In the broader discussion of timing, increased synchronization of neuronal and oscillation

activity can also be associated with memory formation [63]. Human intracranial recordings have

demonstrated that during successful learning [85], there is increased theta synchronization

between the hippocampus and parahippocampal gyrus. Such synchronization is known to be vital

for communication through the brain and is known as communication through coherence [86].

One suggested mechanism to induce this synchronization is through a phase reset. A phase reset

can align ongoing activity to a specific temporal reference, allowing a periodic stimulus to control

12

a neuronal oscillators phase, aligning it the to an appropriate timeframe [87]. Previous research

has shown that a phase reset of theta activity occurs both in the rhinal cortex and the hippocampus

at a fixed interval after image onset [85]. During learning, it is believed that the phase relationship

of inputs with respect to theta is fixed by a theta reset so that appropriate stimuli arrive during the

peak of the theta oscillation [63] (See Figure 5).

Similarly, as discussed, neurons in rodents can fire at specific phases of theta to encode

both specific places, grid locations and time[88]–[90]. To encode spatial locations or temporally

ordered sequences of multiple items, synchronization in the gamma frequency range is

accompanied by a stimulus-locked phase reset of theta oscillation [63]. The phase may encode

place but may also be especially important within the CA1 region for encoding and retrieval [63].

This phase reset may enable opportune timing and communication in the brain [63], [91], whereby

stimuli can activate the system subsequently at earlier phases if encoded using potentiation and

phase reset.

Figure 5 | Theta reset and plasticity: A) Reset enables the stimuli to be encoded at an opportune

time here the peak of the recorded signal, where the neuronal connection can be strengthened. The

natural neuronal rhythm ongoing in black is reset by a stimulus (orange bar) which effectively

enables the external stimulus(pink)to fire on peak of the recorded rhythm, which would not have

occurred without such a reset. B) Alignment of the arrival of stimulus with ongoing rhythms for

memory encoding is enhanced by phase reset of theta activity. Before learning, only stimuli

occurring during the depolarized phase of the cell elicit spikes (first arrow). Learning occurs when

B A

13

a salient stimulus, such as eye movement, induces a theta reset so that stimuli arrive at desired

theta phase A in peak of the rhythm. After this timed activation and synaptic potentiation, even

stimuli occurring during hyperpolarized membrane potentials initiate bursts of action potentials

Modified from Axmacher [63].

Phase reset can be introduced in a number of ways including tail pinching [92], afferent

stimulation, [93]–[95], and light [96]. Such reset provides an optimal window for LTP [96]. For

example, stimulation during the peak of theta produced LTP in rats [92], [96], [97]. Such a reset

might also enable these place cells to fire at specific times during the theta rhythm. Importance of

theta phase and neuronal firing is further supported as impaired theta rhythm does not affect the

firing of a place cell to a specific location, but instead impairs the behavioural performance likely

due to this theta phase misalignment of the firing cell [69].

The ongoing theta rhythm can also be modulated by selective visual attention [98].

Behavioural performance can be enhanced purely based on presenting stimuli during its presence

(eyeblink conditioning in rabbits [99], [100]), or at specific theta phases [101]. Of note in a

subsequent study, the original authors ([101]) did not find phase contingent behavioural

dependency on behavioural performance [102]. Although those authors suggested it might not be

as important, Hasselmo proposed a model by which the interactions between CA1 and CA3 were

strengthened for retrieval during the trough of theta rhythm [46], [103]. However, the trough

measured from CA1 and striatum radiatum is about 180 degrees out of phase with that measured

from the CA3 and fissure [104]. This interaction suggests that the peak of theta rhythm may be

more opportune for encoding external stimuli from afferent inputs via the dentate gyrus and

entorhinal cortex while inputs from CA3 arriving on the trough may be strengthened for retrieval

[46] (see Figure 6). In accordance with reset occurring for specific phase aligned activity, Siegle

showed that optogenetic stimulation of PV positive interneurons (inhibitory activation) at the peak

of theta enhanced encoding and at trough enhanced retrieval (recorded from fissure) [105].

14

Figure 6 | Theta phase relationship: Indication of opportune times for encoding and retrieval.

Adapted from Hasselmo[46].

Additional aspects of timing are essential whereby low frequency and high-frequency

oscillations can interact and communicate within or across structures known as cross-frequency

coupling (CFC) [106], [107]. Specifically, one theory posits that gamma (25-100Hz)

synchronization between hippocampal and parahippocampal regions may also induce LTP in the

CA3 region and that theta and gamma are thought to interact to encode memory items [108] (See

Figure 7). Theta gamma CFC within the hippocampus and between the hippocampus and

prefrontal cortex is involved with memory consolidation during wakefulness and sleep[109]–

[111]. Theta-gamma interactions have also been shown in associative memory [112].

One way to coordinate these communications and timing between sites and frequencies

might be through a phase reset. Others have shown a reset and subsequent memory effects

including, but not limited to, use in memory-related investigations in word paradigms [113]–[115],

Sternberg probes (working memory task where items/probes are presented in a series, with one

item tested) [82], [116] and complex visuals [51]. Long et al. (2017) has shown that theta is

important in the prefrontal cortex and hippocampus, but gamma rhythm may also have some

functionality isolated to hippocampus [117], [118].

15

Figure 7 | Theta gamma coordination: Adapted idea from Lisman and Jensen showing the

interaction between gamma and theta. Neuronal assemblies that are active in different

combinations are believed to represent different items. These items are active in gamma rhythm

as seen are modulated by the theta rhythm [108].

Evidently, there is importance in the temporal aspects of theta rhythm and plasticity.

However, in non-human primates (NHP) and humans, the theta oscillations are not as apparent

and do not appear as a continuous oscillation [119]. It has been shown that theta in a setup during

real world walking and exploration can be demonstrated in humans[120]. Difficulty in setup and

processing make estimating the precise phase of a theta oscillation in humans in real time

technically challenging [119]. Fortuitously, previously we and others have shown that in humans

and NHPs a phase reset in theta occurs at the termination of saccadic eye movement providing a

temporal window into theta phase [121], [122]. This phase reset has been evidenced as an event-

related potential [122] suggesting we can estimate peaks and troughs of theta in the hippocampus

as they relate to saccadic eye movements. What follows is a discussion of event-related potentials

that have been used in investigating memory.

16

1.5 Event-Related Potentials (ERPs)

Event-related potentials are obtained by averaging the voltage of the recorded signals over

many events aligned to event onset. ERPs have been measured in response to auditory, visual and

other sensory stimuli and can be elicited by a number of potential mechanisms[123]–[126]. The

precise neural circuitry that elicits the ERPs is unknown, however, analysis of their features can

provide some insight into neuronal mechanism. ERPs are thought to originate by either added

neuronal assembly firing or phase resetting of ongoing brain oscillations to and internal or external

stimulus such as seen in the aforementioned plasticity mechanism[91], [127]. ERPs can be used

to investigate critical aspects of perceptual processing, attentional selection and cognition [128].

Specific components of the ERP such as amplitude, latency and polarity can be indicative

of different processes, but not necessarily mechanism. Latency is the time from stimulus

presentation. Earlier latency responses are thought to be related to sensory inputs, while later

responses are thought to be dependent on processing stages on stimuli and also factors such as

predictability[128], [129]. These components can be investigated in diseased compared to healthy

participants specifically as it relates to cognition either with scalp electrode recordings [129],

[130] or with intracranial recordings in patients with epilepsy [114], [126], [131].

The amplitude of an ERP hasbeen used to infer the strength of encoding and subsequent

memory effects [51], [132], [133], memory capacity[134], differences between recognition and

recall, and can be themselves indicative of which events will be remembered [135] (for a full

review see [136]). While there is a rich literature of using ERPs to study MTL function in the

context of memory [115], [131] [137], the hippocampal responses to eye movements recorded

with iEEG has only come to light. Eye movement related ERPs have been shown to exist in

macaques [50], [138], and may represent a brain state during which there is heightened

connectivity between MTL and cortex [139]. Such eye movement related ERPs also exist in

humans [122]. Others have shown greater phase concentration aligned to saccade onset for

remembered as opposed to non-remembered items [140]. Thus, ERPs are important for memory

studies and may provide a temporal window into theta phase within the human hippocampus.

To summarize the current literature, it has been established that:

• A connection between MTL structures, specifically the hippocampus and memory

exists;

17

• Activation in the hippocampus is related to recognition and associative memory

and eye movements;

• A relationship between theta rhythm during spatial exploration and memory

encoding exists;

• Theta phase resetting provides an optimal temporal window for LTP, which can be

evidenced through an ERP;

• The ability exists to extract theta in humans through an ERP elicited by eye

movements, which themselves can be indicative of memory formation; and

• Timing is critical to human memory formation

1.6 Deep Brain Stimulation Studies

Despite our understanding of the physiology and timing as a fundamental mechanism

underlying memory, current DBS practices are open loop without feedback ensuring stimulation

is agnostic to the state of the neural system. Such open-loop approaches have been applied (refs)

to augment human memory due to its success in other neurological conditions like Parkinson’s

disease [141] with little if any success. In Parkinson’s disease, DBS involves stimulating at high

frequencies (>130Hz ) the basal ganglia or subthalamic nucleus, which causes a reversible lesion;

however the precise mechanism by which this works is unknown [142]. Resistive properties of

tissue can change over time, so voltage-controlled stimulation, which is often employed in the

clinic, may have variable effects over time [143].

Due to its success in Parkinson’s, DBS of the hypothalamus was attempted for the

treatment for obesity[144]. A serendipitous by-product of this treatment was improved memory.

Given the proximity of hypothalamus to the fornix, an afferent pathway of the hippocampus, it

was thought that stimulation of the fornix might mediate this memory effect [144]. Subsequently,

it was shown that stimulation of the fornix in early Alzheimer’s disease failed to demonstrate

memory improvement [145]. Given the knowledge of the fundamental importance of timing in

memory, it is not surprising that such an open loop stimulation approach failed to demonstrate

memory improvement.

Similar open loop stimulation literature implies that memory can be affected by stimulation

albeit negatively [146]–[148]. For example, in one early study, participants performed a spatial

memory task while receiving open loop DBS at 50 Hz for 5-seconds. Initially, these results seemed

18

very promising as participants showed improvement when the entorhinal cortex was stimulated

[149]. Despite these initial positive results, an in-depth review of electrical stimulation for memory

improvements in patients with epilepsy and Alzheimer’s disease shows varying results in

behavioural changes [150], [151]. Furthermore, in a more recent large-scale study, DBS has been

convincingly demonstrated to degrade memory [148].

However, some memory improvements have been noted using:

• Theta burst stimulation [152], [153];

• Coupling stimulation between sites [154]; and

• Post hoc brain state examination, which is based on the rhythms preceding

stimulation onset and machine learning principles [155].

Given these inconsistent results in open loop stimulation studies, but adjusting parameters support

that DBS can have an effect on memory specifically in the hippocampal network and pathways,

perhaps it is the parameters that need fine tuning [150], [151], [156].

The studies mentioned are all open loop which may not be as effective as closed-loop

stimulation using feedback to determine when to stimulate. Thus, it may be important to stimulate

at a relevant time in memory studies. In animal studies, stimuli contingent on the relevance of the

recorded signal has been shown to improve memory with concomitant modulation of the ongoing

signal [96], [101], [105]. When electrical stimulation matches hippocampal input activity with

respect to both spatial and temporal firing patterns, memory enhancement can result [157], [158].

However, these principles have not been tested in humans. Therefore, based on the animal

literature, we believe that the lack of efficacy of open loop stimulation highlights the importance

of the relative timing of electrical stimulation to ongoing neural activity. Our study explores this

critical question using eye movements to indicate opportune times for closed-loop stimulation.

Timing is one parameter that can be modified in stimulation studies, but intensity is likely

of additional importance. There is no clear evidence as to the best way to set intensity. Current

DBS research uses stimulation below the after-discharge threshold or below when a patient

becomes aware of the stimulation [147], [149]. With this approach, one can anticipate stimulation

currents that are too low resulting in no functional effects, or too high resulting in a pathological

activity (i.e., seizure).

Direct stimulation of the brain is standard practice for cortical mapping either for pre-

surgical evaluation or during surgery for functional mapping [159]. With this direct stimulation,

19

cortical connectivity as evidenced by corticocortical evoked potentials (CCEP) can be evaluated

[160]–[163]. Such CCEPs identify stimulation effects downstream of the hippocampus in the other

synaptically connected brain regions [42]. In fact, from our previous work, fornix stimulation with

multiple pulses appears to evoke a response in the hippocampus [164]. Many intracranial patients

with hippocampal depth electrodes, have electrodes in other MTL structures. The stimulation's

intensity can be established when stimulation-related changes are first noticed in these MTL or

other electrodes. It may, therefore, be important to have a DBS protocol that includes the

underlying principles of timing and connectivity within the brain.

Lastly, other than the stimulation protocol, the way in which hippocampally dependent

memory is tested while stimulated is critical. Therefore, the behavioural task design itself is

relevant to balance between experimental control and naturalistic behaviour that may be a

significant innovation for the memory-based DBS literature. To date, most research has assessed

DBS’ effects with either free recall of word lists [148], [155] or virtual navigation [148],

[149]. The high level of control in word recall tasks allows researchers to pinpoint specific word

memories that were formed during or after DBS. However, the tasks’ lack of natural visual

exploration strips away an essential primate behaviour, visual exploration, which is now

appreciated to be directly linked to hippocampal function [121], [122] and episodic memory [72].

On the other hand, virtual navigation is a highly natural and hippocampally-dependent behaviour,

but because patients are free to make repeated visual explorations through locations while building

a mental map of the full environment, researchers cannot isolate the specific memories that are

modulated by DBS or the timing of the modulation.

20

1.7 Project Rationale Summary

There is a societal need to investigate ways to improve memory. The chance to record

electrical signals from deep brain structures in intracranial patients in the epilepsy monitoring

unit(EMU) offers a unique opportunity to directly manipulate hippocampal-entorhinal cortical

networks to influence memory performance. Hippocampal DBS has demonstrated mixed effects

on memory, but overall appears to negatively interfere with it [147].

The reasons for the varying results of DBS for memory may in part be due to the utilization

of an open loop system, where stimulation is performed with no control or feedback mechanism,

and the nature of the behavioural paradigms that may not specifically test associational memory.

We hypothesize that it is the timing of the stimulation that strongly affects memory. Therefore, the

use of specific and physiologically meaningful hippocampal stimulation may provide consistent

memory enhancements [150]. We base this hypothesis on findings where we and others have

shown in both the human and NHP hippocampi [122][121] that at the termination of each and

every saccadic eye movement, there is a phase reset of hippocampal theta oscillations. We will

exploit this temporal window into theta phase to test the hypothesis of stimulating at optimal

timings for plasticity to improve memory in a closed loop fashion.

21

2.0 Aim and Hypothesis

2.1 Aim: Stimulation Modifies Memory Performance in a Phase-Dependent

Matter

The research conducted in this thesis works towards demonstrating the feasibility and

creating the infrastructure to explore the following hypothesis.

Null Hypothesis 1: Electrical stimulation at the different phases of the post-saccadic ERP will

NOT result in improved memory performance (See Figure 8).

Figure 8 | Visual demonstration of hypothesis: Hippocampal ERP elicited by eye movements can

be used to time stimulation effectively on specific phases of the ERP. One can extract the peak

and trough and stimulate at different times during different elements of the task. Adapted from

Hasselmo [46] and Hoffman et al. [122].

Research Objectives:

To test hippocampal-dependent memory, the current task was developed to achieve the

following objectives that cannot all be met by current standardized tests:

1. Assess memories that are hippocampally dependent;

1)

22

2. Evoke naturalistic visual exploration during memory formation to maximize opportunities

for saccade-triggered/theta reset stimulation;

3. Produce reliable, but moderate, performance levels so that DBS-driven enhancements and

impairments can be measured; and

4. Assess hundreds of unique memories so that multiple stimulation parameters can be

assessed within the same participant.

To test this hypothesis, a behavioural task and associated infrastructure were developed to of

a) assess hippocampal-dependent memory performance, b) obtain intracranial recordings and ERP

responses following saccadic eye movements and c) perform closed-loop stimulation protocol

triggered eye movements. This is described in the multiple task iterations section below.

23

3.0 Methods and Materials/Memory Task Iterations

In the experiments reported here, we tested healthy participants and individuals with

epilepsy undergoing EMU monitoring using visual stimuli to evaluate memory and record brain

activity. This methods section deviates from standard methods sections, as each iteration of the

task lent itself to the next based on the results. Multiple iterations of the task were performed to

isolate results for scene recognition, target performance, testing across the different participant

populations and with or without closed-loop stimulation. Generally, across all tasks, subjects

searched for a target within a scene and were questioned if they remembered the scene and its

associated target. Common elements are further described below.

3.1 Common Elements

3.1.1 Participants:

Participants in this study included healthy controls and individuals with epilepsy in the

EMU. Participants with epilepsy had either implanted electrodes to record iEEG or scalp

electrodes to record surface EEG. These testing groups are detailed further in the sections

pertaining to their respective task iteration. Healthy controls received monetary compensation and

were run according to University of Toronto protocols. EMU patients volunteered and were

considered under protocols for psychological testing and electrical stimulation from University

Health Network (UHN) Research Ethics Board (REB #16-5319-B) with signed consent obtained

for neurospychological testing (Appendix A) or stimulation (Appendix B).

24

3.1.2 Stimuli:

Photographs of natural real scenes were obtained from Dr. Katherine Duncan's Memory

lab (Department of Psychology, University of Toronto) (See Figure 9).

Figure 9 | Example scene: One example of a scene used in the task

Scenes were chosen as stimuli since: 1) our previous work utilized scenes as stimuli [122];

2) natural and complex stimuli are optimally encoded compared to artificially generated

images[165]–[167]; and, 3) they provide the richness of detail that a participant can explore

visually . Targets, were a 70x70 pixel image embedded in each scene (scaled for multiple targets).

The targets were varied in actual image and number of targets embedded depending on the memory

task iteration. Targets location was chosen according to the universal image quality index (UIQI)

[168] which is an algorithm that compares the similarity of two images. The target was compared

at each possible placement within the scene to find a distribution of similarity values within that

scene for each target. This algorithm was selected based on its prior use in a similar task of

25

camouflaging targets in scenes [169]. The UIQI was adjusted to a revised version of the called the

Structural Similarity Index (SSIM), [170] (See Section 3.2.2 for details). Images were presented

at resolutions of either 1280×1024 pixels, or in the multiple target iteration, at 1400x1000. Scenes

were presented to all study participants on a laptop computer monitor.

3.1.3 Behavioural Procedure:

This study was designed with minor variations to the protocol based on cohort; however, all

followed a similar overall task structure with single or multiple targets embedded in a scene.

Specific differences are elaborated upon in each task iteration section.

Common to each iteration and cohort was that participants were positioned 50-60 cm away

from a laptop monitor to perform the behavioural task. Each healthy control sat in a small testing

room under fluorescent lighting with 2–3 experimenters in the room and controlled the laptop

using a mouse and cursor to identify the targets. EMU patients remained in their bed within a

private, semi-private or quad room and an eye tracker was used to identify when they found targets.

Due to sensitivity of movement for the eye tracker, instead of using keyboard patients with

implanted electrodes orally reported their responses to the experimenter.

With EMU patients the eye tracker was calibrated, which consisted if a 9 or 5-point built in

calibration procedure of the eye tracker system (iView RED sampled at 120Hz, from

SensoMotoric Instruments, Teltow, Germany) depending on patient sensitivity. The eye tracker

was connected via USB to the laptop running the behavioural task using Presentation software

(NeuroBehavioral Systems, Albany, CA, USA).

An instruction set was presented to all participants describing the task and demonstrating

how to search for the target in the scene. A practice encoding trial was given for each task iteration.

Encoding/search blocks consisted of 50 scenes for healthy controls and 40 scenes for EMU

patients. Scene order was randomized.

Each encoding trial consisted of a presentation of the scene with target(s) embedded for 3

(patients) or 4 (healthy controls) seconds. Targets were randomly selected for scenes during the

single target iteration, but this order was then maintained across subjects. During the multiple

target implementation, scenes were presented in alternating blocks with the same target embedded

in each block. Participants were instructed to searched for the target(s) in the scene and once found,

they were encouraged (as part of the instructions) to continue exploring the scene for later memory

26

testing. After viewing each scene, the experimenter asked participants to name the target(s) and

how many targets were found. For EMU patient, each scene was preceded by a one second grey

screen with a fixation mark (+) in the centre to focus their eyes for the eye tracker.

Following the first encoding session, participants were presented with the instructions for

memory recall. Memory/retrieval blocks contained the same searched scenes as during the

encoding trials plus an equal number of novel scenes. In this block memory for previously viewed

scenes was tested by having them classify the presented scenes in to old (previously presented) or

new (not previously presented). Subsequently they were asked questions about the confidence of

this response. To test associational memory, if they indicated they had viewed the scene previously

they were asked which target was originally presented or associated with the scene. Analysis had

to be done on the behavioural results as well as the intracranial recordings aligned to the ongoing

eye movements during the behavioural task. To follow as closely to previous work in the lab,

intracranial and eye movement data were processed similarly to our study of the findings of phase

alignment with fixation onset [122]. An overview is described below, with a major exception

including the use of all eye movements in the scene.

3.1.4 Behavioural Data Analysis:

Multiple behavioural task iterations were run to ensure the performance of the task was not

at ceiling (too easy) or floor (too hard) of the task, thereby enabling us to observe DBS induced

increments or decrements to memory. Behavioural data were analyzed using R programming

language to determine performance in corrected recognition (Hit Rate- False Alarms Scenes). The

hit rate was the number of correctly identified scenes divided by the total number of scenes. False

alarms were novel scenes that were identified as scenes presented during the encoding task. Target

memory (correct targets based on total hits) was also analyzed.

3.1.5 Eye Tracking Data Analysis:

Using a dispersion-based algorithm (I-DT) with a minimum fixation duration of 80ms and

maximum dispersion of 100 pixels [171] eye tracking files were preprocessed offline with iView

X iTools IDF Event Detector, to identify fixations. Eye movements in real time were detected

using the inbuilt NBS Presentation software and compared to similarly offline detected eye

movement. Additionally, each eye movement or saccade had an amplitude and velocity associated

27

with it. When corresponding iEEG was recorded with saccades, the saccadic amplitudes and

velocities were compared for each subject and test session to ensure that correlation was present,

as this correlation is indicative of a natural human eye movement [172], [173]

3.1.6 Intracranial Data Analysis:

iEEG recordings from EMU patients were obtained using stereotactic depth electrodes

inserted using a stereotactic frame, or frameless stereotaxy through a burr hole [174]. In addition,

some patients were implanted with strip electrodes of 4 to 6 subdural platinum-iridium electrodes

3-mm diameter and 10-mm diameter-electrode distance (PMT, Chanhassen, MN, USA) targeting

anterior temporal, ventral-medial temporal, and posterior temporal locations and at times a 64-

contact grid of electrodes on cortical structures. A 4-contact subgaleal electrode over the parietal

midline and facing away from the brain was used for ground and reference. Electrode localization

was verified by co-registering a post-operative CT image with a pre-operative MRI structural

image using iELVIS toolbox [175]. Electrodes that were localized to the hippocampus were

analyzed [122]. In one patient all electrodes ERP were analyzed to isolate the response

iEEG data was acquired and digitized at 5 kHz following filtered at 0.1Hz–1 kHz

(NeuroScanSynAmps2 data acquisition system; Compumedics, Charlotte, NC, USA), and stored

to disk for subsequent analysis. In line with clinical practice the data was recorded inverting

polarity and presented as such in all results and figures. iEEG and eye movement data files were

read into MATLAB (The Mathworks Inc., Natick, MA) for ensuing analyses. iEEG signals were

first filtered between 0.5- 200 Hz with a notch filter applied in preprocessing at 30 and 60Hz to

remove line noise and artifacts. Data was further downsampled to 1kHz. ERPs were computed as

the averaging of data aligned one second around event onset. The significance of response was

tested using polarity shuffled individual events and averaging to create null distributions of noise

potentials. If any value of the mean actual response exceeded 2.5% of the maximum value of this

distribution, it was considered significant. A similar approach of taking individual trials rotating

randomly in time, creating an average ERP from these shuffled elements to create a null

distribution of possible ERPs is an established technique and both time and polarity shuffling

tended to show similar results [122], [176]. ERP values were deemed significant if they lay above

the 97.5th percentile of the distribution of maxima, or below 2.5th percentile of the distribution of

28

minima. The first peak and trough after onset were obtained to compare across experiments for

stimulating at these times in the future.

Spectral analysis is further detailed in section 3.4.6. Briefly, time-frequency plots aligned

to saccade onset were calculated using a Hanning window of 750ms, short time fast Fourier

transform and zero padded on edges from −1.2 to 1.2 s (i.e., windows centred from −875 to 875ms)

taken every 20ms. Results are displayed up to 30 Hz for the purposes of this experiment, to show

specifically theta and alpha region. Phase analysis was performed using the inter-trial phase

coherence (ITPC), a measure of the consistency of phase across observations in a given electrode

for a given frequency [176]. It is also known as the phase concentration index initially introduced

as Phase Locking Factor (PLF) [177], and intertrial coherence.

ITPC calculates a phase for each event (i.e., saccade), then determines the central tendency

of the distribution, but is subject to higher values with fewer events. We used the Rayleigh

Normalization, which is the squared value of ITPC multiplied by number of events. False

discovery rate (FDR) correction with 𝛼 = 0.05 was used to correct for multiple comparisons to

indicate significance over individual subjects [176]. Spectral power was computed pre- and post-

event onset to indicate if reset had, in fact, occurred in the theta frequency band [125], as phase

resetting is not associated with a power increase.

Task iteration Type of Patient Total Tested

Before Exclusion

Excluded

Optimizing Targets (Pilot

Phase)

Healthy Controls 17 2

EMU Testing EMU Patients - iEEG 7 1 (iEEG

data)

Feasibility of Closed-Loop

Stimulation

EMU Patients - iEEG 1 0

Multiple Blocked Targets EMU Patients - Scalp

EEG & iEEG

5 1

(behavioural

data)

29

Table 1: Task Iterations Overview

Each iteration of the task will now be described in detail with the differences highlighted

based on knowledge gained from the previous implementation. Table 1 shows the task iteration

and numbers within each subset of tasks.

3.2 Optimizing Targets (Pilot Phase) Methods

3.2.1 Participants

Initially, the task was tested on healthy controls to verify behavioural performance. Healthy

controls (n = 17; 13 females, 3 males, 1 non-binary) with no history of psychiatric or neurological

disorders participated in this study, with 2 excluded for analysis due to issues with behavioural

comprehensions with the task. The number of participants was consistent with other studies

comparing epilepsy to healthy control subjects [178]–[181]. Healthy control participants were

compensated as per University of Toronto guidelines. For this pilot study, age and education-

matched controls were not used; however, in future research to compare across populations, this

will be necessary. The pilot testing was undertaken to identify baseline memory performance in a

control population before proceeding with the EMU patient populations and to optimize target

selection.

3.2.2 Stimuli

Experimental material and methodology were developed specifically for this study to

capture a wide range of indoor and outdoor locations.

The targets were embedded in two separate experiments, one with a geometric set and one

with a semantic set (See Figure 7). The purpose was to optimize performance using targets that

could be remembered.

30

.

Figure 10: | Targets and scenes: A) Geometric B) Semantic (attributed to www.stockio.com) and

C) Example scenes of eventual semantic targets embedded in the scene. Note the targets in this

scene are highlighted with green circles but were not during the presentation of the task.

Embedding Target in Scene and Stair Casing

In a visual search, some subjects will be inherently more skilled than others at finding the

target. To focus the task on memory, rather than those who found targets first, it was important to

create a task that objectively placed images and changed difficulty based on individual skill. The

aforementioned algorithm used to embed targets was called SSIM[170].

𝑆𝑆𝐼𝑀(𝑥, 𝑦) =(2𝜇𝑥𝜇𝑦+𝐶1)(2𝜎𝑥𝑦+𝐶2)

(𝜇𝑥2+𝜇𝑦

2+𝐶1)(𝜎𝑥2+𝜎𝑦

2+𝐶2) (1)

A B

C

31

Where 𝜇𝑥is the mean intensity of target, 𝜇𝑦 is the mean intensity of the scene background to be

overlain, 𝜎 is the standard deviation of pixels and 𝜎𝑥𝑦 is the correlation coefficient. This eventual

equation is a product of the factors of the luminance of the image, the structure of the image and

contrast to obtain a similarity metric between two images. In the original implementation, targets

were placed based on the percentile within the image, (i.e., the strongest similarity strength

comparison between target and image would be the 100th percentile), but this did not necessarily

mean that each target was embedded at the same difficulty. This algorithm works based on the

comparison of the target to the scene using its grey values. The adaptation for colour images was

also verified by contact with the author of the original algorithm and its use in the camouflage

study [169].

This algorithm outputs the similarity between target and scene between -1 and 1 and we

selected the locations based on the distribution of similar placements and coordinates within that

image. Five difficulty levels of SSIM for target embedding were used, where level 5 placement

corresponded to the strongest similarity with respect to all possible placements of the target within

that scene. For all possible placements of target within the scene SSIM values were calculated.

From that distribution of values, the 80th percentile of similarity corresponded to level 1 difficulty

and most difficult was 100th percentile (See Figure 8). These difficulty levels were then used in a

staircasing method to increase or decrease difficulty of the embedded target based on the

participants performance and finding time in the task.

Figure 11 | Embedding target difficulties: Example of embedding geometric target blue star based

on percentile on SSIM (0, 80 and 100), where difficulty levels were from 80-100 in increments of

5% per difficulty level.

32

3.2.3 Behavioural Procedure

Two sets of 50 search scenes were used followed by 100 recall scenes (50 old and 50

novel). Targets were randomly selected to be placed in scenes. Once that order was set, these

targets and order of scenes within an experimental session was used for all participants.

Instructions were given with a demonstration using 10 sample images not used in the experiments.

Eye movements were not recorded for this iteration. The mouse cursor was used to provide

feedback on how quickly targets were found within the timeframe; once the target was found the

cursor would change from green to red. This enabled analysis of the time to find the target leading

to adaptation of embedding difficulty, if necessary. Specifically, if a participant found the target

three times in a row, the difficulty would increase; if they missed the target twice in a row, the

difficulty would decrease.

Each healthy control was presented with two experimental sessions, alternating the order

of geometric or semantic targets in different scenes. For example, the geometric session would be

first for one participant, and the semantic session would be first for the next participant. Scenes

were presented on the screen for three seconds. After encoding session, they were presented 100

scenes (50 old, 50 novel) and were asked if they had previously viewed the scene. Participants

responded “yes” or “no” and were asked to rate their confidence on a scale of 1 (“not sure”)

through 5 (“very sure”). If the participant responded “yes” to viewing the scene, they were asked

to identify the target (i.e., star or cat) or “don’t know”. (Of note, for the first three subjects, the

order of questions was as follows: previously viewed scene, identify which target, confidence in

previous viewing of scene)

3.3 Results for Optimizing Targets

3.3.1 Behavioural Results

3.3.1.1 Scene Memory

Scene memory was well above chance (corrected recognition = .5; chance is 0). There was

also no statistical difference between the tasks with geometric or semantic targets (See Figure 12).

33

Figure 12 | Pilot results: Scene corrected recognition shows no difference between tasks with

either geometric or semantic targets embedded (n = 15).

3.3.1.2 Target Memory

Patients found targets around 90% of the time, but with improved performance (60ms

faster) during the semantic blocks. Our goal was for participants to always find the targets, thus

more search time was implemented in other iterations. Target memory was above chance at 62%.

3.3.2 Adjustments for Patient Testing

Previous literature suggests memory performance would very likely drop in patients due

to epilepsy [179], [182]. Therefore, results from the pilot study prompted the following protocol

changes:

1. Adding one second to search time (t=4s) to enable more viewing time and likely improve

memory performance;

2. Selecting semantic targets instead of geometric targets to try to improve target memory

performance;

3. Reducing scenes from 50 searched and 100 tested to 40 and 80 respectively to decrease

number of stimuli and improve performance

4. Replacing the set of scenes to introduce more variability between scenes in a new set, rather

than have interference between genres of scenes;

5. Changing confidence scale to 1-4, providing 8 levels of confidence with 4 levels for either

yes or no answer.

This modified task will be described in the next section, along with associated analyses and results.

Task Type Corrected Recognition Performance (n = 15)

Corr

ecte

d R

eco

gnit

ion

(Hit

Rat

e – F

alse

Ala

rms)

Task Type Geometric Semantic

34

3.4 EMU Testing Methods

The two objectives of this iteration were to determine if we could in fact obtain an online

detected saccade associated ERP during administration of our new task, and to ensure that

behavioural performance was above chance and not at ceiling in iEEG patients.

3.4.1 Participants

Seven individuals (three females) undergoing iEEG recordings with medically refractory

epilepsy underwent surgical implantation of subdural surface electrodes and depth

macroelectrodes or sterotactic electrodes to localize epileptogenic regions participated in this

iteration. The average age of patients at time of testing was 38 years, and the appropriate

neuropsychological information was reviewed to assure that potential participants had no profound

psychiatric comorbidities, such as severe paranoia or executive dysfunction that might interfere

with participation in the task. Subjects varied in terms of placements of electrodes and radiological

imaging results based on what they needed for medical treatment (see Table 2).

Patient Implantation Gender Handed Imaging Age

1 Bi Front Bi Temp + sEEG

F R Unremarkable

45

2 Bi Temp F R Unremarkable 47

3 R Grid F R cortical dysplasia amygdala 31

4 L Burr hole M R Unremarkable 29

5

R Grid M R

Developmental venous anomaly in

left parietal lobe

36

6 Bi-Frontal sEEG M L cortical dysplasis (r insula) 42

7 Bi temp and R-Front M R essentially unremarkable 38

Table 2: Patient demographics single target task

3.4.2 Stimuli

A new set of stimuli were provided by Dr. Katherine Duncan’s Memory Lab. An

instruction set similar to task with healthy controls was used with 10 sample scenes before

beginning a calibration session to acquire the peak and trough times of the ERPs for each patient.

Memory testing blocks consisted of 40 scenes during encoding and 80 scenes during retrieval.

35

Resolution of images was 1280x1024 using only the semantic targets. To reduce interference,

experimenters subjectively judged the scenes and removed overlapping scenes from the stimuli

beforehand. For example, one of three similar mountain scenes would be used, with the other two

being discarded. This would ideally increase the memory for scenes without similarity

interference. Semantic targets, as in the task test, were selected to be embedded in the new scenes

at 70x70 pixel resolution.

During this iteration, an embedding error took place. Targets used in this iteration were

initially bordered by a black background that was not compared against the scene by the SSIM

embedding implementation since it was not part of the target embedded. After testing the patients

and moving on to the multiple target experiment (see Section 3.7), it was noted that the embedding

algorithm had included the background of the target. This meant that the black pixels from the

background were incorporated into a comparison between targets and scenes. When the

background was removed in subsequent tests, targets were still embedded in similarly difficulty

locations and the targets were being found within the four seconds. Therefore, it did not appear to

be a major problem that the embedding may not have been challenging enough. Test could no be

re-run since patients had already viewed the scenes, been discharged and had electrodes removed.

3.4.3 EMU Testing Behavioural Procedure

The memory task was coded using NBS Presentation. Electrical signals known as TTL

triggers generated by NBS Presentation through the parallel port were used to synchronize the

electrophysiological data, eye event information, and behavioural events for subsequent offline

analysis. These TTL triggers were created for: 1) Start of Experiment, 2) Start of Each Image

(scene, target screen, confidence), 3) Start of Memory Task, 4) Break in Memory Task, 5) Found

Target, and 6) Test/Stimulation Type or 7) Fixation (demonstration). Eye movement TTL events

were evaluated by iView Redm (SensoMotoric Instruments, Teltow, Germany) that tracked eye

movements in the task at a sampling rate of 120Hz based on our previous work [122]. Eye tracker

calibration was done for each participant using a 9 or 5-point calibration with inbuilt calibration.

Obtaining the saccade related ERPs form which peak, and trough timing are obtained

(critical step in performing our closed-loop stimulation. A calibration session, or demonstration,

where the subject searched scenes was used to allow familiarization with the task and to obtain

peak and trough timing from the saccade related ERPs. As described previously, presenting a

36

visual stimulus results in a phase reset [96], [106], [183]; however, we were not interested in scene

onset but in eye movement-elicited activity. During calibration, there were 40 scenes and based

on a mean fixation rate (2/s), at 4s per scene we estimated at least 320 fixation/saccade events

should occur. (See Section 3.4.6.1). These events enabled the acquisition of a saccade related ERP

from each patient to determine the timing of the subsequent peak and trough of the reset theta

oscillation. Peak and trough times would then be used to deliver stimulation at these phases of

theta, similar to what had ben performed in a potentiation study in rodents [96]. However, it is

important to determine the ERP is robust and would correspond to similar timings throughout the

recorded test blocks (see Section 3.4.6 Figure 15B). Therefore, in the current setup, all recordings

were performed without stimulation and timings across tests (where possible) were compared.

Some participant’s data did not have multiple tests due to seizures or synchronization error.

Analyzing across text blocks allowed us to visualize the accuracy of peak and trough times after

saccade onset within that subject and the hippocampal electrodes.

There were three test sessions: 1) the Demo for Calibration Session to obtain ERP timings

and 2) Sem1 and Sem2 which were the blocks where encoding of scenes and memory were tested.

Forty scenes were presented in each search task with each scene presented for 4 seconds. While

searching, the cursor on the screen was programmed to follow eye movements and change to red

once the participant found the target. After each search scene, patients were asked whether or not

they found the target. In the memory block 80 scenes were presented and for each memory scene,

patients were asked if they recalled the scene and how confident they were in this response (scale

1-4). If they had recalled the scene, they were asked which target was associated with that image.

All responses were recorded by the experimenter

3.4.4 Behavioural Data Analysis

The same metrics of performance corrected recognition and target memory were analyzed

in the iEEG patient as in the pilot study. Reaction times and find times could not be utilized since

the subject was not making the responses and there was often poor eye tracker calibration.

3.4.5 Eye Tracking Data Analysis

The original work on which this thesis was based was performed using fixation events that

were detected offline [122]. For our current setup, saccade events needed to be triggered in real

37

time, but needed to be similar in time to the events detected in offline analysis. Additionally, the

saccadic amplitude and peak velocity were extracted from events in each test block to visualize

and confirm that these variables were correlated. Correlation suggests that the saccades are natural

[172].

3.4.6 Intracranial Data Analysis

Neural data was synchronized for offline analysis with eye-tracking data using TTL

triggers sent from the laptop computer to the NeuroScan computer. Electrophysiological data was

pre-processed in each trial by bandpass filtering between 0.5 and 200Hz using a second order

Butterworth filter, and notch filtering at 30Hz and 60Hz, 120Hz, and 180Hz to remove line noise

harmonics and artifacts. All further analysis was performed using custom-written scripts in

MATLAB (The Mathworks Inc, Natick, MA, USA.) Electrode localization was performed by co-

registering pre-op MRI with post-op CT using the iELVIS toolbox [175] (see Figure 13).

Following localization, the location of each of the 4 hippocampal electrodes was confirmed with

the neurosurgeon who implanted them.

Figure 13 | Reconstructed electrode placements: Reconstructed electrode locations for left

hemisphere of an example iEEG EMU Patient. Coloured circles represent individual electrodes.

A) Coronal images of left hippocampal depth electrodes (red). B) Overall implantation of strip

electrodes in the brain.

A B

38

3.4.6.1 Event Related Potentials (ERPs)

Our current unpublished lab work characterizing eye movement response and image onset

suggests that the eye movement ERP itself seems to be correlated with saccade onset rather than

fixation onset [184]. Therefore, results presented here align ERPs to saccade onset. To obtain ERPs

for saccade-onset events, iEEG data from all events and all relevant electrodes were aligned to the

saccade-onset and trimmed to epochs of 1s before and after each event. The mean of these saccade-

onset epochs from all trials and experimental blocks (Demo, Sem1 and Sem2) determined the ERP

for each electrode. The significance of each ERP was determined by a representative distribution

of ERP maxima and minima for each electrode/ERP using non-parametric permutation testing

with randomized polarity inversions (1000 permutations) [122], [176].

Significant peaks of each ERP were identified using custom-written scripts in MATLAB

and were visually inspected to ensure accuracy. Fewer events would affect the significance of the

response (see Figure 14). However, that was not especially relevant for stimulation. This is because

the peak and trough timing, although not significant with fewer events, is still evident. Therefore,

the purpose of this analysis was to show that the ERP existed and there was a subsequent peak and

trough. A full overview of the task, with encoding, retrieval and extracted ERP example can be

seen (Figure 15).

Figure 14 | Saccade ERP comparing event total: Data from the same subject with

>20,000 events in blue and ~300 events in orange. Significant time points are indicated by the

lines at the top of the plot (p < 0.05 cut-off of the saccade-polarity shuffled distribution).

39

Figure 15 | Overview of task and extraction of ERP: A) Overview of Task Encoding phase: visual

search for embedded targets. Retrieval phase where 40 searched and 40 novel images are

presented. Scene recognition and recall of associated target are tested. Note: In this paradigm, the

targets were bird and cat instead of a bunny, but the overview of the task is the same. B)

Demonstration of calibration and peak and trough times aligning. This is the eye movement ERP

recording from the same subject for two tests showing a response to extract timings can be found,

similar across blocks and could be used for later stimulation

B

A

40

Previous literature suggests that the ERP might be due to the oculomotor artifact [185],

[186]. To investigate this, electrooculography (EOG) electrodes were placed on one iEEG EMU

patient. Activity was recorded as per clinical practice with surface electrode patients to have a

general idea of eye movements and artifacts that may be present. Independent component

analysis(ICA), a decomposition of signals that tries to extract unique elements and their

contributions from the different electrodes, was utilized in the MTL depth electrodes and the EOG

[185]. ICA refers to a group of algorithms that maximizes statistical independence of

elements[187]. The goal is to find a linear projection that “unmixes” the component that

contributed to the recorded signal. The components that were contributed maximally to EOG

positions were then removed, and the ERP was visualized to see if the response was still present.

For all electrodes, the ERPs were analyzed with respect to the clinical reference. The

hippocampal-located electrodes were further analyzed for ITPC and power.

3.4.6.2 Inter-Trial Phase Clustering (ITPC)

To calculate ITPC, the time-frequency representation of all saccade was first obtained

using a short-time Fast-Fourier Transform (stFFT). A Hanning window of 750ms, in steps of

20ms, was used to reduce edge artifacts. The following formula was used to compute ITPC:

𝐼𝑇𝑃𝐶𝑡𝑓 = |1

𝑛∑ 𝑒𝒊𝜃𝑟𝑡𝑓𝑛

𝑟=1 | (2)

Here, 𝑛 is the number of epochs (saccade or image-onset) for each electrode, and 𝜃𝑟𝑡𝑓 is

the phase angles of the 𝑟𝑡ℎ event at time 𝑡 and frequency 𝑓. The calculated value 𝐼𝑇𝑃𝐶𝑡𝑓 is the

ITPC value at a given time-frequency point as in Equation 2 [176].

Since the number of events biases ITPC value, and one subject may have more saccades

than another ITPC values were transformed to ITPCz values, also known as Rayleigh’s Z, as

shown in Equation 3 [176].

𝐼𝑇𝑃𝐶𝑧 = 𝑛 × 𝐼𝑇𝑃𝐶2 (3)

Initially, the within-electrode significance of each ITPCz value was obtained using

Equation 4, where n was the number of epochs used to obtain the ITPC values. This initial

41

significance was then corrected for multiple comparisons using the Benjamini Hochberg FDR

correction at 𝛼 < 0.05 significance level. Significant portions could then be masked for

visualization [176].

𝑝𝑖𝑛𝑖𝑡𝑖𝑎𝑙 = 𝑒−𝑛 ×𝐼𝑇𝑃𝐶2= 𝑒−𝐼𝑇𝑃𝐶𝑧 (4)

3.4.6.3 Spectral Power Analysis

Spectral power was not normalized to a baseline period because it was only used to

compare pre- and post-event powers. The average power in the pre-and post-event period was

obtained for theta frequency band to discriminate between a reset and an additive response by

comparing the change in power over individual events and ERP [125]. Pre- and post-power was

calculated, and the mean power of each frequency band was compared using Wilcoxon Signed

Rank Test. This test was chosen as normality of sample distributions could not be verified and

there was a limited number of subjects.

Exclusion: For behavioural results, all seven patients were included, while for intracranial

results P7’s electrodes could not be localized to the hippocampus and are, therefore, not presented.

Additionally, due to software synchronization errors or seizures in the task, some intracranial test

ERPs are not presented. As part of the initial transition between healthy controls and patients, P1

an iEEG patient also performed the task on both the new and the image set performed with healthy

controls. This participant’s recordings employed a different acquisition system (Digital Lynx

acquisition system; Neuralynx, Inc.) and the memory performance data from the healthy control

stimuli are not presented here. Subsequently, experiments were performed using a new set of

scenes, including P1 on this new set.

A full overview of the task can be seen in Figure 15 with an example of the extracted ERP.

42

3.5 Results EMU Testing:

3.5.1 Behavioural Results

3.5.1.1 Scene Memory:

Patients performed above-chance corrected recognition 0.51 (p = 3.15E -5) (see Figure 16).

Corrected Recognition iEEG EMU Patients (n= 7)

Figure 16 | Corrected recognition EMU: Behavioural performance corrected recognition among

patients where chance is red (0.0) and ceiling performance is green.

3.5.1.2 Target Memory:

Target memory was above chance (0.60), but not statistically significant. Reaction times

could not be used since patients did not control the experiment and time to locate the target,

although important in psychological studies, could not be used since the calibration of the eye

tracker was inaccurate (see Section 5 Future Directions).

3.5.1.3 Eye Tracking Results

A problem that we encountered was that online detection yielded a discrepancy of ~88ms

as compared to the offline detection. This discrepancy arose from the time required to collect

enough data in real time to determine eye movement events and algorithmic differences between

online and offline fixation events. These differences introduced temporal jitter, which is not

Subject Number

Corr

ecte

d R

eco

gnit

ion

(Hit

Rat

e – F

alse

Ala

rms)

43

optimal for phase synchrony analysis [176]. This can introduce patient-dependent delays longer

than when peak and trough occur from the ERP response.

Based on our previous work utilizing offline detected fixations, it was necessary to show

that online events could be aligned to offline fixation events. In the first patient, this was performed

based on the location of detected events offline and online, where the correspondence in time was

considerable (>100ms on average). In subsequent patients, this was done using determined offline

and online locations and monocular detection of events. Online and offline saccade detection times

were eventually found to be highly correlated (r = ~ 0.99) and we were able to show that the

detection mechanism could be aligned with our previous offline data within an average

discrepancy of 5ms (see Figure 17).

Figure 17 | Online and offline fixation events: Fixation trace indicating location and time

difference of fixation during image exploration. Online fixation events are red asterisks; offline

are blue circles.

While offline and online events were correlated, the calibration system and raw position

of the eye tracker were compromized by eye glasses. This introduced participant eye tracking data

that were well out of natural viewing statistics such as introducing scenes with only one or two

saccades in four seconds.

44

Another concern for this task was the eye tracker calibration preventing the cursor to

properly track the eyes resulting in seemingly ballistic eye movements. Furthermore, this

prevented us from accurately changing the cursor color when the participant had correctly

identified the target causing the participants to hover around the target, using short and frequent

saccades.

Therefore, we ensured that the eye movement we did record were considered natural as per

regularities recorded in eye movements as defined by a correlation between saccadic amplitude

and saccadic velocity known as the main sequence [172]. As evident for each subject where

intracranial analysis was extracted, saccades seemed to be natural for this task (See Figure 18).

However, the saccadic rate was quite low at 2.57 saccades/s. Possibilities for the low rate could be

due to poor calibration, only one target to find and a relatively small screen.

Figure 18 | Main sequence saccades: Saccade velocity amplitude correlations for each subject in

which intracranial data is presented to verify natural eye movements. There should be a correlation

in detected events. Sessions in which eye data is not present, the intracranial data reasoning for

exclusion is included in the figure.

45

3.5.2 Intracranial Results.

3.5.2.1 ERPS

The saccade related ERP was obtained from all testing blocks performed in iEEG patients,

form which iEEG data was acquired. Since the demo session would be used for stimulation, it was

necessary to show that this ERP could be used across stimulation trials, such that the ERP was

similar in each block and could be utilized as a metric for stimulation. Therefore, the time of the

earliest prominent peak and trough were obtained (Figure 19A) for each test session (Demo, Sem1

and Sem2). The overall electrode ERP from hippocampal electrodes are also shown with statistical

significance in Figure 19B.

As seen in Figure 19, the time of first peak and trough was variable in some patients,

electrodes, and sessions. However, the average time difference between the first peak of all

possible combinations of the session (i.e., timing of extracted peak and trough between Demo and

Sem1, Demo and Sem2, and Sem1 and Sem2) within the subject then averaged across subjects

was 44ms and trough was 17ms. Note that this first peak and trough is calculated by using the peak

detection findpeaks function in MATLAB selecting the first most prominent peak (based on the

amplitude of the peak and peak width) and trough within 300ms of stimulus onset.

46

Figure 19 | ERPs in iEEG participants: A) Response of electrodes for subjects over different tests

sessions aligned to saccade onset, peak indicated in green and trough in red. Each ERP is a different

test block Blue is Demo, orange is Sem1 and yellow is Sem2. B) Average overall events from all

blocks in the same hippocampal electrodes; significance lines are shown. P indicates patient, R or

L indicates laterality of hippocampal electrode.

A

B

47

From our previous work the hippocampal ERP we observe might not be purely

hippocampal [122], [188], [189]. Resets have also been observed in other areas such as the anterior

cingulate cortex and visual pathways [188]. Therefore, to further analyze the origins of the ERP,

as an example the ERP of P3 is viewed across all electrodes. The response itself does not

necessarily seem isolated to the hippocampus (red; Figure 20). The ERP of P3 was selected as it

had both a significant peak and trough as well as wide spread cortical coverage including EOG

electrodes for detecting eye movements.

Figure 20 | Overview of all electrodes of P3 ERPs: Overall saccade aligned response for P3

indicates that the subsequent theta peaks and troughs can be viewed in other temporal lobe

electrodes and that a sharp transient spike before onset is visible in EOG and other depth

electrodes, such as RAD2 and RAD1.

One can see a sharp transient in many of the electrodes before saccade onset including

those close to the temporal pole as well as a theta peak and trough that looks like the hippocampal

ERP (RHD3). The response seems to be apparent in other temporal contacts and neocortical

electrodes, although the hippocampal ERP in this subject’s response is not correlated with any

other electrodes (R> 0.8) suggesting it has its own unique response. It is also worth noting that

there is a sharp transient that appears to be present before onset (See Figure 21) within the EOG

Time(s)

Volt

age

(µ𝑉

)

48

and other electrodes. This transient is further elaborated upon in the discussion. However, it may

be indicative of an oculomotor response [185], [186]. Independent component analysis was

attempted to try and remove the artifact; however, it was seen in multiple components around the

temporal lobe, consistent with previous literature [185]. To further analyze the hippocampal ERP

ICA was run on the MTL depth electrodes and the EOG electrodes. In this respect we could isolate

the components from EOG electrodes that maximally contributed to the ERP (see Figure 21).

49

Figure 21 | ICA component investigation: A) Investigation of components of the medial temporal

lobe electrodes (RHD RAD) and EOG, distinguish in A that components 6 and 7 are maximally

EOG. The hippocampal electrode is designated by a red square. B) Components plotted also

indicate 10 is noise and 7 and 9 are large transients. C) Removal of those four components still

exhibits ERP response as indicated within RHD2 and RHD3, primarily from component 5 in A,

which seems to be across medial temporal lobe electrodes.

A

B

C

Components of ICA from (A) plotted over time

Time(s)

Hippocampal depth ERPs after removing EOG components

Time(s)

Vo

ltag

e (µ

𝑉)

Vo

ltag

e (µ

𝑉)

Components of ICA contributions by electrodes

50

Full ERP response of P3 shows that the ERP is not in its entirety a result of the saccade

transient measured in the EOG electrodes (Figure 20). Consistent with this, when the components

contributed by the EOG (components 6 and 7 from Figure 21A and B) are removed, the ERP peak

and trough in MTL electrodes are still evident (Figure 21C). It appears that the transient spike is

visible in multiple electrodes including even those within CSF electrodes (RAD2), and white

matter suggesting that its presence in intracranial electrodes is due to volume conduction. The ERP

within the MTL with and without the subtraction of the EOG component is not likely purely

volume conduction since waveforms in proximity to the hippocampal electrodes are not identical

in shape or magnitude. Further work on directionality and origin of the response is needed,

however for the purposes of this project, it is important to note that the ERP with peak and trough

for stimulation exists within the hippocampus and other MTL structures.

The intracranial results demonstrated qualitative evidence of an evoked response in 8/10

hippocampal electrodes with significance in four electrodes. Finally, regarding Patient 4,

notwithstanding the poor eye tracker calibration, where at one point, the eye tracker visibly

switched from tracking the left eye to the right eye, is still presented and included in the results

presented. Quantitatively, the four electrodes that displayed significant peak/trough (P/T) had

latency events at 60ms (P), 46.6ms (P), 68 (P) and 199.6 (T) for Patient 2, 3, 5 and 6 respectively.

These results suggest that the response can be detected and is saccade-dependent.

Furthermore, a likely root cause of the significant variability in ERP that we observe arises from

eye tracker related issues. This will be addressed in the Future work section. Since timing and

phase reset was important for the plasticity mechanism described and motivation for electrical

stimulation we needed to disentangle the putative mechanism underlying the phase ERP and phase

concentration we observed. This required further analyses as described below need to be

performed.

3.5.2.2 Intertrial Phase Coherence

The observed increases in ITPC appeared greatest within alpha/theta frequency bands in

individual patients (see Figure 22 A) as well as across subjects (see Figure 22B). Results

demonstrated in Figure 22B replicate the concentration findings from our previous work [122].

51

Figure 22 | Individual ERP and overall average in single target: A) P3 hippocampal electrodes

and ERP responses with phase concentration showing significance in theta range with significance

after FDR correction (q=0.05 and q=0.01). B) The overall response indicates pronounced

concentration in theta/alpha frequency and the overall average response from all patients showing

reset and subsequent peak and trough.

A

B

Time(s)

Fre

qu

ency (

Hz)

Average hippocampal ERP and ITPC

Time(s)

Fre

qu

ency (

Hz)

52

3.5.2.3 Spectral Power Analysis

Inn all hippocampal electrodes, pre-and post-saccade power was not significantly different

(p=0.12) even with only significant response electrodes were analyzed (p=0.057), although it

trended to the same result (Figure 23). Of note, in subsequent discussion with a lab mate it was

suggested that a parametric analysis using pairwise t-test should be used to compare pre-and post

power. This exhibited the opposite effects, however the relationship between ERP power and

individual trial power seemed uncorrelated. This data suggests that the response here may be

caused by a phase reset [125].

Figure 23 | Theta power pre-and post-event onset: The power of the theta band (3-8Hz) shows a

trend in the significant electrodes to be indicative of a phase reset.

These initial data suggested that stimulation based on the peak and trough timing would be

feasible since ERPs could be obtained, and corrected recognition memory performance was below

ceiling and above chance. Given that we had achieved our stated requirements to perform saccade

dependent closed-loop stimulation we next proceeded to test/pilot the closed -loop stimulation

platform that we had developed.

Pre Post Pre Post

Theta power pre-and post saccade in events and ERP

Pow

er (

𝒖𝑽

2/H

z)

53

3.6 Feasibility of Closed Loop Stimulation Methods

3.6.1 Participant

One sixty-eight-year-old female patient with medically refractory epilepsy underwent

surgical implantation of subdural surface electrodes and stereotactic depth electrodes to localize

epileptogenic regions.

3.6.2 Stimuli

Stimuli were used from Dr. Katherine Duncan’s Memory lab, like the previous task

iteration using natural scenes for visual exploration with semantic targets embedded.

3.6.3 Stimulation while performing behavioural task

The behavioural task and analysis were the same as a previous task (section 3.4), except

stimulation was now delivered during the encoding blocks. The participant was presented with a

demonstration of the experiment and 40 scenes to search with 10 memory scenes tested. The

calibration session of the eye tracker proved insufficient to obtain significant peak and trough

timings, but the peak and trough times exhibited were extracted.

To ensure optimization of timed stimulation, the intent was to stimulate on peak and trough

for two reasons: 1) electrode location can switch the polarity of the signal, and 2) some locations

of hippocampus peak may be beneficial to encoding while trough may be beneficial to recall and

vice versa [97], [122]. Additional control stimulation phases include random stimulation (Gaussian

timing based on patient fixation rate) and sham (no stimulation). These timings were determined

during the calibration session.

The calibration of stimulation intensity was modified (See Discussion), and stimulation

was performed during this experiment to demonstrate the feasibility of performing closed-loop

stimulation with our setup. Previous studies have determined intensity through stimulation that

does not elicit after-discharges [147], [149]. Stimulation intensity in this protocol was calibrated

using pulse widths of 300s and 60 single pulses at 1Hz stimulation [161].

Once the proper timings and intensity were established, the behavioural paradigm was

performed with relevant timed electrical stimulation to eye movement. Two blocks of encoding,

Sem1 and Sem2 each with 40 scenes were performed separately. Each of the encoding sessions

was followed by retrieval block (80 scenes: 40 novel and 40 scanned). Stimulation timings were

54

the same for each set of 10 scenes. However, the order of these stimulation timings was shuffled

through all iterations (see Table 4). Stimulation was only done during the encoding of scenes. This

task is identical to that described in Section 3.4.3 but included shuffling stimulation during the

encoding session.

Experiment Scenes 1-10 Scenes 11-20 Scenes 21-30 Scenes 31-40

Patient One Semantic One Peak Trough Random Sham

Patient One Semantic Two Sham Random Trough Peak

Patient Two Semantic One Trough Random Sham Peak

Patient two Semantic Two Peak Trough Sham Random

Table 3: Stimulation Permutations example with two patients

We piloted a full closed-loop stimulation session the detection of saccade termination was

sent to the behavioural software (Presentation™, Neurobehavioral Systems) which then signalled

our purpose-specific Arduino to trigger the stimulator. Bi-polar hippocampal electrical stimulation

was delivered through an Ojemann Grass Stimulator™, triggered with the appropriate stimulation

protocol (peak, trough, sham, random timing). iEEG data was recorded using a NeuraLynx™

amplifier system at a sampling rate of 16 kHz (see Figure 24).

55

Figure 24 | Closed-loop stimulation setup

3.6.4 Behavioural Data

Of import is that it is difficult to analyze corrected recognition for this patient, as hit rate

includes all scenes, while stimulation type is different in these scenes. Therefore, hit rate on scenes

and target memory based on scenes and stimulation types were analyzed.

3.6.5 Eye Tracking Data

The eye tracking data was poorly calibrated in the test session. Fixation rate was extracted

for the purposes of determining stimulation timing.

3.6.6 Intracranial Data

3.6.6.1 ERPS

To determine peak and trough times, both the raw ERP and theta filtered (second order

bandpass Butterworth filter 3-8Hz) were compared and the average times between those signals

were used to stimulate. CCEPs were analyzed posthoc to determine that the stimulation was indeed

delivered, and that there was a physiological response in surrounding electrodes ensuring that our

56

stimulation indeed altered network activity. The ERP for the nearest electrode was obtained by

taking the mean of these stimulation-onset epochs across all trials from experimental blocks (Sem1

and Sem2). Significance was performed as before using non-parametric polarity shuffling on the

ERP after the stimulation; large magnitude transient had occurred.

3.7 Results Feasibility of Closed Loop Stimulation (n=1)

3.7.1 Behavioural Results

3.7.1.1 Scene Memory:

Behavioral results demonstrated that the recognition was overall was similar to other

patients with corrected recognition at 0.48 The breakdown based on stimulation type is presented

in Figure 25, showing some trend towards differences between stimulation types.

Figure 25 | Scene memory based on stimulation: Seems to indicate slight trent for eye

movement locked stimulation improvement.

3.7.1.2 Target Memory:

Target memory for this patient was quite low, hovering just below chance at 0.49. When

split between stimulation types, it is important to note that the results are entirely different from

the scene results, which may be due to the total number of events.

Peak

Hit

Rat

e (%

)

Trough

Random

Sham

Scene hit rate by stimulation type

57

Figure 26 | Target memory results as a function of stimulation timing

The hit rate for scene recognition based on stimulation type was greater for saccade locked

stimulation than non-saccade locked (no stimulation or random) by 7.5% (Figure 25), although,

target memory may have been reduced (Figure 26). Even though there was a small sample size

and other non-ideal conditions, the results suggest that memory may be impacted, however further

testing is required.

3.7.2 Intracranial Results.

3.7.2.1 ERPs

Based on the ERP, electrode HD22 was selected to be stimulated (see Figure 27). Post-

operative imaging confirmed its location within the hippocampus at the junction of the

hippocampus and amygdala. From these analyses the final stimulation parameters selected were

300-microsecond pulses, five pulse stimulation with peak time at 196 and trough time at 316ms.

Peak

Hit

Rat

e (%

)

Trough

Random

Sham

Target hit rate by stimulation type

58

Figure 27 | Calibration hippocampal ERP demo: ERP from calibration filtered in theta

frequency band (3-8 Hz) that we used to time stimulation.

It is important to note that the eye tracker calibration session in determining timings for

stimulation was not optimal which may be factors in the later peak and trough timings compared

to the previous task, as well as the ITPC not being as pronounced, and therefore not shown. It

should also be noted that during calibration of intensity through the hippocampal electrode, the

patient reported seeing a bright white light that might be thought to be phosphine usually from

stimulation over visual areas and not temporal [190]. 3mA stimulation was eventually used as it

elicited a response in surrounding electrodes.

The CCEP from the subsequent stimulation can also be analyzed to see that there was a

significant response due to stimulation of the surrounding electrodes (Figure 28A). Different

stimulation types seemed to produce different amplitudes in the evoked response (Figure 28B).

Calibration hippocampal ERP for sim

Time(s)

Volt

age

(µ𝑉

)

59

Figure 28 | CCEP from stimulation: Examples showing that the CCEP of a nearby MTL electrode

is significant from electrode stimulation overall A and that different stimulation types may

introduce different amplitudes of evoked responses.

Time(s)

Time(s)

Stimulation CCEP

CCEP By Stimulation Type

A

B

Vo

ltag

e(µ

V)

Vo

ltag

e(µ

V)

60

3.7.3 Adjustments for Final Task:

Based on the results of this feasibility study of closed-loop stimulation further task modifications

were implemented:

1. Target adjustments to improve target memory and increase number of saccades.

a. Increasing difficulty of target embedding should increases saccades in the scene.

b. Adding multiple targets should increase saccades in the scene.

c. Blocking target presentation in scene memory should increase target memory.

d. Removal of “don’t know” response in memory will provide more responses to

analyze and should increase target memory.

2. Using an invisible tracking (no cursor on screen) to decrease participant distraction and so

participants are not focused on changing cursor colour should increase target memory and

saccades.

3. Increasing scene presentation resolution will increase search size and should increase

saccades.

4. Adding an additional block of testing (Sem3) should increase number of events to provide

more trials for stimulation and memory analysis.

In what follows we describe the results for three iEEG patients and two EEG patients that were

tested with this modified task incorporating the above modifications.

61

3.8 Final Task Iteration Methods

3.8.1 Participants

Subject Implantation Gender Handed Imaging Age (years)

P1 Depth F R Previous Resection left

temporal, hippocampus

intact

54

P2 Depth M R Normal 22

P3 Depth F R Subtle Volume Loss in

Left Hippocampus

33

S1 Scalp M R NA 27

S2 Scalp F R NA 22

Table 4: Patient Demographics Multiple Targets. P refers to iEEG patients, while S refers to subjects

undergoing scalp EEG recordings.

Three iEEG patients and two EEG patients were studied with the new task. The two

epilepsy patients from the EMU with scalp electrodes (EEG EMU patients), were included to

provide evidence that the target memory performance is similar within the epilepsy population

within the EMU. These scalp EEG patients were also tested on a slightly varied version of the

same task.

3.8.2 Stimuli

Stimuli were used from Dr. Katherine Duncan’s Memory Lab. Larger resolution (1400 x

1000) images that occupied a larger area of the screen were used to increase saccades. In total, 375

images were collected. Two targets different from the studies already described herein, were

embedded in the scenes. The SSIM used to embed targets in the scenes was based on the same

level of similarity for both targets. To utilize the same absolute similarity value across targets and

scenes the minimum SSIM value of the maximum similarity of the two targets was used as the

value for the most challenging placement in that scene. Subsequent target placements for both

targets were placed by decreasing the SSIM by 0.05 and the next placement had to be more than

10 of the target resolution widths from a previously placed target. This would ensure minimal

62

clumping of targets, requiring subjects to saccade to find them. Additionally, since multiple targets

now existed, based on lab testing in two volunteers, scenes where all four targets were found within

two seconds of presentation, were also removed. Of the 375 scenes, 290 were used in the protocol.

3.8.3 Behavioural Procedure

The behavioural procedure was like previous iterations in the sense that participants would

search through scenes and try to find the targets in an encoding block and would be tested formally

in a retrieval block. However, a sample of 10 images was not presented. Instead, two videos with

one scene each were presented to provide the context of what each target was. Participants were

instructed to encode a deep memory of the target to the scene. In the video, the same two scenes

were shown with both sets of targets (Bunny and Coyote) separately embedded in a blocking style

in the same manner as the task.

Slight differences between the two patient groups existed for encoding. iEEG patients were

tested identically to previous iterations where 40 search scenes were presented followed by 80

memory scenes (40 searched + 40 novels). This protocol involved four test blocks. The Demo was

used for the calibration session to introduce the task and obtain future stimulation timings, while

Sem1, Sem2 and Sem3 were used to collect memory performance data. None of the tested subjects

were able to complete all three test blocks based on the nature of testing in a clinical environment.

Due to the transient environment and clinical needs of the patients the task was designed to be

modular.

The scalp EEG patients were tested in a slightly different blocking mechanism for purposes

of investigating delayed memory in a separate project. However, their immediate retrieval test

described in Table 5 was a similar metric to performance in target memory. During encoding, they

were presented with 42 images followed by 28 images to test memory. This was done in three

back-to-back sessions, where short breaks could be taken for things like sips of water. Overall,

they saw 126 search scenes and were tested on 84 scenes (42 searched and 42 novel). Scenes

presentations were subdivided into blocks of 42 denoted ABC. For balance, scenes and targets

were presented in the order shown below. This ensured, that each subject was equally likely to see

the scene as a search scene or test scene in any block, with either target. Only the results for the

two subjects in the immediate test are presented (Table 5).

63

Subject and Group Search Immediate Delayed 24-Hour Delay

SUBJECT 1 ABC A=Search

D=New

B=Search

E=New

C= Search

F=New

SUBJECT 2 DEF D=Search

A=New

E= Search

B=New

F= Search

C=New

Table 5: Scalp Electrode Individuals and Respective Scene Presentation in Task.

3.8.4 Behavioural Data

The same metrics of performance-corrected recognition and target memory were analyzed

in the patient as descibed in other trials presented in this thesis. One of the intracranial subjects

was presented with a task whereby the retrieval stage still had the unsure response for target

memory. Therefore, the intracranial response which is same during encoding is shown, but their

behavioural performance is not presented.

3.8.5 Intracranial Data

The same intracranial analysis was run as done in section 3.4.6. However, due to the small

sample size, the statistics for power were not measured.

3.9 Results Final Iteration Multiple Blocked Targets

3.9.1 Behavioural Results

3.9.1.1 Scene Memory:

Patients performed above chance-corrected recognition at 0.61 (p=0.002; See Figure 29).

This included only participants who completed retrieval without the unsure option for target

memory.

64

Figure 29 | Final task memory performance for scalp(n=2) and iEEG (n=2): Corrected

Recognition for Individuals with Epilepsy with iEEG and scalp EEG recordings and B) Target

Memory for Individuals with Epilepsy with Intracranial and scalp electrodes. Performance is

above chance and statistically significant Chance is orange and ceiling is purple.

3.9.1.2 Target Memory:

Target memory was above chance at 0.60 and was statistically significant (p=0.009).

A

B

65

3.9.2 Intracranial Results.

3.9.2.1 ERPs

In one patient, the task and the triggers were not transmitted and thus unfortunately EEG

was not analyzed. Since the encoding task is the same for P1 who was presented with the unsure

response, their ERPs are shown. The ERPs similar to Section 3.4.6 are visualized to indicate

extracted peak and trough times that could be compared to the test timings of the ERP (Figure 30).

Figure 30 | Final task iteration ERPs: Intracranial ERP Response for Multiple Targets: Peak is

denoted by a green * and trough by a red *.

3.9.2.2 Intertrial Phase Coherence

Intertrial phase coherence in the two subjects tested showed theta phase concentration with

a large concentration in the higher frequencies as well. This is consistent with the transient

occurring before event onset (See Figure 31). It is worth noting that the theta phase concentration

still exists in these two subjects but is less prominent than the higher frequencies. The overall

response was found to have peak at ~46ms and trough at 110ms which is again in line with

previous individual and average results from the single target task.

P1 LH P3 RH1 P3 RH2

66

Figure 31 | Final task ITPC: A) Individual ERP and ITPC responses from subjects in multiple

target iteration. B) Average overall response where beta seems pronounced due to spike but theta

is still present.

From these data we conclude that the task modifications increased memory and saccades:

1) Both scene and target memory were significantly above chance, so the memory

task has been effectively implemented;

2) intracranial ERP was still evident with more saccades from which we can stimulate

B

P3 RH2 A P1 LH P3 RH1

Fre

quen

cy (

Hz)

Average Hippocampal ERP and ITPC

Time(s)

Time(s)

Fre

quen

cy (

Hz)

67

4.0 Discussion

This thesis describes the creation of a behavioural task that tests recognition and associative

hippocampal-dependent memory, which is easily learned and performed in a short period of time.

Our previous findings [122] coupled with our understanding of theta physiology guided our

selection of temporal windows for stimulation using saccadic eye movements, which are known

to reset hippocampal theta oscillations. We reproduced the ERP and the phase alignment from a

previous behavioural task [122]. Not all patients exhibited the ERP, but this might be due to

minimal events due to technical issues with the eye tracker's calibration or finding of the target too

quickly, rather than the absence of the ERP. In light of these findings, the importance of the reset

is discussed, and potential reasons for its presence or absence are presented in terms of timing

stimulation effectively. Additional stimulation parameters are discussed based on the intracranial

response to stimulation in terms of the pattern of stimulation and stimulation intensity. Finally, we

discuss future work that will utilize this stimulation paradigm.

4.1 Behavioural-Recognition, Associative Memory and Eye Movements

We strove to create a task that would improve memory with the ability to detect both

increments and decrements resulting from DBS. Our research design specifically aimed to avoid

a ceiling effect that is sometimes the case with memory paradigms (for example see the Sternberg

probe [116]). The ceiling effect is demonstrated by a lack of false alarms in a behavioural iteration

of a task [191]. The ability to analyze both increments and decrements of performance

modification are essential since both stimulation-induced modifications have been apparent in the

literature [149], [192].

Our methodology was atheoretical, such that no matter which hippocampal memory model

one subscribes to, the task would detect memory modulation. For example, research has shown

that the hippocampus is critical for explicit recognition [51]; however, others have shown that

recall is crucial for hippocampal memory but recognition is not [18]. Alternatively, in a patient

with bilateral hippocampal lesions, recognition was only mildly impaired, but types of associations

and relations between the stimuli were hippocampal-dependent [32].

One can use the type of target associated with the scenes to investigate recognition and

association memory. As eye movements are critical to the motivation of the project, merely placing

these targets in the centre was unlikely to elicit many saccades. Although a number of eye

movements are related to memory strength [26], there are conflicting arguments about the role of

68

eye movements in hippocampally dependent memory [193]. Therefore, to cover many

hippocampal-dependent memory theories, our task was designed to consider recognition,

association and visual exploration. For the visual search, semantic targets were chosen to be

embedded within the scenes. Embedding a bright yellow bird compared to a grey cat in the scenes

to enable similar camouflage of the targets and difficulty in search was challenging, so new targets

were selected.

In the final task iteration, a constant quantitative metric was used to embed the targets in

each scene: two new targets with deeper contextual associations were chosen. These deeper

associations were designed to consider for example, where the rabbit might hide, or the coyote

may capture the bird. This was achieved by showing participants video clips of each target

character to provide the context of targets to scenes – the context being an essential element of

theta and hippocampal memory [194]. However, it is possible that some of these semantic

comparisons may actually interfere with episodic retrieval [195]. This task alone design

contributes substantially to the current body of DBS literature by offering an alternative to word

recall or virtual space exploration [148], [149], [155], [196].

To ensure the suggested closed-loop stimulation could modify performance, a baseline

performance for the task was necessary. Multiple iterations were required to assess performance

on the task and implement changes before stimulation could be applied. Through this process we

ensured that even healthy controls were not at ceiling performance for scene recognition because

visual search tasks demonstrated low false alarm rates [191]. While accuracy was above chance in

the healthy control testing, modifications were made to improve performance for the patient

population. This included selecting of only a semantic set of targets, the addition one second of

scene exploration to provide more exposure to a stimulus thus increasing the strength of its

encoding [197], and clarifying instructions.

Healthy controls performed better on the first iteration of the task than epilepsy patients.

This is consistent with the literature as patients with temporal lobe epilepsy (TLE) can exhibit

cognitive impairments both in short and long-term memory [2], [134]. In fact, a phenomenon

called long-term accelerated forgetting is observed in patients with epilepsy who tend to forget

more rapidly compared to matched, healthy controls [198].

Our corrected recognition was well above chance with room for modification to both

decrease or increase memory with stimulation. However, the saccade rate, potentially due to

calibration concerns and general movement within the scene, presented issues. It is important to

69

note that even though these were issues for stimulating, patients still demonstrated a saccadic ERP

and theta phase concentration. Although the target memory was not originally statistically above

chance in patients, modifications were made to increase eye movements and behavioural

performance. Modifications included using multiple targets in the scenes, blocking the scenes so

patients knew which targets they would be searching for, and removing the “not sure” response.

This resulted in target memory increases to statistically above chance in the patient population.

Additionally, the multiple targets in the scene may have encouraged visual exploration and

saccades. This is important for providing more events from which to extract the ERP and more

opportunities to stimulate based on the eye movement within the scene.

We have confirmed that our memory task will adequately test recognition memory and

associative memory in patients with epilepsy. Furthermore, we have shown that it generates

enough saccadic eye movements, thus providing enough events to both obtain the ERP for

stimulation timings and stimulate accordingly. Finally, the environment is controlled enough that

stimulation effects can be investigated. An overview of goals of the designed task are shown in

Figure 32.

70

Figure 32 | Task for assessing memory: Overview of the task needed to assess hippocampally

dependent memory, incur eye movements and be sensitive for trial specific DBS modifications.

HIPPOCAMPAL

DEPENDENT

CONTROLLED

ENVIRONMENT

NATURALISTIC

BEHAVIOUR

TASK

SENSITIVE

ENCODING

TASK CHANC

E

100%

RECAL

Task allows for naturalistic

visual exploration of scenes

Task includes controlled short

events to assess effects of DBS on

a trial-by-trial basis

Visual search memory assesses

several hippocampal dependent

functions: 1) explicit recognition,

2) associative relations, and 3)

implicit relational scan path

71

4.2 Intracranial Response and Stimulation

In DBS, there is a large parameter space from which to select how to stimulate. This

includes, but is not limited to, electrode selection, frequency, pulse width, timing, duration, and

intensity. Current protocols do not necessarily address fundamental questions or optimizations of

this parameter space, although, one animal study did show tuning the frequency of stimulation

could modify behaviour [199], suggesting that these parameters are essential. To date, rather than

addressing stimulus characteristics, the focus in human trials has been on which structures to

stimulate. Memory modifications secondary to stimulation have been found in the MTL structures,

such as the entorhinal cortex and hippocampus, where hippocampal stimulation has mostly caused

impairment [146]–[148], [196]. However, there are other critical questions about stimulation. In

this project, these were narrowed down to:

1) Where do we stimulate?

2) When do we stimulate?

3) How do we stimulate?

4) What intensity do we stimulate at?

Although the hippocampus is important to memory, the question of where to stimulate is

not trivial. For example, stimulating the entorhinal cortex and amygdala and not the hippocampus

have been shown to introduce memory improvement [149], [200]. Even within the current

behavioural task, other MTL structures and not just the hippocampus demonstrated ERPs from

eye movement. Based on the previous literature, we suggest that the hippocampus may be an

optimal target for stimulation to modify memory, but we have not optimized other stimulation

parameters. The hippocampus is a hub that integrates sensory information and is thought to

communicate with neocortical structures, like the prefrontal cortex, to bind representations [201].

The timing of communication within and between structures might, therefore, be the key. This

emphasizes that we will stimulate in the hippocampus, but the primary focus of this work on when

to stimulate.

The issue of timing can be addressed by looking at neuronal rhythms from intracranial

recordings. Specifically, we discussed the importance of theta as a cognitive rhythm for spatial

navigation and cognition in rodents [46]. However, although theta might be related to memory[72]

theta in humans is evanescent[119]. Humans also explore the world differently compared to

rodents. Theta has been recorded in humans intracranially while walking [120], but while sitting

72

in bed, theta can be phase aligned to eye movements [122]. However, theta arising during walking

did not address the hypothesis that walking is associated with increased periodic saccadic eye

movements, and thus, sequential theta resets resulting in theta ‘oscillations’. Specifically, ERP can

be elicited by eye movements and aligned with phase concentration.

It could be argued that the importance of visual stimuli is how one should time stimulation,

brain oscillations for memory are often compared against visual onset[75]. Lacruz et al. (2010)

showed that if stimulation is timed to the visual onset of the image, memory is impaired [146],

while Inman et al. (2017) showed theta-burst like stimulation aligned to visual onset might

improve memory when stimulating in the amygdala [200]. However, perhaps to indeed consider

timing stimulation to a visual stimulus, it is essential to understand how different presentations of

visual stimuli (i.e., image onset compared with saccadic eye movement) to the retina can generate

a hippocampal ERP [184].

There are a few suggestions for what may elicit a recorded ERP that includes an induced

response (increase in firing), an additive response (specific neuronal assembly) or phase reset

(ensuring timing of stimuli arrival) [91], [202]. According to Shah’s model, and a manuscript in

progress based on similar visual search data, the lack of spectral power change suggests the

saccade response can be characterized as a pure phase reset [125], [184] to optimize the timing of

the incoming stimulus to encode information [203]. This is important, as we have been able to

show a similar hippocampal ERP with 1/10th the number of saccades and shorter viewing time.

Such task optimizations enable us to present more scenes in a shorter period, which is very

important in a relatively unpredictable hospital environment where patients may have visitors,

medications or clinical events.

What might be the source of the hippocampal ERPs be, and how are they related to eye

movements? Since there are no prominent anatomical connections between the oculomotor system

and the hippocampus, it is likely mediated by cortical-cortical connections [22]. Consistent with

this, there are connections between frontal eye fields (FEF) of the dorsolateral prefrontal cortex

(dlPFC) and the hippocampus, which are responsible for generating voluntary saccadic eye

movements [22].

In the hippocampus, ERPs to eye movement are also known to have a transient at onset

which we also showed, perhaps due to saccadic eye movement [185]. The latency of the saccadic

response observed in our research is also similar to the peak firing time of saccade direction cells,

which predict both past and future saccade directions in the entorhinal cortex [204]. A similar

73

saccade-aligned response involving post-saccade phase clustering has also been observed in the

V1 area of macaques [205]. Saccade-related responses have been observed in human and macaque

hippocampi and V1 for saccades observed during salient images, and for saccades performed in

dark environments, although there is evidence that the phase clustering differs based on the

saliency of the incoming visual information [205] [122] [138]. While the grand average saccade

ERP observed here was similar to previously reported saccadic responses, there was considerable

variability between patients, which may be explained by the eye tracker calibration or individual

differences in electrode positioning in hippocampal substructures [51].

The superior colliculus [206] and the FEF [207] are suggested to have voluntary control

over eye movements on a similar temporal scale to the observe ERP. This is further supported

where FEF neurons in the monkey peak before saccade onset and continue until after saccades end

[208]. It is also interesting to note that the phase concentration and ERPs seemed to be evident in

other MTL electrodes such as the parahippocampus (data from our study not presented). The

parahippocampus is known to play a role in visual search behaviour. For instance,

parahippocampal damage has been previously shown to alter visual search behaviour, leading to

increased number of saccades and visual neglect [209] [210]. Specifically, theta rhythm and this

reset are seen to have a connection with memory consolidation both in animals and humans,

whereby a system may reset to appropriately encode incoming sensory information at optimal

times of the theta rhythm [46].

Based on the significant phase concentration and the lack of power increase in individual

trials, we suggest that the saccade onset elicits a pure phase reset. But why is this important?

Multiple brain states seem to exist for memory (encoding and retrieval), attention, and

responses to sensory information. Schroeder et al. (2009) suggested that there are two states of the

brain: an attentive state that can be entrained by temporal relations and enhance slow oscillations,

and a vigilant state for broader changes in the surroundings [211]. Additionally, to multiple brain

states there seem to be two prominent theta oscillations in the human hippocampus [72]. The

higher theta frequency seems to correspond to align visual exploration and is most prominent in

this reset response [212]. These studies in humans suggest this response is like the animal

exploration that involves theta oscillations to encode place and relevant stimuli. A lower frequency

theta oscillation in the delta range is suggested to be a different brain state optimal for processing

stimuli. In fact, the posterior and anterior hippocampus even have different frequencies of

oscillations [213]. Different hippocampi are also dominant for different memories: the left is

74

contextual while right may be more spatially-oriented [37]. Therefore, visual or movement

exploration is not the only consideration for these different states. For example, phase reset has

been proposed as a timing mechanism for information [87], and potential error coding [214].

Previous studies in rats have shown that the frequency in which the phase reset occurs is dependent

on the specific afferent pathways stimulated [93]. This suggests that the multiple states and the

reset observed following saccade-onset may involve many afferent pathways.

Different structures such as the superior colliculus and pulvinar are involved in oculomotor

decisions and voluntary movement that may indirectly integrate within the hippocampus [215]. In

fact, the amygdala and hippocampus are considered to be potential integration centres for

neocortical sites [42]. This is further supported by the ascending pathway and suggested input

pathway to the hippocampal formation from the reticularis pontis oralis thought to drive theta and

early sensory integration [216]. This ERP could also be a medial temporal lobe response of

integration, which suggests that the amygdala may coordinate a visual response and emotional

relevance with the more extensive cortical networks rather than specifically processing it [215].

Finally, it is important to note that attention or task relevance, which the pulvinar is thought to

influence, may be relevant to the resulting ERP as noted by previous work suggesting saccades in

the dark were not associated with phase alignment [122] although the ERP was present in

macaques [138].

Oculomotor decisions of where and when to look can involve many brain regions, which

may benefit from such timing mechanisms as phase resetting for synchronization purposes (for

review of eye movements and hippocampus, see [217]). A potential mechanism for these saccades

might be the olivary nucleus, considered the master clock with a stable, sinusoidal frequency

between 1 – 4Hz [218]. This frequency has been shown to reset to excitatory stimulation and has

been suggested from a computational model of saccades [219], [220]. There is also evidence of

connectivity in memory consolidation between the hippocampus and the prefrontal cortex which

can include coordination for decision making and even anxiety (a form of dealing with conflict)

[221]–[224]. Also, atrophy of the prefrontal cortex, beta-amyloid plaques, and reduced sleep

consolidation have been shown to impair hippocampal declarative memory [225]–[227], further

suggesting the importance of the hippocampal-prefrontal interactions for memory processes. There

are also thalamocortical projections to the prefrontal cortex and the hippocampus involved in goal-

oriented spatial navigation [228]. In fact, we have postulated that an additional reset between

thalamocortical networks and anterior cingulate cortex to coordinate large networks across goal

75

and attention oriented tasks [188], [189]. The posterior cingulate cortex (PCC) on the other hand,

is considered to be an anatomical pathway through which the MTL may receive spatial

information, which is further supported by our studies showing a decline in declarative memory

caused by reduced connectivity between the PCC and the hippocampus [229].

Therefore, many connections between hippocampus and neocortex along visual sensory

information pathways require varying synchronization depending on the task, which may be

achieved through a phase reset. Phase resetting within the hippocampus may also be necessary as

it is known that neurons phase lock delta, theta and gamma oscillations [230]. We suggest that

there may be many benefits and causes to this reset [188], [189] and its importance in memory.

Stimulation of the hippocampus locked to saccadic eye movement, and thus the theta

rhythm, isolates the specific parameters for stimulation location and timing. The other two critical

factors include the stimulation pattern and the stimulus amplitude. Again, stimulation patterns can

vary between continuous square waves, with pulses at varying frequencies. Open loop stimulation

has varied parameters such as voltage or current controlled stimulation with varying pulse widths

and frequencies [231]. Since we are using the idea of neuronal plasticity and timing, the previous

animal literature suggests that, physiologically, theta or low delta burst stimulation is optimal for

inducing LTP in the hippocampus [97], [232], [233]. Theta burst stimulation itself varies in the

specifics of the frequency, number of pulses in a train and number of trains, but it is usually

implemented as a high-frequency stimulation in trains delivered at theta frequency. In-vitro, in-

vivo and even in humans (theta burst stimulation of fornix and amygdala locked to visual stimuli)

results in potentiation and improvements in memory [97], [152], [200]. In rodents, multiple pulses

were required to elicit such a response [92]. In our previous work with macaques, a train of at least

four 0.1ms pulses was required to elicit a CCEP in the hippocampus [164]. For our protocol, we,

therefore, chose to use this theta burst stimulation locked to the eye movement with 5 pulses with

200 microsecond pulse widths aligned to the peak or trough of the ERP (see Figure 33).

76

Figure 33 | Theta burst stimulation example: Example of five pulse stimulation on theta rhythm

peak with 4 trains (which would be a result of eye movements saccade rate 4/s in our paradigm)

and 5 pulses per train at 100us each phase of the biphasic pulse.

The final parameter to be adjusted was the intensity of stimulation. Again, too low a current

may not cause an evoked response [164]. Literature suggests using a stimulation current level that

is subthreshold to after discharge generation or side effects within the patient [147], [149]. We

chose a different approach by using a metric currently designed for testing functional connectivity.

CCEP literature suggests that 60 pulses at 300us are enough to map out connectivity and

statistically significant evoked responses [161]. Although we had initially selected 1Hz, we

subsequently wanted to evoke a significant response using a theta burst type protocol. Therefore,

based on the saccadic rate for each individual patient, we obtained a personalized intensity. To

ensure appropriate stimulation parameters, CCEPs were used to ensure the stimulation has caused

an effect downstream of the stimulation. This saccade rate can be extracted from the original

saccadic rate during the calibration scene search of the behavioural task. To ensure that

potentiation was possible, we showed that CCEP arising from different stimulation timing elicited

different amplitudes to the CCEP. This can further be explored using the 5-biphasic pulse (0.1ms

pulse width). Previous literature suggests that 4-5 pulses should elicit potentiation in the

hippocampus in-vitro [234][235] and in-vivo [92]. Our previous work showed that more than three

2ms 0.1ms Time(s) 0 1

Vo

ltag

e(µ

V)

77

pulses of 0.1ms pulse width in fornix were required to evoke a hippocampal response [164].

Therefore, this stimulation pattern and intensity is not only physiologically relevant but matches

the same pattern that will be delivered during the memory task.

5.0 Limitations

5.1 Patient Factors and Stay Within the EMU

The participants of this study were individuals with TLE, and for this reason are being

implanted with iEEG electrodes, implying some potential dysfunction of the mesial temporal lobe

structures and cognition [3]. Such pathology implies neuronal and glial cell loss [236], which can

also have an effect on the amplitude of the hippocampal ERPs [106] and possibly our ability to

detect a phase reset. However, such patients often have only one diseased side, with an intact

contralateral hippocampus allowing us to investigate seemingly normal hippocampal

activity/physiology [85]. Such pathology results in patients with material specific memory

problems, with visual-spatial memory being affected if pathology is in the non-dominant

hippocampus, and verbal memory if there is a pathology in the dominant hippocampus. Yet despite

these cognitive effects it is important to note this task was administered to patients and they were

able to perform these tasks above chance, suggesting intact associational hippocampal memory

systems. However, it is well accepted that memory is a network phenomenon and other studies

have demonstrated that stimulating somewhere in the memory network may also improve memory

[151], [154]. Thus, our temporally specific stimulation of a highly connected hub, the

hippocampus, may also potentially provide the neuromodulatory effects to augment human

memory [155], despite the pathology present in the hippocampus [155].

Medications are another potential confounder as they: 1) affect the number of seizures a

patient is likely to have, 2) can degrade memory and concentration, and 3) the dosage prescribed

is often changed throughout the patient’s stay in the hospital to increase seizures for diagnostic

purposes. In general, seizures do not occur until many days after electrode implantation since

medication levels are often held constant immediately post-operatively. This represents an

opportune window for testing, and one that will pay attention to in future work.

78

5.2 The testing environment

In addition to the variability of the patients, the hospital environment is also a limiting

variable. It affects the ability to collect data in a controlled nature, specifically because patients

stay in either private, semi-private or quad rooms. The hospital environment, even in a private

room, has uncontrollable noise and distractions such as intercom announcements and interactions

with caregiving teams Distractions were noted in research documents, and patients tended to

remain focused. We attempted to mitigate interruptions by coordinating with technicians to ensure

testing did not overlap with dressing changes. Additionally, we developed a rapport with hospital

caregivers whereby they usually returned following behavioural task completion to ensure delivery

of care but also coordinated timings for participants to engage with research.

5.3 Technical/Experimental Complications

The quality of the eye tracking is important to the task as we extract the ERP due to aligned

eye movement events and stimulate based off real-time detection of online saccades. It is a primary

element of the project and could affect both the behavioural recording of eye movements and

timing of the ERP response. The current eye tracker had relatively poor calibration especially with

patients with glasses. Furthermore, the sampling rate of 120 Hz was problematic. With online

event detection, there was a limitation in terms of events that were detected as an online eye

movement. To lessen this issue with real-time detection the most reliable online detection was

monocular eye-tracking setting for detection. This however also introduced issues as was evident

in one participant (patient 4), where the eye tracker calibrated to the left eye during the

experimental session, switched to tracking the right eye. The results from this patient were that

that was no observed peak or trough in their ERP.

An additional issue with the eye tracker was that the sampling speed limied us to an 8ms

resolution of event detection. Thus, we may have missed events or sensitivity of response due to

temporal jitter. In a worst-case scenario, there maybe no detected ERP from which to obtain

stimulation timings which did happen in some patients. The averaged overall response from

patients that demonstrated a peak between 45-60ms and trough at ~110-130ms will be utilized,

which appears to be consistent with a manuscript in progress using thousands of events [184] A

high-speed 1kHz sampling speed eye tracker with optimal calibration in the field was also

acquired. which can detect eye movement fixation events within 32ms of event onset. If the timing

79

of the saccade detected peak/trough is <32ms, which is less than the online time of detection of an

event, we will stimulate as soon as possible. Although this stimulation will not be timed precisely

to peak, stimulation within 90 degrees of the detected peak and trough still exhibited a behavioural

effect in rodents, suggesting this slight delay might not be a concern [105].

It is noteworthy that the recorded ERP may not be isolated to the hippocampus alone and may,

in fact, be an ERP across the MTL. Thus, there may be other locations to investigate stimulation

or coupled stimulation. We are limited by the current stimulator in that we can only select one

electrode pair for bipolar stimulation. This is a minor limitation as we had only ever intended to

stimulate the healthy hippocampus. Although, it is important to note that in one study, bilateral

stimulation was required to impair memory [147]. Therefore, to ease this problem, a NeuraLynx™

stimulator which can stimulate eight electrodes is also being investigated for use.

In addition to hardware, the task implementation itself has some limitations that can affect

behavioural performance and the ERP elicited by eye movements. For example, visualizing a

cursor following eye movements on the stimulus presentation laptop's screen enabled a real-time

view of this eye tracker calibration, since it was of utmost importance to get appropriate

stimulation timing. However, many participants reported that the cursor distracted them from the

task. Additionally, the cursor was designed to turn red upon fixation on target to indicate that the

target was found. This colour change is was done to differentiate when the target was found as in

the original work with the visual search tasks, once a target is found, the trial ends [122]. In this

task, due to the relative brevity of scene presentation (4s) and desire to expose all participants to

the same duration of the scene, participants could still explore the scene after finding the target.

However, on some trials, some participants adopted strategies where they would remain fixated

on target to turn the cursor red or return to the centre of the screen to take a holistic view of the

scene for later memory test. Therefore, saccadic rate and dispersion of eye movements within the

scene were sometimes limited. For example, a large number of the saccades in this experiment

detected are 8ms in duration, which some researchersave even excluded from their analysis of eye

movements [237]. The number of eye movement events and their reliability is critical, since less

events can impact the overall ERP (see Figure 11). Additionally, if these eye movements and traces

are sporadic due to the calibration, one cannot investigate scan path of specific regions in the scene

or targets effectively. These limitations were mitigated through the removal of the visible cursor

to decrease distraction and embedding of multiple targets to increase saccadic exploration.

80

In using scenes as stimuli during the behavioural task, some patients may have other episodic

memories tied to the scenes that are out of our control. For example, one subject identified that

she had travelled to a number of venues and many scenes seemed similar to where she travelled.

This could impact the memory performance. This problem is inherent to the use of scenes as visual

stimuli. To mitigate this potential interference we selected a wide variety of scenes from different

settings to minimize participants’ prior exposure to scenes. We will also take note of which scenes

a subject has explicitly indicated they remembered due to previous exposure not as part of the task,

and potentially remove such scenes from anlalysis.

Other limitations of the task and software include errors in trigger synchronization of EEG,

and behavioural and eye tracking data. In software, since we are triggering in real time off eye

movements, it is important to consider conflict on the port such as the trigger for finding the target.

Thus, events triggered should be prioritized and limited only to critical events such as stimuli onset

and eye movements or appropriate pauses need to be considered to ensure no conflict on the output

port when synchronizing data. When triggers are not sent, data can not be aligned to obtain an

ERP or investigate subsequent memory effects. Therefore, a specific step-by-step protocol with a

table for notes will be be followed, and this protocol will contain common elements by which

every research team testing with intracranial patients must abide.

Only approximately ten TLE patients annually are available for this type of testing. While

other pilot stimulation studies have worked with as little as one and up to twelve participants [144],

[147], [149], [154], this presents an opportunity for cross-site collaborations. The intracranial

analysis has also been designed such that we can test within each individual patient to determine

the ERP and its significance.

6.0 Future Work

Future work will largely focus on the exploration of receptive states for brain stimulation.

Timing of stimulation and synaptic plasticity is only one of the ingredients required for successful

memory formation; the patterns of neural activity stored in the resulting memory trace must also

contain high-quality information about the encoded event. It has been suggested that there are good

brain memory states within the hippocampus for stimulation [155]. Cognitively, researchers have

identified a key factor that regulates the quality of hippocampal input: attention [238]–[242]. We

have also discussed attention briefly as it pertains to eye movements. Moreover, attention seems

81

to be a crucial bottleneck for memory formation, because people have a limited capacity to sustain

it throughout everyday activities [243], [244]. That is, attention waxes and wanes across seconds

to minutes and these fluctuations increase in dementia [245], [246]. These slowly shifting

attentional states could severely limit memory formation and current attempts to treat memory

impairments with DBS.

This hypothesis is strongly substantiated by two recent discoveries. First, memory formation

is robustly modulated by sustained attention. DeBettencourt et al. (2017) estimated attentional

states during memory formation by quantifying low-frequency (>3 second) shifts in response time

and found that images presented at peak attentional states were remembered over 15% more often

than those presented at low states [247]. The second discovery was that the impact of DBS on

memory depends on slowly shifting global brain states. Ezzyat et al. (2017) identified brain states

that were “‘good” or “bad” for memory encoding based on whole-brain spectral patterns [155].

They then showed that applying DBS during “bad” encoding states, enhanced memory formation

for words presented in the seconds following DBS termination, as though DBS broke people out

of persistent “bad” states. Conversely, applying DBS during “good” states impaired subsequent

memory formation. Although these results reflect a post-hoc analysis of an open-loop protocol,

they provide an important proof-of-concept for our closed-loop pursuit. They demonstrate the

importance of considering brain states when applying DBS and identify that relevant whole-brain

states fluctuate over many seconds.

Unlike the well-substantiated literature connecting plasticity in the current project to theta

phases, however, there are no simple physiological markers of attentional states. The power and/or

phase of delta, theta, beta, alpha, and gamma oscillations have each been implicated in attention

[248]–[251]. Analogously, the encoding states identified by Ezzyat et al. (2017) were related to

shifts across the spectrum of oscillations, with increases in higher-frequency and decreases in

lower-frequency power across the brain [155]. One aim of the future project will be to investigate

whether there are in fact receptive brain states to receive stimulation using a precision based

memory task and machine learning to classify and predict how stimulation affected memory

performance. Additionally, both the current thesis and this project specifically consider encoding

states of the brain. Both projects could also include stimulation done at encoding and retrieval of

information, as studies have shown the phase of rhythm is different for encoding compared to

retrieval of information[46].

82

Single unit neural recordings will enable recording of actual activation of neurons within

specific phases of the ongoing rhythm phases, lending further insight into the plasticity

mechanism. These neurons can be recorded to indicate subsequent memory effects or even used

for microstimulation [153] and have also been shown to be locked to theta to strengthen memory

[252]. With these 24/7 intracranial recordings, we can also investigate how memory is “replayed”

over a night sleep and consolidated [253].

Since the Eyelink1000 has been purchased, we can be reasonably confident in the eye-

tracking data obtained. For future work, electrical stimulation will be delivered in trains of

individual saccade rate/s for 4s with 2s breaks for a total of 60 trains. Intensity will not exceed

8mA [147], as an evoked response will likely be obtained at less than 3mA [164], [254].Also,

other metrics of eye movements could be incorporated, including looking at pupil diameter, which

has provided evidence for novelty [255], and even implicit scan path in recognition [23]. One other

way to use controlled stimulation and to test more robust visuals could be to employ virtual reality.

Perhaps other senses could also be investigated since nasal rhythms have been shown to entrain

theta rhythms [256]. Therefore, it may be a combination of senses at optimal times in hippocampal

consolidation that we might be able to simulate.

Finally, from the current intracranially collected data, we can investigate the intracranial

rhythms of both visual scene onset and eye movements to interrogate subsequent memory effects

and how the response can be characterized between forgotten and remembered items.

7.0 Conclusion

In conclusion, this proof of principle study has shown that we can record the desired

electrophysiological response at the time where we would like to stimulate. This memory

framework was designed to be hippocampally dependent and optimal for the hospital environment

in terms of being controlled and modular while enabling naturalistic viewing. This enables data

collection in a short amount of time. The ERP from this task was shown to be similar to the

previously found ERP on which this project was based. Also, we have shown that we could

stimulate in a closed-loop fashion, where even in preliminary non-optimal saccade locked

stimulation, preliminary memory modifications compared to random or sham stimulation overall

was demonstrated. This stimulation protocol considers physiologically relevant time, location and

pattern to optimally induce potentiation and behavioural outcomes as seen in animals. Specifically,

83

this timing is based on the theta rhythm which is considered the behavioural and exploratory

navigational rhythm in animals. The current project explores theta and eye movements further

through use of DBS to modify memory.

We believe that we successfully combined principles of hippocampally dependent

memory, theta rhythm, exploration and appropriately timed electrical stimulation in the

hippocampus. This discovery can be summarized by one of the last reviews of the late Howard

Eichenbaum, “The Role of the Hippocampus in Navigation is Memory [257].”

84

8.0 References

[1] G. S. Watson and J. B. Leverenz, “Profile of cognitive impairment in Parkinson’s disease.,”

Brain Pathol., vol. 20, no. 3, pp. 640–5, May 2010.

[2] B. Hermann and M. Seidenberg, “Epilepsy and cognition.,” Epilepsy Curr., vol. 7, no. 1, pp.

1–6, 2007.

[3] C. E. Elger, C. Helmstaedter, and M. Kurthen, “Chronic epilepsy and cognition,” Lancet

Neurol., vol. 3, no. November, pp. 663–672, 2004.

[4] “WHO | Dementia,” World Health Organization, 2017. [Online]. Available:

http://www.who.int/mediacentre/factsheets/fs362/en/. [Accessed: 22-Oct-2015].

[5] L. W. Chambers, C. Bancej, and I. Mcdowell, “Prevalence and Monetary Costs of Dementia

in Canada,” Alzheimers Soc. Canada, no. May 2015, p. 70, 2016.

[6] DARPA, “Restoring Active Memory,” Federal Business Opportunities, 2013. [Online].

Available: https://www.fbo.gov/spg/ODA/DARPA/CMO/DARPA-BAA-14-

08/listing.html. [Accessed: 06-Dec-2017].

[7] L. Luo and F. I. M. Craik, “Aging and memory: A cognitive approach,” Can. J. Psychiatry,

vol. 53, no. 6, pp. 346–353, 2008.

[8] W. B. Scoville and B. Milner, “Loss of recent memory after bilateral hippocampal lesions.

1957.,” J. Neuropsychiatry Clin. Neurosci., vol. 20, no. 11, pp. 11–21, Feb. 1957.

[9] Epilepsy Canada, “Epilepsy Facts - Epilepsy Canada.” [Online]. Available:

http://www.epilepsy.ca/epilepsy-facts.html. [Accessed: 25-Oct-2015].

[10] “WHO | Epilepsy,” WHO, 2017. [Online]. Available:

http://www.who.int/mediacentre/factsheets/fs999/en/. [Accessed: 09-Dec-2017].

[11] C. Helmstaedter, “Effects of chronic epilepsy on declarative memory systems.,” Prog. Brain

Res., vol. 135, pp. 439–53, Jan. 2002.

[12] L. R. Squire and J. T. Wixted, “The cognitive neuroscience of human memory since H.M.,”

Annu. Rev. Neurosci., vol. 34, pp. 259–288, 2011.

[13] R. Cabeza and M. Moscovitch, “Memory Systems, Processing Modes, and Components:

Functional Neuroimaging Evidence,” Perspect. Psychol. Sci., vol. 8, no. 1, pp. 49–55, 2013.

[14] L. R. Squire and S. M. Zola, “Structure and function of declarative and nondeclarative

memory systems.,” Proc. Natl. Acad. Sci. U. S. A., vol. 93, no. 24, pp. 13515–13522, 1996.

[15] M. Moscovitch, “Memory and Working-with-Memory: A Component Process Model Based

on Modules and Central Systems,” J. Cogn. Neurosci., vol. 4, no. 3, pp. 257–267, 1992.

85

[16] E. Tulving, “Organization of Memory: Quo Vadis?,” The Cognitive Neurosciences. pp. 839–

847, 1995.

[17] L. R. Squire, “Memory systems of the brain: A brief history and current perspective,”

Neurobiol. Learn. Mem., vol. 82, no. 3, pp. 171–177, 2004.

[18] H. Eichenbaum, A. P. P. Yonelinas, and C. Ranganath, “The Medial Temporal Lobe and

Recognition Memory,” Annu. Rev. Neurosci., vol. 30, no. 1, pp. 123–152, 2007.

[19] R. K. Olsen, S. N. Moses, L. Riggs, and J. D. Ryan, “The hippocampus supports multiple

cognitive processes through relational binding and comparison,” Front. Hum. Neurosci., vol.

6, no. May, pp. 1–13, 2012.

[20] B. P. Staresina and L. Davachi, “Selective and shared contributions of the hippocampus and

perirhinal cortex to episodic item and associative encoding.,” J. Cogn. Neurosci., vol. 20, no.

8, pp. 1478–1489, 2008.

[21] G. Buzsáki and E. I. Moser, “Memory, navigation and theta rhythm in the hippocampal-

entorhinal system.,” Nat. Neurosci., vol. 16, no. 2, pp. 130–8, Feb. 2013.

[22] K. Shen, G. Bezgin, R. Selvam, A. R. McIntosh, and J. D. Ryan, “An Anatomical Interface

between Memory and Oculomotor Systems,” J. Cogn. Neurosci., vol. 28, no. 11, pp. 1772–

1783, 2016.

[23] D. E. Hannula and C. Ranganath, “The eyes have it: hippocampal activity predicts expression

of memory in eye movements.,” Neuron, vol. 63, no. 5, pp. 592–9, 2009.

[24] Z.-X. Liu, K. Shen, R. K. Olsen, and J. D. Ryan, “Visual Sampling Predicts Hippocampal

Activity,” J. Neurosci., vol. 37, no. 3, pp. 599–609, 2017.

[25] I. van der Linde, U. Rajashekar, A. C. Bovik, and L. K. Cormack, “Visual memory for fixated

regions of natural images dissociates attraction and recognition,” Perception, vol. 38, no. 8,

pp. 1152–1171, 2009.

[26] A. Kafkas and D. Montaldi, “Recognition memory strength is predicted by pupillary

responses at encoding while fixation patterns distinguish recollection from familiarity,” Q. J.

Exp. Psychol., vol. 64, no. 10, pp. 1971–1989, 2011.

[27] M. Zhao, T. M. Gersch, B. S. Schnitzer, B. A. Dosher, and E. Kowler, “Eye movements and

attention,” Vision Res., no. 74, pp. 40–60, 2012.

[28] D. E. Hannula, J. D. Ryan, D. Tranel, and N. J. Cohen, “Rapid onset relational memory

effects are evident in eye movement behavior, but not in hippocampal amnesia.,” J. Cogn.

Neurosci., vol. 19, no. 10, pp. 1690–705, Oct. 2007.

[29] J. Jonides, R. L. Lewis, D. E. Nee, C. a. Lustig, M. G. Berman, and K. S. Moore, “The Mind

and Brain of Short-Term Memory,” Annu. Rev. Psychol., vol. 59, no. 1, pp. 193–224, 2008.

86

[30] L. R. Squire, C. E. L. Stark, and R. E. Clark, “the Medial Temporal Lobe*,” vol. 27, no. 1,

pp. 279–306, 2004.

[31] K. S. Giovanello, M. Verfaellie, and M. M. Keane, “Disproportionate deficit in associative

recognition relative to item recognition in global amnesia,” Cogn. Affect. Behav. Neurosci.,

vol. 3, no. 3, pp. 186–194, Sep. 2003.

[32] A. R. Mayes et al., “Associative recognition in a patient with selective hippocampal lesions

and relatively normal item recognition,” Hippocampus, vol. 14, no. 6, pp. 763–784, Jan.

2004.

[33] K. Romero and M. Moscovitch, “Amnesia : New theoretical insights,” pp. 1–27.

[34] H. Eichenbaum, “Hippocampus: Cognitive processes and neural representations that underlie

declarative memory,” Neuron, vol. 44, no. 1, pp. 109–120, 2004.

[35] N. a Suthana, A. D. Ekstrom, S. Moshirvaziri, B. Knowlton, and S. Y. Bookheimer, “Human

Hippocampal CA1 Involvement during Allocentric Encoding of Spatial Information,” J.

Neurosci., vol. 29, no. 34, pp. 10512–10519, Aug. 2009.

[36] P. K. Pilly and S. Grossberg, “How Do Spatial Learning and Memory Occur in the Brain ?

Coordinated Learning of Entorhinal Grid Cells and Hippocampal Place Cells,” J. Cogn.

Neurosci., pp. 1031–1054, 2012.

[37] N. Burgess, E. A. Maguire, and J. O’Keefe, “The Human Hippocampus and Spatial and

Episodic Memory,” Neuron, vol. 35, no. 4, pp. 625–641, Aug. 2002.

[38] J. Chen, R. K. Olsen, A. R. Preston, G. H. Glover, and A. D. Wagner, “Associative retrieval

processes in the human medial temporal lobe: hippocampal retrieval success and CA1

mismatch detection.,” Learn. Mem., vol. 18, no. 8, pp. 523–528, 2011.

[39] M. E. Hasselmo and B. P. Wyble, “Free recall and recognition in a network model of the

hippocampus: Simulating effects of scopolamine on human memory function,” Behav. Brain

Res., vol. 89, no. 1–2, pp. 1–34, Copyright (1997), with permission from Elsevier..

[40] L. R. Squire and P. Alvarez, “Retrograde amnesia and memory consolidation: a

neurobiological perspective,” Curr. Opin. Neurobiol., vol. 5, no. 2, pp. 169–177, Apr. 1995.

[41] B. McNaughton and J. McClelland, “Why There Are Complementary Learning Systems in

the Hippocampus and Neocortex: Insights From the Successes and Failure of Connectionist

Models of Learning and Memory,” Psychol. Rev., vol. 102, no. 3, pp. 419–457, 1995.

[42] P. Mégevand et al., “The hippocampus and amygdala are integrators of neocortical influence:

a cortico-cortical evoked potential study,” Brain Connect., vol. ahead of p, 2017.

[43] D. Kumaran and E. A. Maguire, “Match Mismatch Processes Underlie Human Hippocampal

Responses to Associative Novelty,” J. Neurosci., vol. 27, no. 32, pp. 8517–8524, 2007.

87

[44] M. E. Hasselmo, “The Role of Hippocampal Regions CA3 and CA1 in Matching Entorhinal

Input With Retrieval of Associations Between Objects and Context: Theoretical Comment

on Lee et al. (2005).,” Behav. Neurosci., vol. 119, no. 1, pp. 342–345, 2005.

[45] J. R. Manns, E. A. Zilli, K. C. Ong, M. E. Hasselmo, and H. Eichenbaum, “Hippocampal

CA1 spiking during encoding and retrieval: Relation to theta phase,” Neurobiol. Learn.

Mem., vol. 87, no. 1, pp. 9–20, 2007.

[46] M. E. Hasselmo, C. Bodelón, and B. P. Wyble, “A proposed function for hippocampal theta

rhythm: separate phases of encoding and retrieval enhance reversal of prior learning.,”

Neural Comput., vol. 14, no. 4, pp. 793–817, May 2002.

[47] R. Knight, “Contribution of human hippocampal region to novelty detection.,” Nature, vol.

383, no. 6597, pp. 256–259, 1996.

[48] L. Nyberg, “Any novelty in hippocampal formation and memory?,” Curr. Opin. Neurol., vol.

18, no. 4, pp. 424–428, 2005.

[49] E. Düzel, W. D. Penny, and N. Burgess, “Brain oscillations and memory,” Curr. Opin.

Neurobiol., vol. 20, no. 2, pp. 245–257, 2010.

[50] M. J. Jutras and E. A. Buffalo, “Recognition memory signals in the macaque hippocampus.,”

Proc. Natl. Acad. Sci. U. S. A., vol. 107, no. 1, pp. 401–6, Jan. 2010.

[51] K. A. Paller and G. McCarthy, “Field potentials in the human hippocampus during the

encoding and recognition of visual stimuli,” Hippocampus, vol. 12, no. 3, pp. 415–420, 2002.

[52] J. K. Kleen et al., “Oscillation Phase Locking and Late ERP Components of Intracranial

Hippocampal Recordings Correlate to Patient Performance in a Working Memory Task,”

Front. Hum. Neurosci., vol. 10, no. June, pp. 1–14, 2016.

[53] J. M. Henderson and M. S. Castelhano, “Eye Movements and Visual Memory for Scenes,”

Cogn. Process. Eye Guid., pp. 213–235, 2007.

[54] C. N. Smith and L. R. Squire, “Experience-dependent eye movements reflect hippocampus-

dependent (aware) memory.,” J. Neurosci., vol. 28, no. 48, pp. 12825–33, Nov. 2008.

[55] M. Ibbotson and B. Krekelberg, “Visual perception and saccadic eye movements,” Curr.

Opin. Neurobiol., vol. 21, no. 4, pp. 553–558, 2011.

[56] M. Le-Hoa Võ and J. M. Wolfe, “The role of memory for visual search in scenes,” Ann. N.

Y. Acad. Sci., vol. 1339, no. 1, pp. 72–81, 2015.

[57] K. A. Norman and R. C. O’Reilly, “Modeling hippocampal and neocortical contributions to

recognition memory: A complementary-learning-systems approach.,” Psychol. Rev., vol.

110, no. 4, pp. 611–646, Oct. 2003.

[58] A. Staniloiu and H. J. Markowitsch, “Towards solving the riddle of forgetting in functional

88

amnesia: Recent advances and current opinions,” Front. Psychol., vol. 3, no. NOV, pp. 1–

23, 2012.

[59] M. Kang et al., “Biochimica et Biophysica Acta Proteogenomics of the human hippocampus :

The road ahead ☆,” BBA - Proteins Proteomics, vol. 1854, no. 7, pp. 788–797, 2015.

Copyright (2015), with permission from Elsevier

[60] E. L. Johnson and R. T. Knight, “Intracranial recordings and human memory,” Curr. Opin.

Neurobiol., vol. 31, pp. 18–25, 2015.

[61] M. Tudor, L. Tudor, and K. I. Tudor, “[Hans Berger (1873-1941)--the history of

electroencephalography].,” Acta Med. Croatica, vol. 59, no. 4, pp. 307–13, 2005.

[62] G. Buzsáki, C. a. Anastassiou, and C. Koch, “The origin of extracellular fields and currents

— EEG, ECoG, LFP and spikes,” Nat. Rev. Neurosci., vol. 13, no. 6, pp. 407–420, 2012.

[63] N. Axmacher, F. Mormann, G. Fernández, C. E. Elger, and J. Fell, “Memory formation by

neuronal synchronization,” Brain Res. Rev., vol. 52, no. 1, pp. 170–182, Aug. 2006.

[64] G. Buzsáki, Rhythms of the Brain. Oxford University Press, 2006.

[65] W. Klimesch, R. Freunberger, P. Sauseng, and W. Gruber, “A short review of slow phase

synchronization and memory: Evidence for control processes in different memory systems?,”

Brain Res., vol. 1235, pp. 31–44, Copyright (2008), with permission from Elsevier..

[66] J. O’Keefe and J. Dostrovsky, “The hippocampus as a spatial map. Preliminary evidence

from unit activity in the freely-moving rat,” Brain Res., vol. 34, no. 1, pp. 171–175, 1971.

[67] G. Dragoi, “Internal operations in the hippocampus: single cell and ensemble temporal

coding,” Front. Syst. Neurosci., vol. 7, no. August, pp. 1–4, 2013.

[68] X. Xu, L. An, X. Mi, and T. Zhang, “Impairment of cognitive function and synaptic plasticity

associated with alteration of information flow in theta and gamma oscillations in melamine-

treated rats.,” PLoS One, vol. 8, no. 10, p. e77796, 2013.

[69] D. Robbe and G. Buzsáki, “Alteration of theta timescale dynamics of hippocampal place cells

by a cannabinoid is associated with memory impairment David,” J Neurosci, vol. 29, no. 40,

pp. 320–331, 2009.

[70] B. Givens, “Low Doses of Ethanol Impair Spatial Working Memory and Reduce

Hippocampal Theta Activity,” Alcohol. Clin. Exp. Res., vol. 19, no. 3, pp. 763–767, 1995.

[71] S. Leutgeb and S. J. Mizumori, “Excitotoxic septal lesions result in spatial memory deficits

and altered flexibility of hippocampal single-unit representations.,” J. Neurosci., vol. 19, no.

15, pp. 6661–72, 1999.

[72] B. C. Lega, J. Jacobs, and M. Kahana, “Human hippocampal theta oscillations and the

formation of episodic memories.,” Hippocampus, vol. 22, no. 4, pp. 748–61, Apr. 2012.

89

[73] A. D. Ekstrom, J. B. Caplan, E. Ho, K. Shattuck, I. Fried, and M. J. Kahana, “Human

hippocampal theta activity during virtual navigation,” Hippocampus, vol. 15, no. 7, pp. 881–

889, 2005.

[74] J. Jacobs, “Hippocampal theta oscillations are slower in humans than in rodents: implications

for models of spatial navigation and memory.,” Philos. Trans. R. Soc. Lond. B. Biol. Sci.,

vol. 369, no. 1635, p. 20130304, Feb. 2014.

[75] S. Hanslmayr and T. Staudigl, “How brain oscillations form memories--a processing based

perspective on oscillatory subsequent memory effects.,” Neuroimage, vol. 85 Pt 2, pp. 648–

55, Jan. 2014.

[76] O. S. S. Vinogradova, “Expression, control, and probable functional significance of the

neuronal theta-rhythm,” Prog. Neurobiol., vol. 45, no. 6, pp. 523–583, Apr. 1995.

[77] R. Goutagny, J. Jackson, and S. Williams, “Self-generated theta oscillations in the

hippocampus.,” Nat. Neurosci., vol. 12, no. 12, pp. 1491–1493, 2009.

[78] L. L. Colgin, “Mechanisms and functions of theta rhythms.,” Annu. Rev. Neurosci., vol. 36,

pp. 295–312, Jul. 2013.

[79] R. W. Berg, “Rhythmic Whisking by Rat: Retraction as Well as Protraction of the Vibrissae

Is Under Active Muscular Control,” J. Neurophysiol., vol. 89, no. 1, pp. 104–117, 2002.

[80] F. Macrides, H. B. Eichenbaum, and W. B. Forbes, “Temporal relationship between sniffing

and the limbic theta rhythm during odor discrimination reversal learning.,” J. Neurosci., vol.

2, no. 12, pp. 1705–1717, 1982.

[81] X. G. Troncoso et al., “Saccades and microsaccades during visual fixation, exploration, and

search: Foundations for a common saccadic generator,” J. Vis., vol. 9, no. 8, pp. 447–447,

Jan. 2010.

[82] C. D. Tesche and J. Karhu, “Theta oscillations index human hippocampal activation during

a working memory task,” Proc. Natl. Acad. Sci., vol. 97, no. 2, pp. 919–924, Jan. 2000.

[83] D. O. Hebb, “The organization of behavior: A Neuropsychological Approach,” The American

Journal of Psychology, vol. 63, no. 4. p. 633, 1949.

[84] T. Kitamura, S. K. Ogawa, D. S. Roy, and T. Okuyama, “Engrams and circuits crucial for

systems consolidation of a memory,” vol. 78, no. April, pp. 73–78, 2017.

[85] J. Fell et al., “Human memory formation is accompanied by rhinal-hippocampal coupling

and decoupling.,” Nat. Neurosci., vol. 4, no. 12, pp. 1259–1264, Dec. 2001.

[86] P. Fries, “Rhythms for Cognition: Communication through Coherence,” Neuron, vol. 88, no.

1, pp. 220–235, 2015.

[87] C. C. Canavier, “Phase-resetting as a tool of information transmission,” Curr. Opin.

90

Neurobiol., vol. 31, pp. 206–213, Apr. 2015.

[88] M. E. Hasselmo, “Grid cell mechanisms and function: Contributions of entorhinal persistent

spiking and phase resetting,” Hippocampus, vol. 18, no. 12, pp. 1213–1229, 2008.

[89] H. Eichenbaum, “Time cells in the hippocampus: a new dimension for mapping memories.,”

Nat. Rev. Neurosci., vol. 15, no. 11, pp. 732–44, Nov. 2014.

[90] J. O’Keefe and M. L. Recce, “Phase relationship between hippocampal place units and the

EEG theta rhythm.,” Hippocampus, vol. 3, no. 3, pp. 317–330, Jul. 1993.

[91] P. Sauseng, W. Klimesch, W. R. Gruber, S. Hanslmayr, R. Freunberger, and M. Doppelmayr,

“Are event-related potential components generated by phase resetting of brain oscillations?

A critical discussion.,” Neuroscience, vol. 146, no. 4, pp. 1435–44, Jun. 2007.

[92] C. Hölscher, R. Anwyl, and M. J. Rowan, “Stimulation on the positive phase of hippocampal

theta rhythm induces long-term potentiation that can Be depotentiated by stimulation on the

negative phase in area CA1 in vivo.,” J. Neurosci., vol. 17, no. 16, pp. 6470–6477, 1997.

[93] W. Buño, J. L. Garcia-Sanchez, and E. Garcia-Austt, “Reset of hippocampal rhythmical

activities by afferent stimulation.,” Brain Res. Bull., vol. 3, no. 1, pp. 21–8, 1978.

[94] B. Givens, “Stimulus-evoked resetting of the dentate theta rhythm: relation to working

memory.,” Neuroreport, vol. 8, no. 1, pp. 159–163, 1996.

[95] J. M. Williams and B. Givens, “Stimulation-induced reset of hippocampal theta in the freely

performing rat.,” Hippocampus, vol. 13, no. 1, pp. 109–16, Jan. 2003.

[96] H. McCartney, A. D. Johnson, Z. M. Weil, and B. Givens, “Theta reset produces optimal

conditions for long-term potentiation.,” Hippocampus, vol. 14, no. 6, pp. 684–7, 2004.

[97] J. M. Hyman, B. P. Wyble, V. Goyal, C. A. Rossi, and M. E. Hasselmo, “Stimulation in

hippocampal region CA1 in behaving rats yields long-term potentiation when delivered to

the peak of theta and long-term depression when delivered to the trough.,” J. Neurosci., vol.

23, no. 37, pp. 11725–31, Dec. 2003.

[98] P. Fries, J. H. Reynolds, A. E. Rorie, and R. Desimone, “Modulation of Oscillatory Neuronal

Synchronization by Selective Visual Attention,” Am. Assoc. Adv. Sci., vol. 291, no. 5508, pp.

1560–1563, 2001.

[99] A. L. Griffin, Y. Asaka, R. D. Darling, and S. D. Berry, “Theta-Contingent Trial Presentation

Accelerates Learning Rate and Enhances Hippocampal Plasticity During Trace Eyeblink

Conditioning.,” Behav. Neurosci., vol. 118, no. 2, pp. 403–411, 2004.

[100] R. D. Darling et al., “Eyeblink conditioning contingent on hippocampal theta enhances

hippocampal and medial prefrontal responses,” vol. 1, pp. 2213–2224, 2011.

[101] M. S. Nokia, T. Waselius, J. E. Mikkonen, J. Wikgren, and M. Penttonen, “Phase matters:

91

responding to and learning about peripheral stimuli depends on hippocampal θ phase at

stimulus onset,” Learn. Mem., vol. 22, no. 6, pp. 307–317, 2015.

[102] T. Waselius, E. Pöllänen, J. Wikgren, M. Penttonen, and M. S. Nokia, “Hippocampal theta

phase-contingent memory retrieval in delay and trace eyeblink conditioning,” Behav. Brain

Res., vol. 337, no. April 2017, pp. 264–270, 2017.

[103] M. E. Hasselmo, “What is the function of hippocampal theta rhythm?—Linking behavioral

data to phasic properties of field potential and unit recording data,” Hippocampus, vol. 15,

no. 7, pp. 936–949, 2005.

[104] G. Buzsaki, P. Rappelsberger, and L. Kellenyi, “Depth Profiles of Hippocampal Rhythmic

Slow Activity (’Theta Rhythm’) Depend on Behaviour,” Electroencephalograpt Clin.

Neurophysiol., vol. 61, pp. 77–88, 1985.

[105] J. H. Siegle and M. A. Wilson, “Enhancement of encoding and retrieval functions through

theta phase-specific manipulation of hippocampus,” Elife, vol. 2014, no. 3, pp. 1–18, 2014.

[106] F. Mormann et al., “Phase/amplitude reset and theta-gamma interaction in the human medial

temporal lobe during a continuous word recognition memory task,” Hippocampus, vol. 15,

no. 7, pp. 890–900, Jan. 2005.

[107] R. T. Canolty and R. T. Knight, “The functional role of cross-frequency coupling,” Trends

Cogn. Sci., vol. 14, no. 11, pp. 506–515, 2010.

[108] J. E. Lisman and O. Jensen, “The θ-γ neural code.,” Neuron, vol. 77, no. 6, pp. 1002–16,

Mar. 2013.

[109] D. Ji and M. a Wilson, “Coordinated memory replay in the visual cortex and hippocampus

during sleep.,” Nat. Neurosci., vol. 10, no. 1, pp. 100–107, 2007.

[110] J. Born and S. Diekelmann, “The memory function of sleep,” Nat. Rev. Neurosci., vol. 11,

no. 2, pp. 114–127, Feb. 2010.

[111] N. Maingret, G. Girardeau, R. Todorova, M. Goutierre, and M. Zugaro, “Hippocampo-

cortical coupling mediates memory consolidation during sleep,” Nat Neurosci, vol. 19, no.

7, pp. 959–964, 2016.

[112] J. A. Greenberg, J. F. Burke, R. Haque, M. J. Kahana, and K. A. Zaghloul, “Decreases in

theta and increases in high frequency activity underlie associative memory encoding.,”

Neuroimage, vol. 114, pp. 257–63, Jul. 2015.

[113] J. Fell, E. Ludowig, T. Rosburg, N. Axmacher, and C. E. Elger, “Phase-locking within human

mediotemporal lobe predicts memory formation,” Neuroimage, vol. 43, no. 2, pp. 410–419,

2008.

[114] G. Fernandez, A. Effern, T. Grunwald, and N. Pezer, “Real-Time Tracking of Memory

92

Formation in the Human Rhinal Cortex and Hippocampus,” Sci. (New York, NY), vol. 1582,

no. 1999, 1999.

[115] S. Guderian, B. H. Schott, A. Richardson-Klavehn, E. Duzel, and E. Düzel, “Medial temporal

theta state before an event predicts episodic encoding success in humans,” Proc. Natl. Acad.

Sci., vol. 106, no. 13, pp. 5365–5370, Mar. 2009.

[116] S. Raghavachari, J. E. Lisman, M. Tully, J. R. Madsen, E. B. Bromfield, and M. J. Kahana,

“Theta oscillations in human cortex during a working-memory task: evidence for local

generators.,” J. Neurophysiol., vol. 95, no. 3, pp. 1630–1638, 2006.

[117] M. Mölle and J. Born, “Slow oscillations orchestrating fast oscillations and memory

consolidation.,” Prog. Brain Res., vol. 193, pp. 93–110, Jan. 2011.

[118] N. M. Long et al., “Contextually Mediated Spontaneous Retrieval Is Specific to the

Hippocampus,” Curr. Biol., vol. 27, no. 7, pp. 1074–1079, 2017.

[119] J. D. Green and A. A. Arduini, “Hippocampal electrical activity in arousal.,” J.

Neurophysiol., vol. 17, no. 6, pp. 533–57, Nov. 1954.

[120] V. D. Bohbot, M. S. Copara, J. Gotman, and A. D. Ekstrom, “Low-frequency theta

oscillations in the human hippocampus during real-world and virtual navigation,” Nat.

Commun., vol. 8, pp. 1–7, 2017.

[121] M. J. Jutras, P. Fries, and E. A. Buffalo, “Oscillatory activity in the monkey hippocampus

during visual exploration and memory formation.,” Proc. Natl. Acad. Sci. U. S. A., vol. 110,

no. 32, pp. 13144–9, Aug. 2013.

[122] K. L. Hoffman, M. C. Dragan, T. K. Leonard, C. Micheli, R. Montefusco-Siegmund, and T.

A. Valiante, “Saccades during visual exploration align hippocampal 3-8 Hz rhythms in

human and non-human primates.,” Front. Syst. Neurosci., vol. 7, p. 43, Jan. 2013.

[123] S. Makeig et al., “Dynamic Brain Sources of Visual Evoked Responses,” Science (80-. ).,

vol. 295, no. 5555, 2002.

[124] W. D. Penny, S. J. Kiebel, J. M. Kilner, and M. D. Rugg, “Event-related brain dynamics,”

Trends Neurosci., vol. 25, no. 8, pp. 387–389, 2002.

[125] A. S. Shah et al., “Neural Dynamics and the Fundamental Mechanisms of Event-related Brain

Potentials,” Cereb. Cortex, vol. 14, no. 5, pp. 476–483, Mar. 2004.

[126] B. P. Staresina, J. Fell, J. C. Dunn, N. Axmacher, and R. N. Henson, “Using state-trace

analysis to dissociate the functions of the human hippocampus and perirhinal cortex in

recognition memory.,” Proc. Natl. Acad. Sci. U. S. A., vol. 110, no. 8, pp. 3119–3124, 2013.

[127] W. Klimesch, S. Hanslmayr, P. Sauseng, and W. R. Gruber, “Distinguishing the evoked

response from phase reset: A comment to Mäkinen et al.,” Neuroimage, vol. 29, no. 3, pp.

93

808–811, 2006.

[128] G. F. Woodman, “A brief introduction to the use of event-related potentials (ERPs) in studies

of perception and attention,” Atten. Percept. Psychophysiol., vol. 72, no. 8, pp. 1–29, 2010.

[129] J. M. Olichney, J. C. Yang, J. Taylor, and M. Kutas, “Cognitive event-related potentials:

Biomarkers of synaptic dysfunction across the stages of Alzheimer’s disease,” Adv.

Alzheimer’s Dis., vol. 2, no. 0 3, pp. 421–434, 2011.

[130] M. D. Rugg and T. Curran, “Event-related potentials and recognition memory,” Trends Cogn.

Sci., vol. 11, no. 6, pp. 251–257, 2007.

[131] F. Mormann, G. Fernández, P. Klaver, B. Weber, C. E. Elger, and J. Fell, “Declarative

memory formation in hippocampal sclerosis: an intracranial event-related potentials study.,”

Neuroreport, vol. 18, no. 4, pp. 317–21, 2007.

[132] A. J. Watrous, J. Fell, A. D. Ekstrom, and N. Axmacher, “More than spikes: common

oscillatory mechanisms for content specific neural representations during perception and

memory.,” Curr. Opin. Neurobiol., vol. 31, pp. 33–9, Apr. 2015.

[133] N. Axmacher et al., “Intracranial EEG correlates of expectancy and memory formation in the

human hippocampus and nucleus accumbens.,” Neuron, vol. 65, no. 4, pp. 541–9, Feb. 2010.

[134] A. Grippo, L. Pelosi, V. Mehta, and L. D. Blumhardt, “Working memory in temporal lobe

epilepsy: an event-related potential study.,” Electroencephalogr. Clin. Neurophysiol., vol.

99, no. 3, pp. 200–13, Sep. 1996.

[135] K. A. Paller, G. McCarthy, and C. C. Wood, “ERPs predictive of subsequent recall and

recognition performance,” Biol. Psychol., vol. 26, no. 1–3, pp. 269–276, Jun. 1988.

[136] J. L. Voss and K. A. Paller, Neural Substrates of Remembering: Event-Related Potential

Studies☆, Third Edit., no. September 2016. Elsevier, 2008.

[137] J. A. Meltzer et al., “Effects of working memory load on oscillatory power in human

intracranial EEG.,” Cereb. Cortex, vol. 18, no. 8, pp. 1843–55, Aug. 2008.

[138] S. Sobotka and J. L. Ringo, “Saccadic eye movements, even in darkness, generate event-

related potentials recorded in medial sputum and medial temporal cortex.,” Brain Res., vol.

756, no. 1–2, pp. 168–173, 1997.

[139] S. Sobotka, W. Zuo, and J. L. Ringo, “Is the Functional Connectivity Within Temporal Lobe

Influenced By Saccadic Eye Movements?,” J. Neurophysiol., vol. 88, no. 4, 2002.

[140] T. Staudigl, E. Hartl, S. Noachtar, C. F. Doeller, and O. Jensen, “Saccades phase-locked to

alpha oscillations in the occipital and medial temporal lobe enhance memory encoding,”

bioRxiv, 2017.

[141] A. L. Benabid et al., “Long-term suppression of tremor by chronic stimulation of the ventral

94

intermediate thalamic nucleus.,” Lancet (London, England), vol. 337, no. 8738, pp. 403–6,

Feb. 1991.

[142] T. M. Herrington, J. J. Cheng, and E. N. Eskandar, “Mechanisms of deep brain stimulation,”

J. Neurophysiol., vol. 115, no. 1, pp. 19–38, 2016.

[143] I. Beaulieu-Boire and A. Fasano, “Current or voltage? Another Shakespearean dilemma,”

Eur. J. Neurol., vol. 22, no. 6, pp. 887–888, 2015.

[144] C. Hamani et al., “Memory enhancement induced by hypothalamic/fornix deep brain

stimulation,” Ann. Neurol., vol. 63, no. 1, pp. 119–123, 2008.

[145] T. Sankar, N. Lipsman, and A. M. Lozano, “Deep Brain Stimulation for Disorders of Memory

and Cognition,” Neurotherapeutics, vol. 11, no. 3, pp. 527–534, Apr. 2014.

[146] S. G. Coleshill et al., “Material-Specific Recognition Memory Deficits Elicited by Unilateral

Hippocampal Electrical Stimulation,” J. Neurosci., vol. 24, no. 7, 2004.

[147] M. E. Lacruz, A. Valentín, J. J. G. Seoane, R. G. Morris, R. P. Selway, and G. Alarcón,

“Single pulse electrical stimulation of the hippocampus is sufficient to impair human episodic

memory.,” Neuroscience, vol. 170, no. 2, pp. 623–32, Oct. 2010.

[148] J. Jacobs et al., “Direct Electrical Stimulation of the Human Entorhinal Region and

Hippocampus Impairs Memory,” Neuron, vol. 92, no. 5, pp. 983–990, Dec. 2016.

[149] N. Suthana et al., “Memory Enhancement and Deep-Brain Stimulation of the Entorhinal

Area,” N. Engl. J. Med., vol. 366, no. 6, pp. 502–510, Feb. 2012.

[150] N. Suthana and I. Fried, “Deep brain stimulation for enhancement of learning and memory.,”

Neuroimage, vol. 85 Pt 3, pp. 996–1002, Jan. 2014.

[151] K. Kim, A. D. Ekstrom, and N. Tandon, “A network approach for modulating memory

processes via direct and indirect brain stimulation: Toward a causal approach for the neural

basis of memory,” Neurobiol. Learn. Mem., vol. 134, no. Part A, pp. 162–177, 2016.

[152] J. P. Miller, J. A. Sweet, C. M. Bailey, C. N. Munyon, H. O. Luders, and P. S. Fastenau,

“Visual-spatial memory may be enhanced with theta burst deep brain stimulation of the

fornix: A preliminary investigation with four cases,” Brain, vol. 138, no. 7, pp. 1833–1842,

Jul. 2015.

[153] A. S. Titiz et al., “Theta-burst microstimulation in the human entorhinal area improves

memory specificity,” Elife, vol. 6, Oct. 2017.

[154] J. Fell et al., “Memory modulation by weak synchronous deep brain stimulation: a pilot

study.,” Brain Stimul., vol. 6, no. 3, pp. 270–3, May 2013.

[155] Y. Ezzyat et al., “Direct Brain Stimulation Modulates Encoding States and Memory

Performance in Humans,” Curr. Biol., vol. 27, no. 9, pp. 1251–1258, 2017.

95

[156] S. K. B. Bick and E. N. Eskandar, “Neuromodulation for restoring memory,” Neurosurg.

Focus, vol. 40, no. 5, p. E5, May 2016.

[157] T. W. Berger, R. E. Hampson, D. Song, A. Goonawardena, V. Z. Marmarelis, and S. A.

Deadwyler, “A cortical neural prosthesis for restoring and enhancing memory.,” J. Neural

Eng., vol. 8, no. 4, p. 46017, Aug. 2011.

[158] R. E. Hampson et al., “A Nonlinear Model for Hippocampal Cognitive Prosthesis: Memory

Facilitation by Hippocampal Ensemble Stimulation,” IEEE Trans. Neural Syst. Rehabil.

Eng., vol. 20, no. 2, pp. 184–197, Mar. 2012.

[159] A. Ikeda, S. Miyamoto, and H. Shibasaki, “Cortical motor mapping in epilepsy patients:

information from subdural electrodes in presurgical evaluation.,” Epilepsia, vol. 43 Suppl 9,

pp. 56–60, Jan. 2002.

[160] C. J. Keller, C. J. Honey, P. Mégevand, L. Entz, I. Ulbert, and A. D. Mehta, “Mapping human

brain networks with cortico-cortical evoked potentials.,” Philos. Trans. R. Soc. Lond. B. Biol.

Sci., vol. 369, no. 1653, Oct. 2014.

[161] R. Enatsu et al., “Connections of the limbic network: a corticocortical evoked potentials

study.,” Cortex., vol. 62, pp. 20–33, Jan. 2015.

[162] C. J. Keller et al., “Corticocortical evoked potentials reveal projectors and integrators in

human brain networks.,” J. Neurosci., vol. 34, no. 27, pp. 9152–63, 2014.

[163] Y. Kubota et al., “In vivo human hippocampal cingulate connectivity: a corticocortical

evoked potentials (CCEPs) study.,” Clin. Neurophysiol., vol. 124, no. 8, pp. 1547–56, Aug.

2013.

[164] O. Talakoub, A. Gomez Palacio Schjetnan, T. A. Valiante, M. R. Popovic, and K. L.

Hoffman, “Closed-Loop Interruption of Hippocampal Ripples through Fornix Stimulation in

the Non-Human Primate,” Brain Stimul., vol. 9, no. 6, pp. 911–918, Nov. 2016.

[165] J. Fiser, C. Chiu, and M. Weliky, “Small modulation of ongoing cortical dynamics by sensory

input during natural vision,” Nature, vol. 431, no. 7008, pp. 573–578, 2004.

[166] J. D. Victor, “Temporal aspects of neural coding in the retina and lateral geniculate.,”

Network, vol. 10, no. 4, pp. R1–R66, 1999.

[167] C. Kayser, K. P. Körding, and P. König, “Processing of complex stimuli and natural scenes

in the visual cortex,” Curr. Opin. Neurobiol., vol. 14, no. 4, pp. 468–473, 2004.

[168] Z. Wang and A. Bovik, “A universal image quality index,” IEEE Signal Process. Lett., vol.

9, no. 3, pp. 81–84, 2002.

[169] C. J. Lin et al., “Developing and Evaluating a Target-Background Similarity Metric for

Camouflage Detection,” PLoS One, vol. 9, no. 2, p. e87310, Feb. 2014.

96

[170] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From

error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–

612, 2004.

[171] D. D. Salvucci and J. H. Goldberg, “Identifying fixations and saccades in eye-tracking

protocols,” Proc. Symp. Eye Track. Res. Appl. - ETRA ’00, no. October, pp. 71–78, 2000.

[172] A. T. Bahill, M. R. Clark, and L. Stark, “The Main Sequence, A Tool for Studying Human

Eye Movements.,” Math. Biosci., vol. 24, pp. 191–204, 1975.

[173] G. Bargary, J. M. Bosten, P. T. Goodbourn, A. J. Lawrance-Owen, R. E. Hogg, and J. D.

Mollon, “Individual differences in human eye movements: An oculomotor signature?,”

Vision Res., 2017.

[174] T. A. Valiante, “Selective Amygdalohippocampectomy,” in Textbook of Stereotactic and

Functional Neurosurgery, Volume 1, Springer Science & Business Media, 2009.

[175] D. M. Groppe et al., “iELVis: An open source MATLAB toolbox for localizing and

visualizing human intracranial electrode data,” J. Neurosci. Methods, vol. 281, pp. 40–48,

Apr. 2017.

[176] M. X. Cohen, “Analyzing Neural Time Series Data: Theory and Practice,” MIT Press. p. 600,

2014.

[177] C. Tallon-Baudry, O. Bertrand, C. Delpuech, and J. Pernier, “Stimulus specificity of phase-

locked and non-phase-locked 40 Hz visual responses in human.,” J. Neurosci., vol. 16, no.

13, pp. 4240–9, 1996.

[178] Z. Fitzgerald, Z. Thayer, A. Mohamed, and L. A. Miller, “Examining factors related to

accelerated long-term forgetting in epilepsy using ambulatory EEG monitoring,” Epilepsia,

vol. 54, no. 5, pp. 819–827, May 2013.

[179] C. Helmstaedter and C. . Elger, “Behavioral markers for self- and other-attribution of

memory: a study in patients with temporal lobe epilepsy and healthy volunteers,” Epilepsy

Res., vol. 41, no. 3, pp. 235–243, 2000.

[180] S. Lah, T. Lee, S. Grayson, and L. Miller, “Effects of Temporal Lobe Epilepsy on Retrograde

Memory,” Epilepsia, vol. 47, no. 3, pp. 615–625, Mar. 2006.

[181] N. Muhlert et al., “Accelerated long-term forgetting in temporal lobe but not idiopathic

generalised epilepsy,” Neuropsychologia, vol. 49, no. 9, pp. 2417–2426, 2011.

[182] C. Hoppe, C. E. Elger, and C. Helmstaedter, “Long-term memory impairment in patients

with focal epilepsy,” Epilepsia, vol. 48, no. SUPPL. 9, pp. 26–29, 2007.

[183] D. S. Rizzuto et al., “Reset of human neocortical oscillations during a working memory

task.,” Proc. Natl. Acad. Sci. U. S. A., vol. 100, no. 13, pp. 7931–6, Jun. 2003.

97

[184] C. Katz, K. Patel, O. Talakoub, K. Hoffman, and T. A. Valiante, “Characterizing neural

potentials related to eye movements and image presentation in the human mesial temporal

lobe Kramay Patel Neuron to Brain Laboratory,” 2018.

[185] C. K. Kovach, N. Tsuchiya, H. Kawasaki, H. Oya, M. A. Howard, and R. Adolphs,

“Manifestation of ocular-muscle EMG contamination in human intracranial recordings,”

Neuroimage, vol. 54, no. 1, pp. 213–233, 2011.

[186] S. Yuval-Greenberg, O. Tomer, A. S. Keren, I. Nelken, and L. Y. Deouell, “Transient

Induced Gamma-Band Response in EEG as a Manifestation of Miniature Saccades,” Neuron,

vol. 58, no. 3, pp. 429–441, 2008.

[187] A. Hyvärinen and E. Oja, “Independent component analysis: Algorithms and applications,”

Neural Networks, vol. 13, no. 4–5. pp. 411–430, 2000.

[188] B. Voloh and T. Womelsdorf, “A Role of Phase-Resetting in Coordinating Large Scale

Neural Networks During Attention and Goal-Directed Behavior.,” Front. Syst. Neurosci.,

vol. 10, p. 18, 2016.

[189] B. Voloh, T. A. Valiante, S. Everling, and T. Womelsdorf, “Theta–gamma coordination

between anterior cingulate and prefrontal cortex indexes correct attention shifts,” Proc. Natl.

Acad. Sci., vol. 112, no. 27, pp. 8457–8462, 2015.

[190] D. K. Murphey, J. H. R. Maunsell, M. S. Beauchamp, and D. Yoshor, “Perceiving electrical

stimulation of identified human visual areas.,” Proc. Natl. Acad. Sci. U. S. A., vol. 106, no.

13, pp. 5389–5393, 2009.

[191] V. L. Chau, E. F. Murphy, R. S. Rosenbaum, J. D. Ryan, and K. L. Hoffman, “A Flicker

Change Detection Task Reveals Object-in-Scene Memory Across Species.,” Front. Behav.

Neurosci., vol. 5, p. 58, Jan. 2011.

[192] J. Jacobs et al., “Direct Electrical Stimulation of the Human Entorhinal Region and

Hippocampus Impairs Memory,” Neuron, vol. 92, no. 5, pp. 983–990, 2016.

[193] C. N. Smith and L. R. Squire, “When eye movements express memory for old and new scenes

in the absence of awareness and independent of hippocampus,” Learn. Mem., vol. 24, no. 2,

pp. 95–103, 2017.

[194] T. Staudigl and S. Hanslmayr, “Theta oscillations at encoding mediate the context-dependent

nature of human episodic memory.,” Curr. Biol., vol. 23, no. 12, pp. 1101–6, Jun. 2013.

[195] N. M. Long and M. J. Kahana, “Modulation of Task Demands Suggests That Semantic

Processing Interferes With the Formation of Episodic Associations,” J. Exp. Psychol. Learn.

Mem. Cogn., vol. 43, no. 2, pp. 167–176, 2016.

[196] M. B. Merkow, J. F. Burke, A. G. Ramayya, A. D. Sharan, M. R. Sperling, and M. J. Kahana,

98

“Stimulation of the human medial temporal lobe between learning and recall selectively

enhances forgetting,” Brain Stimul., pp. 1–6, 2016.

[197] B. Tversky and T. Sherman, “Picture memory improves with longer on time and off time.,”

J. Exp. Psychol. Hum. Learn. Mem., vol. 1, no. 2, pp. 114–118, 1975.

[198] Z. Fitzgerald, A. Mohamed, M. Ricci, Z. Thayer, and L. Miller, “Accelerated long-term

forgetting: A newly identified memory impairment in epilepsy,” J. Clin. Neurosci., vol. 20,

no. 11, pp. 1486–1491, 2013.

[199] G. La Corte, Y. Wei, N. Chernyy, B. J. Gluckman, and S. J. Schiff, “Frequency dependence

of behavioral modulation by hippocampal electrical stimulation.,” J. Neurophysiol., vol. 111,

no. 3, pp. 470–80, 2014.

[200] C. S. Inman et al., “Tuning direct electrical amygdala stimulation parameters for declarative

memory enhancement in humans,” in Society For Neuroscience, 2017.

[201] F. P. Battaglia, K. Benchenane, A. Sirota, C. M. a Pennartz, and S. I. Wiener, “The

hippocampus: Hub of brain network communication for memory,” Trends Cogn. Sci., vol.

15, no. 7, pp. 310–318, 2011.

[202] B. A. Lopour, A. Tavassoli, I. Fried, and D. L. Ringach, “Coding of information in the phase

of local field potentials within human medial temporal lobe,” Neuron, vol. 79, no. 3, pp. 594–

606, Aug. 2013.

[203] E. S. Brazhnik, O. S. Vinogradova, and A. M. Karanov, “Frequency modulation of neuronal

theta-bursts in rabbit’s septum by low-frequency repetitive stimulation of the afferent

pathways.,” Neuroscience, vol. 14, no. 2, pp. 501–8, Feb. 1985.

[204] N. J. Killian, S. M. Potter, and E. A. Buffalo, “Saccade direction encoding in the primate

entorhinal cortex during visual exploration,” Proc. Natl. Acad. Sci., pp. 1–6, 2015.

[205] J. Ito, P. Maldonado, W. Singer, and S. Grun, “Saccade-Related Modulations of Neuronal

Excitability Support Synchrony of Visually Elicited Spikes,” Cereb. Cortex, vol. 21, no. 11,

pp. 2482–2497, Nov. 2011.

[206] G. D. Horwitz and W. T. Newsome, “Target selection for saccadic eye movements: prelude

activity in the superior colliculus during a direction-discrimination task.,” J. Neurophysiol.,

vol. 86, no. 5, pp. 2543–58, 2001.

[207] J. I. Gold and M. N. Shadlen, “Representation of a perceptual decision in developing

oculomotor commands,” Nature, vol. 404, no. 6776, pp. 390–394, 2000.

[208] M. A. Segraves and K. Park, “The relationship of monkey frontal eye field activity to saccade

dynamics.,” J. Neurophysiol., vol. 69, no. 6, pp. 1880–9, 1993.

[209] D. J. Mort et al., “The anatomy of visual neglect,” Brain, vol. 126, no. 9, pp. 1986–1997,

99

Sep. 2003.

[210] S. Nemanic, M. C. Alvarado, and J. Bachevalier, “The Hippocampal/Parahippocampal

Regions and Recognition Memory: Insights from Visual Paired Comparison versus Object-

Delayed Nonmatching in Monkeys,” J. Neurosci., vol. 24, no. 8, pp. 2013–2026, 2004.

[211] C. E. Schroeder and P. Lakatos, “Low-frequency neuronal oscillations as instruments of

sensory selection,” Trends Neurosci., vol. 32, no. 1, pp. 9–18, 2009.

[212] G. Pezzulo, C. Kemere, and M. A. A. van der Meer, “Internally generated hippocampal

sequences as a vantage point to probe future-oriented cognition,” Ann. N. Y. Acad. Sci., vol.

1396, no. 1, pp. 144–165, 2017.

[213] J.-J. J. Lin et al., “Theta band power increases in the posterior hippocampus predict

successful episodic memory encoding in humans,” Hippocampus, vol. 27, no. 10, pp. 1040–

1053, Jun. 2017.

[214] J. M. Hyman, M. E. Hasselmo, and J. K. Seamans, “What is the Functional Relevance of

Prefrontal Cortex Entrainment to Hippocampal Theta Rhythms?,” Front. Neurosci., vol. 5,

p. 24, 2011.

[215] L. Pessoa and R. Adolphs, “Emotion processing and the amygdala: from a ‘low road’ to

‘many roads’ of evaluating biological significance.,” Nat. Rev. Neurosci., vol. 11, no. 11, pp.

773–783, 2010.

[216] B. H. Bland and S. D. Oddie, “Theta band oscillation and synchrony in the hippocampal

formation and associated structures: the case for its role in sensorimotor integration,” Behav.

Brain Res., vol. 127, no. 1–2, pp. 119–136, Dec. 2001.

[217] M. L. R. Meister and E. A. Buffalo, “Getting directions from the hippocampus: The neural

connection between looking and memory,” Neurobiol. Learn. Mem., Dec. 2015.

[218] Y. Lefler, B. Torben-Nielsen, and Y. Yarom, “Oscillatory activity, phase differences, and

phase resetting in the inferior olivary nucleus.,” Front. Syst. Neurosci., vol. 7, p. 22, 2013.

[219] K. Khodakhah, “Neuroscience: Decrypting a brain enigma.,” Nature, vol. 526, no. 7573, pp.

326–327, 2015.

[220] R. M. Hope, “Cognitive Control of Saccadic Behavior in the Antisaccade Task: a Model of

Voluntary and Involuntary Eye Movements,” Dissertation, vol. 2016, no. April, 2016.

[221] C. R. Oehrn et al., “Neural communication patterns underlying conflict detection, resolution,

and adaptation.,” J. Neurosci., vol. 34, no. 31, pp. 10438–52, Jul. 2014.

[222] A. R. Preston and H. Eichenbaum, “Interplay of Hippocampus and Prefrontal Cortex in

Memory,” Curr. Biol., vol. 23, no. 17, pp. R764–R773, 2013.

[223] E. Antonova, D. Parslow, M. Brammer, G. R. Dawson, S. H. D. Jackson, and R. G. Morris,

100

“Age-related neural activity during allocentric spatial memory,” Memory, vol. 17, no. 2, pp.

125–143, Feb. 2009.

[224] D. M. Bannerman et al., “Hippocampal synaptic plasticity, spatial memory and anxiety,” Nat.

Rev. Neurosci., vol. 15, pp. 181–192, 2014.

[225] B. A. Mander et al., “Impaired prefrontal sleep spindle regulation of hippocampal-dependent

learning in older adults.,” Cereb. Cortex, vol. 24, no. 12, pp. 3301–9, Dec. 2014.

[226] B. A. Mander et al., “Prefrontal atrophy, disrupted NREM slow waves and impaired

hippocampal-dependent memory in aging.,” Nat. Neurosci., vol. 16, no. 3, pp. 357–64, Mar.

2013.

[227] B. A. Mander et al., “β-amyloid disrupts human NREM slow waves and related

hippocampus-dependent memory consolidation.,” Nat. Neurosci., vol. 18, no. 7, pp. 1051–

1057, Jun. 2015.

[228] H. T. Ito, S.-J. Zhang, M. P. Witter, E. I. Moser, and M.-B. Moser, “A prefrontal–thalamo–

hippocampal circuit for goal-directed spatial navigation,” Nature, vol. 522, no. 7554, pp. 50–

55, May 2015.

[229] C. McCormick, M. Quraan, M. Cohn, T. a. Valiante, and M. P. McAndrews, “Default mode

network connectivity indicates episodic memory capacity in mesial temporal lobe epilepsy,”

Epilepsia, vol. 54, no. 5, pp. 809–818, 2013.

[230] J. Jacobs, M. J. Kahana, A. D. Ekstrom, and I. Fried, “Brain Oscillations Control Timing of

Single-Neuron Activity in Humans,” J. Neurosci., vol. 27, no. 14, pp. 3839–3844, 2007.

[231] A. M. Kuncel and W. M. Grill, “Selection of stimulus parameters for deep brain stimulation,”

Clin. Neurophysiol., vol. 115, no. 11, pp. 2431–2441, Nov. 2004.

[232] J. Larson, D. Wong, and G. Lynch, “Patterned stimulation at the theta frequency is optimal

for the induction of hippocampal long-term potentiation.,” Brain Res., vol. 368, no. 2, pp.

347–50, Mar. 1986.

[233] L. M. Grover, E. Kim, J. D. Cooke, and W. R. Holmes, “LTP in hippocampal area CA1 is

induced by burst stimulation over a broad frequency range centered around delta.,” Learn.

Mem., vol. 16, no. 1, pp. 69–81, Jan. 2009.

[234] P. T. Huerta and J. E. Lisman, “Synaptic Plasticity During the Cholinergic Theta- Frequency

Oscillation In Vitro,” Science (80-. )., vol. 61, no. 1936, pp. 58–61, 1996.

[235] P. T. Huerta and J. E. Lisman, “Bidirectional synaptic plasticity induced by a single burst

during cholinergic theta oscillation in CA1 in vitro,” Neuron, vol. 15, no. 5, pp. 1053–1063,

Nov. 1995.

[236] A. R. Wyler, F. Curtis Dohan, J. B. Schweitzer, and A. D. Berry, “A grading system for

101

mesial temporal pathology (hippocampal sclerosis) from anterior temporal lobectomy,” J.

Epilepsy, vol. 5, no. 4, pp. 220–225, 1992.

[237] B. W. Corrigan, R. A. Gulli, G. Doucet, and J. C. Martinez-Trujillo, “Characterizing Eye

Movement Behaviors and Kinematics of Non-Human Primates During Virtual Navigation

Tasks,” J. Vis., vol. 17, p. Forthcoming, 2018.

[238] M. M. Chun and N. B. Turk-Browne, “Interactions between attention and memory,” Curr.

Opin. Neurobiol., vol. 17, no. 2, pp. 177–184, Apr. 2007.

[239] M. Mather, D. Clewett, M. Sakaki, and C. W. Harley, “Norepinephrine ignites local hotspots

of neuronal excitation: How arousal amplifies selectivity in perception and memory,” Behav.

Brain Sci., vol. 39, p. e200, Jul. 2016.

[240] S. A. Hillyard, E. K. Vogel, and S. J. Luck, “Sensory gain control (amplification) as a

mechanism of selective attention: electrophysiological and neuroimaging evidence,” Philos.

Trans. R. Soc. London B Biol. Sci., vol. 353, no. 1373, 1998.

[241] F. Briggs, G. R. Mangun, and W. M. Usrey, “Attention enhances synaptic efficacy and the

signal-to-noise ratio in neural circuits.,” Nature, vol. 499, no. 7459, pp. 476–80, Jul. 2013.

[242] M. Aly and N. B. Turk-Browne, “How Hippocampal Memory Shapes, and Is Shaped by,

Attention,” in The Hippocampus from Cells to Systems, Cham: Springer International

Publishing, 2017, pp. 369–403.

[243] M. Sarter, B. Givens, and J. P. Bruno, “The cognitive neuroscience of sustained attention:

where top-down meets bottom-up,” Brain Res. Rev., vol. 35, no. 2, pp. 146–160, 2001.

[244] F. C. Fortenbaugh, J. DeGutis, and M. Esterman, “Recent theoretical, neural, and clinical

advances in sustained attention research,” Ann. N. Y. Acad. Sci., vol. 1396, no. 1, pp. 70–91,

May 2017.

[245] R. J. Perry, P. Watson, and J. R. Hodges, “The nature and staging of attention dysfunction in

early (minimal and mild) Alzheimer’s disease: relationship to episodic and semantic memory

impairment.,” Neuropsychologia, vol. 38, no. 3, pp. 252–71, 2000.

[246] A. M. Berardi, R. Parasuraman, and J. V Haxby, “Sustained attention in mild Alzheimer’s

disease.,” Dev. Neuropsychol., vol. 28, no. 1, pp. 507–37, 2005.

[247] M. T. deBettencourt, K. A. Norman, and N. B. Turk-Browne, “Forgetting from lapses of

sustained attention,” Psychon. Bull. Rev., Jun. 2017.

[248] T. Womelsdorf, S. Ardid, S. Everling, and T. A. Valiante, “Burst Firing Synchronizes

Prefrontal and Anterior Cingulate Cortex during Attentional Control,” Curr. Biol., vol. 24,

no. 22, pp. 2613–2621, Nov. 2014.

[249] A. N. Landau, H. M. Schreyer, S. van Pelt, and P. Fries, “Distributed Attention Is

102

Implemented through Theta-Rhythmic Gamma Modulation,” Curr. Biol., vol. 25, no. 17, pp.

2332–2337, Aug. 2015.

[250] O. Jensen, J. Kaiser, and J.-P. Lachaux, “Human gamma-frequency oscillations associated

with attention and memory,” Trends Neurosci., vol. 30, no. 7, pp. 317–324, Jul. 2007.

[251] W. Klimesch, “Alpha-band oscillations, attention, and controlled access to stored

information,” Trends Cogn. Sci., vol. 16, no. 12, pp. 606–617, Dec. 2012.

[252] U. Rutishauser, I. B. Ross, A. N. Mamelak, and E. M. Schuman, “Human memory strength

is predicted by theta-frequency phase-locking of single neurons,” Nature, vol. 464, no. 7290,

pp. 903–907, 2010.

[253] L. Marshall and J. Born, “The contribution of sleep to hippocampus-dependent memory

consolidation,” Trends Cogn. Sci., vol. 11, no. 10, pp. 442–450, 2007.

[254] Y. Ezzyat et al., “Targeted brain stimulation to modulate memory performance in humans,”

pp. 1–21, 2015.

[255] M. M. Bradley and P. J. Lang, “Memory, emotion, and pupil diameter: Repetition of natural

scenes,” Psychophysiology, vol. 52, no. 9, pp. 1186–1193, 2015.

[256] C. Zelano et al., “Nasal Respiration Entrains Human Limbic Oscillations and Modulates

Cognitive Function,” J. Neurosci., vol. 36, no. 49, pp. 12448–12467, Dec. 2016.

[257] H. Eichenbaum, “The role of the hippocampus in navigation is memory,” J. Neurophysiol.,

vol. 117, no. 4, pp. 1785–1796, 2017.

103

9.0 Appendices

9.1 Appendix A: Consent Form to Participate in Neuropsychological

Testing

Version 6 – August 23, 2017

CONSENT TO PARTICIPATE IN A RESEARCH STUDY

Study Title

Neuropsychological and Neurophysiological Testing in Functional

Neurosurgery Patients

Principal Investigator Taufik Valiante MD PhD Dept of Neurosurgery

Toronto Western Hospital

University Health Network

4th Floor 4W 436

Phone : Co-Investigators Dr. Andres M. Lozano, MD, PhD Dr. Nir Lipsman, MD, PhD Dept of Neurosurgery Dept of Neurosurgery

Toronto Western Hospital Toronto Western Hospital

University Health Network University Health Network

4th Floor 4W 431 4th Floor 4W 431

Phone : Phone :

Dr. Christopher Honey PhD Dr. Thilo Womelsdorf PhD Johns Hopkins University York University,

3400 N. Charles Street Department of Biology, Faculty of Science, Ames Hall, Rm 227 4700 Keele Street,

Baltimore, MD 21218 Toronto, Ontario,

USA Tel.

Dr. Kari Hoffman PhD

Centre for Vision Research Depts of Psychology, Biology

York University

Phone:

104

Funding Source No funding is required for this study

24 Hour Pager Number Dr. Nir Lipsman

Neurosurgery Resident, Study Co-ordinator

Page 1/5

105

Version 6 – August 23, 2017

Introduction

You are being asked to take part in a research study. Please read this explanation about the study

and its risks and benefits before you decide if you would like to take part. You should take as

much time as you need to make your decision. You should ask the study doctor or study staff to

explain anything that you do not understand and make sure that all of your questions have been

answered before signing this consent form. Before you make your decision, feel free to talk about

this study with anyone you wish. Participation in this study is voluntary. Background and Purpose

You are currently a patient who is either being considered for or who has already undergone a

functional neurosurgery procedure. Functional neurosurgery is an area of neurosurgery where the

goal is to alter the functioning of the brain in order to relieve symptoms of specific disorders, such

as epilepsy, and other neurologic and psychiatric conditions. Surgery is done in order to modulate

(or change) activity in the brain as well as to record activity from within it in order to learn more

about its functions. There is currently much interest in the scientific field in learning which structures in the brain

participate in memory, attention and decision-making. Since there is not one specific part of the

brain that serves these functions, studying various brain regions, and how they interact, is the best

way to answer these questions. This study involves testing those functions in the brain, using

computerized tasks, before surgery as well as after surgery, to see if modulating (or changing)

activity within the brain will influence performance of these tasks. These tasks will be simple,

and will involve things like reading words, looking at faces or pictures, or playing a game. If you

are a patient in the epilepsy monitoring unit, we will be recording activity from your brain as you

perform these tasks. If you are a patient in clinic (before or after surgery) we will be measuring

just your performance on the tasks. The purpose of this study is to investigate which regions of the brain participate in memory,

attention and decision-making decisions, and to see whether functional neurosurgery procedures,

influence performance of these tasks. You will not notice or feel anything while activity is being

recorded from your brain, or when you are engaged in any of these tasks. This is a non-invasive

study, does not require any procedure, and will not affect your treatment, or surgical plan. This is

a research study, and the experimental results obtained will in no way influence your clinical

management or have any impact on your condition or its treatment. Study Design

Several tasks have been developed to test the functional activity of your brain, and they include

memory, emotion and decision making tasks. Depending on the condition you have and the

procedure you are being considered for, you may undergo some or all of these tasks. They are all

non-invasive, and require no procedure to be done, other than sitting quietly at a computer. For each condition and task, we hope to include approximately 10 patients. Approximatley a

total of 150 patients will be in the study. It is important to remember that:

Page 2/5

106

Version 6 – August 23, 2017 i. Not every patient will undergo every task as the number of tasks assigned will be

determined by patient’s condition (e.g. epilepsy patients may undergo an emotion, memory

and decision making task, but an Alzheimer’s patient, just a memory task). ii. Each task can last up to 10-30 minutes, however a total visit duration may be up to 2 hrs. iii. We will repeat the same tasks at 6 or 12 months after surgery (in specific

patient populations). This study involves you looking at a computer laptop and participating in several tasks. Some

tasks will involve reading words, others will involve looking at pictures, and some will involve

playing a simple computer game. Each study task will last between 10 minutes and 30 minutes.

You will have the option, at all times, to stop the study task, should you feel tired. You will also

have the option to continue the study tasks on another day. All parts of this study will take place at the University Health Network. Depending on your

condition, thse tasks will be done either in the outpatient psychiatry, outpatient neurosurgery or in

the epilepsy monitoring unit (EMU). All identifiable study materials will be kept in a safe, and

secure location at the University Health Network. Unidentifuable data may stored securely offsite

in the course of this study. Also there will not be a need for blood work, neuroimaging or any

invasive tests.

Reminders

It is important to remember the following things during this study: There is no need to adjust your medications or treatments in any way if you are to

participate in this study

Ask your study team about anything that worries you. Tell your study team if you change your mind about being in this study.

Risks Related to Being in the Study

There are no medical risks if you take part in this study, but being in this study, and participating

in some of these tasks, may make you feel tired or fatigued. You may refuse to answer questions

or stop the study task(s) at any time if there is any discomfort. Participating in this study means

that it will prolong your visit to your doctor (psychiatrist or neurosurgeon) by approximately 30-

60 minutes. Should you be an inpatient in the EMU, participation will not prolong your stay in

the hospital. Participation in this study will not influence your treatment or surgical plan in any

way. Also, for patients in the EMU, should you have a seizure, you will be removed from the

study, and standard treatment procedures will take place. There is a chance that some tasks may elicit negative feelings in some patients. For example,

some tasks involve looking at facial emotional expressions, which some patients may find

disturbing or uncomfortable. Further, some tasks will test memory, and patients with memory

disturbances, such as Alzheimer’s Disease, may find this uncomfortable. We will make every

effort to minimize discomfort and inconvenience, and you have the right to stop testing at any

point.

Page 3/5

107

Version 6 – August 23, 2017

Benefits to Being in the Study

You will not receive any direct benefit from being in this study. Information gained from this

study may help us learn about the functioning of the brain, such as emotions, decision-making,

reward, memory and attention, and may one day help others with brain dysfunction. Voluntary Participation

Your participation in this study is voluntary. You may decide not to be in this study, or to be in

the study now and then change your mind later. You may leave the study at any time without

affecting your care. You may refuse to answer any question you do not want to answer, or not

answer an interview question by saying “pass”. We will give you new information that is learned during the study that might affect your decision

to stay in the study. Alternatives to Being in the Study

This study is not designed to test a new treatment or intervention. Your alternative to

participation is not to participate. Confidentiality

If you agree to join this study, the study doctor and his/her study team will look at your personal

health information and collect only the information they need for the study. Personal health

information is any information that could be used to identify you and includes your:

• name,

• address, • date of birth, • new or existing medical records, that includes types, dates and results of medical tests

or procedures. The information that is collected for the study will be kept in a locked and secure area by the

study doctor for 10 years. Only the study team or the people or groups listed below will be

allowed to look at your records. Your participation in this study also may be recorded in your

medical record at this hospital. Representatives of the University Health Network Research Ethics Board may look at the study

records and at your personal health information to check that the information collected for the

study is correct and to make sure the study followed proper laws and guidelines. All information collected during this study, including your personal health information, will be

kept confidential and will not be shared with anyone outside the study unless required by law.

You will not be named in any reports, publications, or presentations that may come from this

study. If you decide to leave the study, the information about you that was collected before you left the

study will still be used. No new information will be collected without your permission.

Page 4/5

108

Version 6 – August 23, 2017

In Case You Are Harmed in the Study

If you become ill, injured or harmed as a result of taking part in this study, you will receive care. The

reasonable costs of such care will be covered for any injury, illness or harm that is directly a result of

being in this study. In no way does signing this consent form waive your legal rights nor does it relieve

the investigators, sponsors or involved institutions from their legal and professional responsibilities.

You do not give up any of your legal rights by signing this consent form. Conflicts of Interest The hospital and researchers have an interest in completing this study. Their interests should not

influence your decision to participate in this study. You should not feel pressured to join this study Expenses Associated with Participating in the Study

You will not have to pay for any of the procedures involved with this study.

Questions About the Study

If you have any questions, concerns or would like to speak to the study team for any reason,

please call: If you have any questions about your rights as a research participant or have concerns about this

study, call the Research Ethics Board Chair or the Research Ethics office at XX. The REB is a

group of people who oversee the ethical conduct of research studies.These people are not part

of the study team. Everything that you discuss will be kept confidential.

Consent

This study has been explained to me and any questions I had have been answered. I know that I may leave the study at any time. I agree to take part in this study.

Print Study Participant’s Name Signature Date

(You will be given a signed copy of this consent form)

My signature means that I have explained the study to the participant named above. I have

answered all questions.

Print Name of Person Obtaining Consent Signature Date

Page 5/5

109

9.2 Appendix B: Consent Form to Participate in Stimulation Study

CONSENT FORM TO PARTICIPATE IN A RESEARCH STUDY

Study Title: Contingent Stimulation for Augmentation of Memory and Theta

Rhythm s in Humans

Study Doctor: Taufik Valiante MD PhD

Department of Neurosurgery

Toronto Western Hospital

University Health Network

4th Floor 4W 436

Funders: NSERC and Brain Canada

Introduction

You are being asked to take part in a research study. Please read this explanation about

the study and its risks and benefits before you decide if you would like to take part. You

should take as much time as you need to make your decision. You should ask the study

doctor or study staff to explain anything that you do not understand and make sure that

all of your questions have been answered before signing this consent form. Before you

110

make your decision, feel free to talk about this study with anyone you wish. Participation

in this study is voluntary.

Background and Purpose

Memory deterioration is a significant health issue and the age of the world’s population

is only increasing. Memory deficits cause severe impairment of quality of life in

individuals with epilepsy. Approximately 0.6% of Canadians have epilepsy and it is widely

acknowledged that memory is severely impaired in temporal lobe epilepsy (TLE). TLE is

the most important focal epilepsy because of its high prevalence, its potential for drug-

resistance, and its commonly disabling effects on memory functions.

You are currently a patient with drug resistant temporal lobe epilepsy who is being

considered for a neurosurgery procedure and will undergo surface electrode monitoring

or implantation of electrodes to the surface and within the brain (intracranial), to record

seizures in order to locate your seizure onset more accurately. This procedure is being

done in order to determine your eligibility as a surgical candidate.

To determine surgical candidacy, you will undergo a number of tests including structural

magnetic resonance imaging (MRI), neuropsychological testing and admission to an

epilepsy-monitoring unit (EMU). Other tests may be required following these non-

invasive tests, including invasive electroencephalography (iEEG),

magnetoencephalography (MEG), and positron emission tomography (PET).

These monitoring procedures give the researchers the opportunity to investigate

structures in the brain that participate in memory, attention and decision-making. Since

there is not one specific part of the brain that serves these functions, studying various

brain regions, and how they interact, is the best way to answer these questions. This

study involves testing those functions in the brain, using computerized tasks.

There is currently much interest in the scientific field of neurostimulation (using

electrodes to stimulate parts of the nervous system, in particular the brain to stop seizure

111

formation). This study focuses on neurostimulation, which uses the information gathered

from your iEEG recordings and ongoing behavioural tasks to learn about changing

memory performance. This will be done during your stay at the Epilepsy monitoring unit

(EMU) where you will be monitored for several days to possibly weeks through video and

iEEG recordings to locate the source of your seizures and epilepsy process.

The purpose of this study is to investigate how well neurostimulation can change your

memory performance. You may feel some discomfort, however similar studies have

noted that patients are unaware when the neurostimulation is happening. This is a

research study and the experimental results obtained will in no way influence your clinical

management or have any impact on your condition and treatment plan. Ten people will

participate in this study.

Study Design

Behavioural Aspect

Several tasks have been developed within our lab to test the functional activity of your brain.

These include memory, emotion and decision making tasks. Depending on how you feel

throughout the experiment you may undergo some or all of these tasks. They are all non-invasive,

and require no procedure to be done, other than sitting quietly at a computer.

It is important to remember that:

i. Not every patient will undergo every task as the number of tasks assigned will be

determined by the patient’s condition (e.g. epilepsy patients may undergo an emotion,

memory and decision making task if time permits through their stay)

ii. Each task can last up to 10-30 minutes, however the total visit duration may be up to 2

hrs.

iii. We will repeat the same task(s) during two separate sessions over the course of your stay

in the EMU.

112

This study involves you looking at a computer laptop and participating in several tasks. Some

tasks will involve reading words, others will involve looking at pictures, and some will involve

playing a simple computer game such as searching for an object within a scene. Each study task

will last between 10 minutes and 30 minutes. You will have the option, at all times, to stop the

study task, should you feel tired. You will also have the option to continue the study tasks on

another day.

Stimulation Aspect

You will undergo implantation of electrodes (on and inside the brain) for intracranial

recordings as part of your standard clinical care in the EMU. Electrodes localized in the

hippocampus (area of the brain that controls memory) will be selected for stimulation.

A computer program will record your brain activity and an eye tracker will record your

eye movements. They will be used to find specific timings during the behavioural task to

stimulate that region of the brain. This eye tracker has been used in many previous

studies as well as other behavioural studies.

Electrical stimulation in humans has been done in epilepsy patients before to improve

cognitive performance (thinking and reasoning abilities), and we would like to determine

if the same effectiveness is possible in the human temporal lobe epilepsy population with

these specific timings and behavioural tasks.

Risks Related to being in the Study

Stimulating the area of the brain found to cause the seizures has the possibility of causing

an extra seizure, although this is unlikely given that many studies have found that brain

stimulation during an oncoming seizure actually stops the seizure. Also, the frequency

with which we will stimulate the brain will be well below that which usually causes the

start of a seizure.

113

No matter the unlikeliness, the risks are present and you may stop the study at any time

if there is any discomfort or if you decide to no longer participate in the study.

Participating in this study means you will be subjected to brain stimulation at various

times during behavioural tasks. Aside from the known risks of the process of iEEG

recordings, the following are potential risks and serious adverse events involved with this

study intervention:

• Simple partial seizure (sensory) (1%)

• Complex partial seizures (0.5%-5.2%)

• Tonic clonic seizures increase (0.5%-3.7%)

• Memory impairment (0.5%)

• Tonic-clonic seizures exacerbated (0.5%)

• Simple partial seizures (motor) (0.5%)

In the EMU, participation will not prolong your stay in the hospital. Participation in this

study will not influence your treatment or surgical plan in any way.

Benefits

You will not receive any direct benefit from being in this study. Information learned from

this study may help people with epilepsy and cognitive impairments in the future.

Alternatives to Being in the Study

Your alternative to participating is not to participate. If you choose not to participate in

this study, you will continue to receive the usual standard of care and your care will not

be affected by your decision in any way.

114

Confidentiality

Personal Health Information

If you agree to join this study, the study doctor and his/her study team will look at your

personal health information and collect only the information they need for the study.

Personal health information is any information that could identify you and includes your:

• name,

• address,

• date of birth,

• new or existing medical records, that includes types, dates and results of medical

tests or procedures.

Representatives of the University Health Network (UHN) including the UHN Research

Ethics Board may come to the hospital to look at the study records and at your personal

health information to check that the information collected for the study is correct and to

make sure the study is following proper laws and guidelines.

The study doctor will keep any personal health information about you in a secure and

confidential location for 10 years. A list linking your study number with your name will be

kept by the study doctor in a secure place, separate from your study file.

Your participation in this study will also be recorded in your medical record at this

hospital. This is for clinical safety purposes.

Research Information in Shared Clinical Records

If you participate in this study, information about you from this research project may be

stored in your hospital file and in the UHN computer system. The UHN shares the patient

information stored on its computers with other hospitals and health care providers in

Ontario so they can access the information if it is needed for your clinical care. The study

team can tell you what information about you will be stored electronically and may be

115

shared outside of the UHN. If you have any concerns about this, or have any questions,

please contact the UHN Privacy Office at 416-340-4800, x6937 (or by email at

privacy@uhn.ca).

Study Information that Does Not Identify You

Some study information will be sent outside of the hospital to the sponsor. Any

information about you that is sent out of the hospital will have a code and will not show

your name or address, or any information that directly identifies you.

All information collected during this study, including your personal health information, will

be kept confidential and will not be shared with anyone outside the study unless required

by law. You will not be named in any reports, publications, or presentations that may

come from this study.

Voluntary Participation

Your participation in this study is voluntary. You may decide not to be in this study, or to

be in the study now, and then change your mind later. You may leave the study at any

time without affecting your care. We will give you new information that is learned during

the study that might affect your decision to stay in the study.

Withdrawal from Study

You will be withdrawn from this study should you express an unwillingness to continue

to participate in the contingent neurostimulation trial. You will have the option, prior to

starting the stimulation process, to participate or not and your voluntary decision will be

respected. If your iEEG fails to locate the seizure focus or there is evidence of bilateral

seizure focus you will also be withdrawn from the study.

116

Costs and Reimbursement

You will not have to pay for any of the procedures involved with this study.

Rights as a Participant

If you are harmed as a direct result of taking part in this study, all necessary medical treatment

will be made available to you at no cost.

By signing this form you do not give up any of your legal rights against the investigators,

sponsor or involved institutions for compensation, nor does this form relieve the

investigators, sponsor or involved institutions of their legal and professional

responsibilities

Conflicts of Interest

The hospital and researchers have an interest in completing this study. Their interests

should not influence your decision to participate in this study. You should not feel

pressured to join this study.

Questions about the Study

If you have any questions, concerns or would like to speak to the study team for any

reason, please call

If you have any questions about your rights as a research participant or have concerns

about this study, call the Research Ethics Board Chair or the Research Ethics office. The

REB is a group of people who oversee the ethical conduct of research studies. These

117

people are not part of the study team. Everything that you discuss will be kept

confidential.

Consent:

This study has been explained to me and any questions I had have been answered.

I know that I may leave the study at any time. I agree to take part in this study.

Prin S udy P r icip n ’s N m Signature Date

(You will be given a signed copy of this consent form)

My signature means that I have explained the study to the participant named above. I

have answered all questions.

_

Print Name of Person Obtaining Consent Signature Date