Combining Optical Brain Imaging and Physiological Signals to Study Cognitive Function

Combining Optical Brain Imaging and Physiological Signals to Study Cognitive Function

Experts discuss the fundamentals of fNIRS and present new research capabilities enabled through the integration of optical brain imaging technology and physiological recording systems.

InsideScientific is an online educational environment designed for life science researchers. Our goal is to aid in

the sharing and distribution of scientific information regarding innovative technologies, protocols, research tools

and laboratory services.

http://insidescientific.com/

Continuous Wave Functional Near Infrared (fNIR) Spectroscopy Imaging Systems

www.fnirdevices.com 3© fNIR Devices LLC. 2016

Optical Brain Imaging: Physiological and Physical Principles

Hasan Ayaz, PhDAssociate Research Professor, Drexel University

http://fnirdevices.com/



Functional Near-Infrared Monitor [fNIR]

>> Cortical activation-related to hemodynamic changes in natural environments and diverse field conditions

Brain Activity Monitor



Functional Near-Infrared Monitor [fNIR]

>> Cortical activation-related to hemodynamic changes in natural environments and diverse field conditions

Brain Activity Monitor



Physiological Principles of fNIR:

• Neurons consume energy (glucose) when activated

• Oxygen is required to metabolize the glucose

• As clusters of neurons are activated, there is an increased need for oxygen in that area

• Oxygen is transported to neural tissue via oxy-hemoglobin in the blood

Oxy-Hb and deoxy-Hb are correlates of brain activity through oxygen consumption by neurons

• The oxygen exchange occurs in the capillary beds

• As oxy-hemoglobin gives up oxygen to the neural tissue, it is transformed into deoxygenated hemoglobin

David J. Heeger & David Ress. (2002) What does fMRI tell us about neuronal activity? Nature Reviews Neuroscience 3, 142-151

Neural Activity & Hemodynamic Response

http://www.nature.com/nrn/journal/v3/n2/abs/nrn730.html



Physical Principles of fNIR: Photon Migration in Tissue



Physical Principles of fNIR: Photon Migration in Tissue

H Obrig , R Wenzel, M Kohl, S Horst, P Wobst, J Steinbrink, F Thomas, A Villringer. Near-infrared

spectroscopy: does it function in functional activation studies of the adult brain? International

Journal of Psychophysiology 35(2-3):125-142, 2000

Photons that enter the tissue undergo two types of interaction:

1. Scattering(cell membranes)

2. Absorption(Hb, HbO2, water)

http://www.ncbi.nlm.nih.gov/pubmed/10677642



fNIR Instrumentation

Modern Portable System (Desktop)

2010…






2010…




2015…






McKendrick, R., Ayaz, H., Olmstead, R., & Parasuraman, R. (2014). Enhancing dual-task performance with verbal and spatial working memory training: Continuous monitoring of cerebral hemodynamics with NIRS. Neuroimage, 85, Part 3(0), 1014-1026.

Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A., McKendrick, R., & Parasuraman, R. (2013). Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: Empirical examples and a technological development. Frontiers in Human Neuroscience, 7, 1-13.

Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B., & Onaral, B. (2012). Optical brain monitoring for operator training and mental workload assessment. Neuroimage, 59(1), 36-47.

Select References

2015…






• Cell-phone sized, • Battery operated,• Fast-setup/no gel

required• Suitable for field

applications

Modern Wireless system (Pocket)




• Cell-phone sized, • Battery operated,• Fast-setup/no gel

required• Suitable for field

applications

Modern Wireless system (Pocket)

fNIR Instrumentation Select References

Mckendrick, R., Parasuraman, R., Murtza, R., Formwalt, A., Baccus, W., Paczynski, M., & Ayaz, H. (2016). Into The Wild: Neuroergonomic Differentiation of Hand-Held and Augmented Reality Wearable Displays During Outdoor Navigation with Functional Near Infrared Spectroscopy. Frontiers in Human Neuroscience, 10.

McKendrick, R., Parasuraman, R., & Ayaz, H. (2015). Wearable functional Near Infrared Spectroscopy (fNIRS) and transcranial Direct Current Stimulation (tDCS): Expanding Vistas for Neurocognitive Augmentation. Frontiers in Systems Neuroscience, 9(27).

Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A., McKendrick, R., & Parasuraman, R. (2013). Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: Empirical examples and a technological development. Frontiers in Human Neuroscience, 7, 1-13.






fNIR Signal Measurement

Absorption spectrum of main chromophores in tissue:

• Low absorption between 700-900nm provides an optical window to the tissue




Modified Beer Lambert Law

(MBLL) allows calculation of

oxygenation changes from

light intensity measures.

Modified Beer Lambert Law

Can be solved for concentration changes for non-

singular F matrix




MBLL is repeated

continuously to generate

time-series each for oxy-

Hb, deox-Hb, total-Hb,

and oxygenation.



• Raw fNIR signals are light intensity values at detectors

• The physiologically irrelevant data (such as respiration and heart pulsation effects) and equipment noise, etc. are first eliminated from the raw fNIR measurements

• Saturation, motion artifact and other noise needs to be eliminated (or excluded from analysis

Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., & Onaral, B. (2011). Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial Navigation. J Vis Exp(56), e3443. doi: 10.3791/3443

fNIR Signal Processing





Motion Artifact Detection & Removal

• Low-pass / Band-pass Filters

• Wavelet Analysis

• Independent Component Analysis (ICA)

• Principle Component Analysis (PCA)

• Coefficient of Variance related (SMAR, etc.)

• Optimal Filtering (Adaptive; Wiener; Kalman)













Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., & Onaral, B. (2011). Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial Navigation. J Vis Exp(56), e3443. doi: 10.3791/3443

Izzetoglu, M., Chitrapu, P., Bunce, S., & Onaral, B. (2010). Motion artifact cancellation in NIR spectroscopy using discrete Kalmanfiltering. Biomed Eng Online, 9(1), 16.

Ayaz, H., Izzetoglu, M., Shewokis, P. A., & Onaral, B. (2010). Sliding-window Motion Artifact Rejection for Functional Near-Infrared Spectroscopy. Conf Proc IEEE Eng Med Biol Soc, 6567-6570.

Izzetoglu, M., Devaraj, A., Bunce, S., & Onaral, B. (2005). Motion artifact cancellation in NIR spectroscopy using Wiener filtering. IEEE Trans Biomed Eng, 52(5), 934-938.









Select References







Raw light intensity HbO/HbR Changes

Filtered HbO/HbRTask/condition blocks (time synced markers)Mean, Median, Stddev, Time-Peak Range, Min, Max

Preprocessing(And. LP, detrending, etc.)

Beer–Lambert Law

Subject X, Ch1

0 100 200 300 400 500-1.5

-1

-0.5

0HbR/HbO Raw

HbR

HbO

0 100 200 300 400 500-0.2

-0.1

0

0.1

0.2HbR/HbO filtred (10Hz)

HbR

HbO

0 100 200 300 400 500-0.2

-0.1

0

0.1

0.2HbR/HbO filtred (1Hz)

HbR

HbO Feature Extraction

0 100 200 300 400 5002

2.2

2.4

2.6

2.8

695nm

830nm

730nm

850nm

3000

2500

2000




To learn more about fNIR Devices technology, visit www.fnirdevices.com





Optical Brain Imaging: Field Applications

Kurtulus Izzetoglu, PhD Associate Research Professor, Drexel University




Non-invasive ‘anesthesia care’ monitoring to detect depth of anesthesia and over sedation

fNIR to Monitor Depth of Anesthesia



deoxy-Hb displayed very slow rate of change in deep anesthesia,

whereas this rate of change is significantly increased when the patient emerges to wakefulness.




deoxy-Hb Changes in Voxel 12

0.000

0.500

1.000

1.500

2.000

2.500

3.000

3.500

X-4min X-3min X-2min X-1min X Y-

4min

Y-

3min

Y-

2min

Y-

1min

Y

Time

mic

ro

Mo

lar

Wound Closure Eye Opening

………..

3.4% {

48.8%{

Deep Anesthesia Light Anesthesia

deoxy-Hb displayed very slow rate of change in deep anesthesia,

whereas this rate of change is significantly increased when the patient emerges to wakefulness.


Izzetoglu K, et al (2011). The Evolution of Field Deployable fNIR Spectroscopy From Bench to Clinical Settings. J of Innovative Optical Health Sciences

http://www.worldscientific.com/doi/abs/10.1142/S1793545811001587

http://www.worldscientific.com/doi/abs/10.1142/S1793545811001587



fNIR for assessment of

cognitive impairment

and recovery following

traumatic brain injury

(TBI)

Healthy Subject TBI Patient

Neurorehabilitation

Merzagora A.C., Butti M. et al., Clinical Neurophysiology , 2009




Cz

-300 700

-20

20

Time (ms)

Am

pli

tud

e (

V)

Pz

Non-Target

Target

-300 700

-20

20

Time (ms)

Am

pli

tud

e (

V)

-0.015

0.01

12

Time (s)

o

xyg

en

ati

on

(

M)

fNIR EEG

fNIR & EEG combined to

study and assess the

cognitive impairments

of the patients in their

everyday life activities.

Neurorehabilitation

Merzagora A.C., Butti M. et al., Clinical Neurophysiology , 2009




Holtzer, R, et al (2015) Online fronto-cortical control of simple and attention-demanding locomotion in humans. Neuroimage

Assessment of cortical control of locomotion, using fNIR, was highly reproducible.

Oxygenation levels were increased and maintained in attention-demanding locomotion.

Definitive role for the PFC in higher-order control of locomotion.

fNIR to determine online cortical control of locomotion in humans.

Cognitive Aging




Air Traffic Controller

Cognitive Workload Monitor

UAV Ground Operator

Training & Workload Monitor

Pilot

Expertise Development Monitor

fNIR can “bring more of brain on task”* …enable human in the loop studies

*DARPA: Dylan Schmorrow, Ph.D. LCDR, MSC, US Navy, Program Manager, ITO

Human Performance Assessment

Safe & Effective Piloting



Air Traffic Controller Workload

FAA Next Generation Air Transportation System (NextGen) Study

ATC Part-Task

Communication type: Voice and Data

Workload manipulated by number of aircrafts in the sector: n= 6,12,18

Ayaz, H., et al., Optical brain monitoring for operator training and mental workload assessment , NeuroImage (2012),






FAA Next Generation Air Transportation System (NextGen) Study

ATC Part-Task

Communication type: Voice and Data

Workload manipulated by number of aircrafts in the sector: n= 6,12,18






Oden, K., et al (2015) Empirical Support for Brain-Based Assessment in Simulation-Based Training

objective measure of expertise level to validate adequate training

real-time monitor of mental workload

Pilot Expertise-Workload Assessment















• Amenable to integration with other sensors – multi modality monitoring

• Shorter preparation time – easy to calibrate and baseline

• Less intrusive – critical in field studies

• Easy to interpret and relate to task-dependent analyses – direct measure of the activity

• Lends itself to easy engineering/customization – important to support ‘translational’ research

fNIR For Natural Environments & Field Conditions

Demonstration of fnirSoft Software

Click To Access Video

https://attendee.gotowebinar.com/register/5715954510859071236

How To Integrate fNIR Data With Other Physiological Signals

Frazer Findlay CEO, BIOPAC Systems, Inc.

BIOPAC—Inspiring people and enabling discovery about life© BIOPAC Systems, Inc. 2016

www.biopac.com 40

Over 97% of top universities run BIOPAC Systems

THE WORLD DISCOVERSWITH BIOPACSolutions for life science research and education


www.biopac.com 41

Combining fNIR with other Physiological Signals

BioNomadix

Wireless Physiological

Data

Mobita System

32-Channel EEG / Biopotentials

B-Alert System

9-Channels of EEG and Cognitive States Metrics

MP36R System

4-Universal Channels

MP160 System

16-Channels of Tethered or

Wireless Signals

To learn more about BIOPAC solutions for signal integration, visit www.biopac.com

https://www.biopac.com/


www.biopac.com 42

Stimulus Presentation with Triggers for Synchronization

SuperLab® - E-Prime®Presentation®

Virtual Reality

BIOPAC VR Solutions offer real-time feedback loops and event marking


www.biopac.com 43

Stationary Eye Tracking

with Visual StimulationUnrestrained Mobile

Eye Tracking


www.biopac.com 44

Facial Expressions and Video Synchronization


www.biopac.com 45

fNIR Imager 1000

Continuous wave NIR spectroscopy control box

and 16 channel sensor

STP100C Trigger BNC;

fNIR trigger via BNC

MP160 System

16-Channels Wired or

Wireless Unit

Hardware Communication

Front

FrontBack


www.biopac.com 46

Front

FrontBack

Hardware Communication

MP160 receives select trigger information from fNIRcontrol unit…

START STOP BASELINE BEGIN BASELINE END RECORD


www.biopac.com 47

Hardware Communication with Visual Stimulus

Visual stimulus is delivered to subject at a stationary monitor or by VR...

Front

FrontBack


www.biopac.com 48

Hardware Communication with Visual Stimulus

Visual stimulus is delivered to subject at a stationary monitor or by VR...

Triggers sent to the MP160 System and fNIR computer

Demonstration of AcqKnowledge Software

Click To Access Video



www.biopac.com 50

• Combine fNIR with a variety physiological monitoring systems for– ECG, EEG, EMG, EOG, EDA, Respiration, etc.

• Interface with stimulus presentation systems such as E-Prime, SuperLab, Presentation and BIOPAC Virtual Reality systems

• Monitor subject eye position, gaze path and facial expressions

• Maintain synchronization across devices

Summary

Thank You!

Kurtulus Izzetoglu, PhD Frazer FindlayHasan Ayaz, PhD

If you have questions for the presenters please contact them by email.

For additional information on the solutions presented in this webinar please visit:

http://www.biopac.com http://www.fnirdevices.com Hasan Ayaz, PhD [email protected]

Kurtulus Izzetoglu, PhD [email protected]

Frazer Findlay [email protected] To View Full Recording

https://www.biopac.com/


http://insidescientific.com/
