Distributed Computational Architectures for Integrated Time-Dynamic Neuroimaging

Distributed Computational Architectures forIntegrated Time-Dynamic Neuroimaging

Dr. Allen D. Malony

[email protected]

Computer & Information Science DepartmentComputational Science Institute

CIBER

University of Oregon

April 19, 2023 Hill Center

Who Am I

Associate Professor, CIS Department, UO Computer Science specialties / interests

parallel performance analysis (primary) environments computational science (secondary)

software development environments distributed and parallel computing environment

Cognitive Neuroscience interests two-year association with Don Tucker (Psychology, UO) Carmel Neuroinformatics workshop (2000, presentation) HBP Neuroinformatics Review Panel (2000, 2001) HBP Annual Meeting (2000, presentation)


Talk Outline

Computational science and cognitive neuroscience Brain dynamics analysis problem (my view)

integrated electromagnetic analysis system Motivating case studies

observations: computation and informatics Computational architectures

models and technology key ideas

Opportunities and the Neural Informatics Center Final Thoughts


Computational Science & Cognitive Neuroscience

Computational methods applied to scientific research high-performance simulation of complex phenomena large-scale data analysis and visualization

Understand functional activity of the human cortex multiple cognitive domains multiple experimental paradigms and methods

Need for coupled/integrated modeling and analysis electrical and magnetic, cortical and theoretical

Need for robust tools: computational & informatic

Problem solving environment for brain analysis


Brain Dynamics Analysis Problem (My View)

Identify functional components in cognitive contexts Interpret with respect to cognitive theoretical models Requirements: spatial (structure), temporal (activity) Imaging techniques for analyzing brain dynamics

blood flow neuroimaging (PET, fMRI) good spatial resolution functional brain mapping temporal limitations to tracking of dynamic activities

electromagnetic measures (EEG/ERP, MEG) msec temporal resolution to distinguish components spatial resolution sub-optimal (source localization) potential to map electrical activity to cortex surface


Electromagnetic Analysis Methodology Multi-trial analysis

signal analysis and response analysis averaging across subjects and trials (S/N ratio)

distortion (smearing) of estimated source response noise artifacts, signal variation (individuals, trials) improvements: artifact removal, selective averaging

create component response models ERP identification factor analysis: PCA, ICA, … error in source factors: variability, statistics

Multi-subject and single-subject analysis quantify differences of individual from population


Single-Trial Analysis Capability

Improve fidelity of single-subject response model higher information content than multi-trial/subject reduce analysis error from trial/subject variability knowledge of subject population, stimulus deviations

Diagnosis (identification) of cognitive state known stimulus blind stimulus match response to known component response model

Problems greater noise greater complexity


Single-Trial Analysis Methodology

Integrate methods for analyzing brain dynamics Improve resolution and robustness of techniques

increase measurement density (128 to 256 channels) Coupled modeling: constraints and cross-validation

component response model cortical activity model tuned models for single individual

Build models in experimental paradigm context Match single-trial measurements to models

known stimulus multiple trial models blind stimulus multiple stimulus/trial models

Training and learning


Integrated Electromagnetic Brain Analysis

Single-trialAnalysis

Structural /Functional

MRI

DenseArray EEG /

MEG

ConstraintAnalysis

Head Analysis

Source Analysis

Signal Analysis

Response Analysis

Experimentsubject

temporaldynamics

neuralconstraints

CorticalActivity Model

ComponentResponse Model

spatial patternrecognition

temporal patternrecognition

Cortical ActivityKnowledge Base

Component ResponseKnowledge Base

EEGMEG

Carmel Workshop

Integrated Electromagnetic Analysis System


Case Study: Readiness Potential

Self-paced button pressing task slow negative shifts in potential contralateral to hand

Single subject examination multi-trial (150 trials) averaged ERP analysis

Dense-array scalp electrical measurement 129 electrode array (EGI Geodesic Sensor Net)

Modeling of brain electrical activity MRI and CT data analysis with tissue segmentation realistic boundary element meshes (2K ’s for brain) source localization with dipole modeling

Can ERP analysis accurately localize cortical activity?


mesh generation,source localization constrained to cortical surface

processed EEG

Experimental Methodology

BrainVoyager

EMSE

CT / MRI

Interpolator 3D

NetStationEEG segmented

tissues

16x256bits permicrosec(30MB/m)


Electrical Activity of Scalp and Brain

Expected brain activity Correlated with fMRI

experimental studies Topographic and cortex

mapped spatial analysis

-404 ms -56 ms 0 ms 160 ms

Lateralize Readiness Potential (LRP)


Optimizing Spatial Resolution for ERP

Adequate spatial sampling Accurate head surface mapping Accurate sensor registration Measured skull conductivity Convergence with MEG MEG-compatible EEG Convergence with fMRI fMRI-compatible EEG Test spatial resolution with know pathological sources

EEG as link for converging analysis? What problems exist?


Electrical Impedance Tomography

Small (10µA) currents are injectedbetween electrode pair

Resulting potential is measuredfrom all remaining electrodes

Measures used to estimateconductivity of each tissue compartment

Boundary element forward solution 4-shell polyhedron model (1280 faces) direct (31244 sec) and iterative approaches (933 sec)

Finite element forward solution greater computational requirements


Case Study: Self-Monitored Motivated Action

Learning task with feedback (Gehring et al. (1993)) left- or right-hand button press response "incorrect" feedback on error "OK" or “late” feedback if correct timed expectancy and motivated response

Error-Related Negativity (ERN) large medial negative response on error self-monitoring when motivated action goes wrong

What is the nature and complexity of the ERN withrespect to dynamic components of brain activity?


Cognitive Experiments and Brain Dynamics

Visualize the dynamic operations of brain Example: fMRI blood flow response to reading a word Dense-array EEG / MEG frontal lobe activity (ERN)

significant changes in milliseconds frontal oscillations and separate time courses

BrainVoyager


ERN Analysis using ICA (Makeig, Salk Institute)

Average analysis smears temporal/spatial dynamics Single-trial analysis may expose greater detail Independent Components Analysis (ICA)

find independent EEG component contributors temporal and spatial components accounting for artifacts components accounting for functional sources (ERN)

analysis over single trials Two components account for averaged ERN

response-locked ERN difference wave dominated show temporal and functional independence


ERP and Component Envelopes (Left/Correct)

Component #2

Component #7

• Complementary behavior

• Both active at strongest ERN channels


ERPs averaged across response hand

Neither C2nor C7 explainthe waveforms

Component sumdoes explain the waveforms and shows ERN response


Topographic Imaging and Dipole Modeling

Component #2 Component #7

Averaged ERN

Brain ElectricalSource Analysis

(BESA)


ICA Component #2 Dynamics

Stimulus locked Memory of deadline


ICA Component #7 Dynamics

Phase reset byresponse, largestafter incorrect


Optimize Temporal Information

Inherent problem – both electrical and magnetic Trial averaging methodologies can mask dynamics Techniques to boost signal to noise ratio

Selective averaging Stimulus and response locking

Techniques to estimate time function fMRI timing models EEG/MEG time function for fMRI signal extraction

Single trial analysis with individual modeling

What problems exist?


Case Study Observations

Diverse set of tools function and implementation separate tools (monolithic) and not integrated incompatibilities and limitations for interoperation

Complex analysis processes multiple processes applied (process pipeline) high-level, hierarchical process methodology scientific discovery through integrated techniques heterogeneous, flexible, extensible capabilities increasingly high computational demands

Multiple, interdisciplinary scientific domains


High-Performance Computational Environments

Integrated database, analysis, and visualization Distributed tool infrastructure

diverse tools across multiple platforms interoperation requirements user interaction requirements support portability, flexibility, extensibility

Scalable, high-performance parallel computing increase data resolution minimize solution time

High-level access to tools web-based access


Computational Systems: Models and Technology

Domain-specific, problem-specific environments (PSE) TIERRA

Scientific “workbench” SCIRun

Programming environments numerical frameworks

POOMA application coupling

PVM / MPI CUMULVS PAWS SILOON / PDT

Metacomputing / GRID Legion Globus

Heterogeneous distributed computing / coupling NetSolve INTERLACE HARNESS

Web-based environments ViNE PUNCH VNC


TIERRA (Computational Science Institute, UO)

Tomographic Imaging Environment for Ridge Research and Analysis

High-performance, domain-specific environment for seismis tomography parallelized tomography code runtime distributed array access computational steering via MatLab frontend full problem solving process for seismic tomography

Led to new discoveries for three-dimensional melt migration beneath the East Pacific Rise


TIERRA Architecture

KEY IDEAS

Domain specific

Support for the entire process


SCIRun (Johnson, University of Utah)

Scientific programming environment large-scale simulations “computational workbench” visual programming interface dataflow model of computing

modules: operation or algorithm with I/O ports network: set of modules and their interconnections widgets: 3D user interaction

data types: Mesh, Surface, Matrix, Field, Geometry extensible module library computational steering


SCIRun User Interface

Visual programming lets users select, arrange, and connect modules into a desired network

Interactive steering of design, computation, and visualization allows more rapid convergence


ICA for EEG Source Localization with SCIRun

PCA decomposition forEEG signal/noise subspaces

ICA activity map separationon signal subspace

Solution to a single dipolesource forward problem underlying model is shown

in the MRI planes dipole source is indicated by red and blue spheres electric field visualized by cropped scalp potential

map and wire-frame equipotential isosurface

KEY IDEAS

Integrated application development environment

“Component-based” application programming

High-level data objects


POOMA (Advanced Computing Lab, LANL)

Parallel Object-Oriented Methods and Applications Goals

use object-oriented programming to help manage complexity of modern scientific simulation codes

extract physics content of simulations from details of parallel, high-performance computing

framework approach: allows flexible code structure, object reuse across problem domains

build upon standards to maintain code portability

An object-oriented framework for scientific computing applications on parallel computers


POOMA Approach

C++ class library high-level, generally data-parallel API

Generic programming classes modeled after STL style heavy use of C++ templates

Parallelism encapsulated message-passing for distributed memory machines multi-threaded shared memory (POOMA II)

Cross platform code development and scalable parallelism


Compile-timePolymorphism

ComputerScience

Physics

ApplicationApplication

AlgorithmAlgorithm

LocalLocal

ParallelParallel

GlobalGlobal

STL ExpressionTemplates

DomainDecomposition

MessagePassing

LoadBalancing

Fields MeshesParticles

InterpolatorsFFTDifferentialOperators

MC++NTTPLINAC

POOMA Framework

KEY IDEAS

Numerical programming framework

Encapsulated parallelism

High-level API’s / data support


PDT (Malony, University of Oregon)

Program Database Toolkit Program analysis

multi-language(Fortran, C,C++, Java)

commercial-grade parsers

IL to programdatabase (PDB)

API for PDBaccess / query

Tools: instrumentation, code wrapping, documentation


SILOON (Advanced Computing Lab, LANL; UO)

Scripting Interface Language for OO Numerics Toolkit and run-time support for building easy-to-use

external interfaces to existing numerical codes Scripting language to “glue” components together

KEY IDEAS

Support for application interaction control

Support for application code wrapping

Application / tool coupling

Data exchange support


Metasystems and Metacomputing

Many resources accessible on the internet computers, data, devices, people

Extend single system model to internet domain wide-area (department, campus, region, country) scalable, transparent access to resources hides network complexity (“as if on your machine”)

Extend computing model to internet domain shared persistent space of objects (data, execution) heterogeneous distributed and parallel processing meta-applications (multi-component, hierarchical)

Deal with complex environment / primitive tools


“The GRID”

New applications based on high-speed coupling of people, computers, databases, instruments, ... computer-enhanced instruments collaborative engineering browsing of remote datasets use of remote software data-intensive computing very large-scale simulation large-scale parameter studies


GRID Architectural Picture

KEY IDEAS

Metasystems infrastructure / services

Metacomputing applications programming

GRID resources


NetSolve (Dongarra, University of Tennessee)

Client-server systemto access distributedcomputational / DBHW/SW resources

Distributed computing:resources, processes,data, users

Load-balancing policy for efficiency / performance Integration with arbitrary software components

C, Fortran, Java, MatLab, Mathematica, Excel BLAS, (Sca)LAPACK, MINPACK, FFTPACK


NetSolve Usage

“Blue collar” GRID-based computing users can set things up (without “su” privileges) no deep network programming knowledge required

Scenarios clients, servers, and agents anywhere on Internet clients, servers, and agents on an Intranet clients, servers, and agent on the same machine

Focus on MATLAB users OO-style language (objects are matrices) one of most popular desktop systems for numerical

computing (> 400K users)


NetSolve – The Client

NetSolve API hides complexity of numerical software Computation is location transparent Provides access to virtual libraries:

Component GRID-based framework Central management of library resources User not concerned with most up-to-date versions Automatic tie to Netlib repository

Synchronous or asynchronous calls User-level parallelism


Agent gateway to computational services performs load balancing and resource management

Server various software installed on various hardware configurable and extendable framework to easily add software many numerical libraries being integrated supports parallel computing

NetSolve – The Agent and Server


MCell (Bartol, Salk Institute; Salpeter, Cornell)

Monte Carlo simulator of cellular microphysiology Study how neurotransmitters diffuse and activate

receptors in synapses between different cells NetSolve distributes

processing workloadand allows access tocomputational resources

Simultaneous evaluationof large number ofdifferent parametercombinations


INTERLACE (Malony, University of Oregon)

INTERoperation and Linking Architecture for Computational Engines

Goals framework for building high-performance computing

environments from existing tools reusable components in heterogeneous environment abstract connection mechanisms for control/data flow resource management for dynamic operation use standard software technologies parallel and distributed computational environments

http://www.cs.uoregon.edu/research/paracomp/proj/interlace/


INTERLACE Components Computational engines: libraries or programs

providing specific functions Computational server: program interfacing multiple

engines with middleware Wrappers: server-engine interface for data/control Middleware: server-to-server interoperation software

KEY IDEAS

High-level numeric computational services

Access to metasystem resources

Wrapping/linking of computational engines

Dynamic, adaptable, extensible

High-level metasystems programming support


ViNE (Malony, University of Oregon)

Virtual Notebook Environment High-level, shared

notebooks, data, andtools in distributed,heterogenous system

Architecture leaves: notebook

functions and data stems: notebook

communication Web-based access


ViNE Experiment Builder

List of available, named data, tools, and experiments Visual dataflow model of experiment process Wrapped tools and databases

wrappedMATLAB

“tool”


Brain Electrophysiology Lab Notebook

Dense array EEG datasets

Commercial of the shelf statistical and numerical packages

Multiple machines types

Notebook content automatically generated from experiment results


PUNCH

Purdue University Network-Computing Hubs Educational and research computing “portals”

across the Purdue “enterprise” with affiliated institutions

Resource sharing by Purdue users computers, software, laboratory equipment educational materials

Distance education allows sharing of courses and instructors

Collaborative research


PUNCH – User’s and Developer’s View

Set of network-based laboratories that provide software tools for various fields

Specialized WWW-server interfaces WWW-browsers access software and download data run tools and view results

Tool specification Virtual laboratory

developmentenvironment


PUNCH Web Page

Hubs


PUNCH Tool Display Support via VNC

MATLABcommandwindow

X Windowsdisplay

MATLABinteractive

window

MATLABgraphicswindow

KEY IDEAS

Web-based access to tools

Web-based applications development

Web-based data, results, process management


Opportunities and the Neural Informatics Center

Integrated time-dynamic neuroimaging poses methodological, computational, and informatic challenges

Apply computer science technology to create problem solving environment for brain analysis neuroscientist defines methods and processes add value to environment through its use

Neural Informatics Center (NIC) within BBMI focus on single trial analysis problem advanced EEG/ERP analysis and integrated fMRI BEM/FEM brain models (EEG, CT, MRI)


Final Thoughts

Enable high-level problem solving environments Tools to enable scientists to compose solutions from

a set of building blocks Seamless access to local and remote resources Enabling infrastructure

framework standards and interfaces implementations of reusable components

Collaboration environments Future Neural Informatics Grid

Documents

Distributed Computational Architectures for Integrated Time-Dynamic Neuroimaging