View
217
Download
0
Category
Preview:
DESCRIPTION
Dept. Psychiatry Dept. Computing Science Computational Psychiatry Group
Citation preview
BIRS 2016:Opening the analysis black box:
Improving robustness and interpretation
Matthew Brown, PhDUniversity of Alberta, Canada
Overview
1. About us2. Preprocessing quality assurance3. Interpretation of group vs. individual
differences4. Trial type fMRI signatures
Dept. PsychiatryDept. Computing Science
ComputationalPsychiatryGroup
• Diagnosis– What disease?
• Prognosis– Predict patient response to treatment options
Clinical decision-making
What are we detecting?• 10 psychosis patients, 10 controls, fMRI• Highly diagnostic Fourier power distribution
from voxels IN THE EYES• Eye movement disturbances in psychosis
ADHD-200 and ABIDE datasets• n=1000 approx.• ADHD patients or autism patients• Structural MRI, resting state fMRI• Simple diagnosis
– Classify patients vs. controls– Accuracy 50-70% in various papers
• Some papers reported higher 75%+ accuracyBUT cherry-picking sites?
ADHD-200 Global Competition• Best-performing algorithm, but did not win• Used only non-imaging features:
– Age, gender, handedness, IQ, site of scan– 3-class classification (ADHD-c, ADHD-i, control)– 63% hold-out accuracy (vs. 54% chance)
Using non-imaging features
Brown et al. 2012
Chance accuracyValid
ation
Accu
racy
(%)
Histogram of oriented gradient (HOG) featuresImage from Ghiassianet al. under review.
Also see Dalal and Triggs 2005. IEEE Computer Society Conference on. vol. 1. IEEE, p.886–893.
ADHD-200 and ABIDE datasets• Ghiassian et al. under review• State of the art (as of 1.5 years ago)• 2-class classification (patients vs. controls)
ADHD-200 ABIDEChance 55% 51%Non-imaging 69% 60%Non-imaging + Structural MRI
70% 64%
Non-imaging + Functional MRI
64% 65%
Overview
1. About us2. Preprocessing quality assurance3. Interpretation of group vs. individual
differences4. Trial type fMRI signatures
Registration failure Subject 1 Subject 15
Fixed ->
Standard preprocessing methods failed for 1 of 21 subjects.
Inter-site variability
Sen et al. in preparationPCA Component 1
PCA
Com
pone
nt 2
ADHD-200 Subjects Projected onto PCA component space
Each colour is a different scanning site.
Even with standard normalization procedures, inter-site structure remains in the data.
Overview
1. About us2. Preprocessing quality assurance3. Interpretation of group vs. individual
differences4. Trial type fMRI signatures
Clinical research
Huntington’sImage from Wikipedia
Healthy
One goal: Associate disease with biological features
ADHD-200 resting state fMRI functional connectivity analysis
ICA
Brown et al. 2012
ADHD patients vs. controls
“Default mode” network Patients vs. controls
Brown et al. 2012
“Desired” simple interpretation: “Patients are different from controls. This difference tells us something about the disease.”
Group vs. individual differences
PatientsControls
Statistically significant group differences, but substantial overlap between individual patients and controls.
Brown et al. 2012
Interpretation
• Simple interpretation “patients are different from controls”
• Overlap precludes simple interpretation• Yet many papers provide precisely and only
the simple interpretation
PatientsControls
Brown et al. 2012
Overview
1. About us2. Preprocessing quality assurance3. Interpretation of group vs. individual
differences4. Trial type fMRI signatures
Black box analysis
AnalysisSoftware
General linear model regression
Model voxel i’s timecourse
Model matrix for trial type k
Two different models for hemodynamic response function
SPM canonicalmodel
Finite impulseresponse model
Check deconvolved timecourses
Basically agree on shape (but not statistical differences in this case)
SPM canonical model
Finite impulse responsemodel, same region
Check deconvolved timecoursesSPM canonical model
Finite impulse responsemodel, same region
Noise in deconvolvedtimecourses
Another exampleSPM canonical model
Finite impulse responsemodel, same region
Noise in deconvolvedtimecourses
GLM analysis
• Check deconvolved timecourses• What is the model fitting
– Noise vs. signal• Model selection
– regularization
Summary
Quality check everythingVisualization
Intermediate steps and final resultsParticularly important for non-technicalend-users
Acknowledgements
People: Azad, Benoit, Dursun, Ghiassian, Greenshaw, Greiner, Juhas, Purdon, Ramasubbu, Rish, Sen, Silverstone
Funding: AICML, AIHS, CIHR, Norlien Foundation, AHS, AMHB, UAlberta
Questions?
Invitation
Continue informing other researchers about analysis pitfalls and caveats.
Questions?
Title
Recommended