FBIRN Image Processing Stream (FIPS 2.0) Douglas N. Greve

fBIRN Image Processing Stream (FIPS 2.0)

Douglas N. Greve

FIPS Overview

FIPS: Project Management Software for fMRI• Data storage and retrieval• Data sharing• Multi-site calibration• Query-Driven, Automated analysis• Quality Assurance and Failure Detection• FSL 3.3.X (4.0.X implemented, but testing)• FreeSurfer 4.0 • Linux/MacOS• Command-line Driven (ie, no GUI)

Resources

Wiki (as of 10/15/07 still documents 1.0)http://132.239.132.249:8080/xwiki/bin/view/Project-FIPS-Public/Home

[email protected]

You should include at least the following: • FIPS Version, FSL Version, and FreeSurfer Version (if nec) • Computer type and Operating System • Command or commands • Full command line used • A statement of the problem • Terminal output and/or log files

command --helpArgument flags have two dashes (hard to see in powerpoint)

Outline• Project Management with FIPS• Importing data• Analyzing data• Viewing data• Exporting data• Integrating with FreeSurfer• Quality Assurance/Control• Managing permissions • Managing the Database• Managing Disk Space• Process Tracking• FIPS Environment Variables

Advanced Topics

• Stimulus Schedules• Preprocessing outside of FEAT• Cross-Run, within Visit Analysis FFx• Time Point Outliers File

Outline

• Project Management with FIPS• Importing data• Analyzing data• Viewing data• Exporting data

Project Management of Large Living Data Sets

• How much data do I have?• Where is it?• What is it?• How much has been analyzed? With what? When?• Are any currently being analyzed?• Did any analyses fail? Why?• Are there outliers in the data? Which ones?• Who has permission to read/write data?• Do I need more disk space?

Query-Driven

Project Management Tools

• XML file accompanies each data set (fips-process.xml)

• imported with data• maintained, modified by fips

• Hierarchy• Directory Structure• Naming convention

• Database (fips.database.dat)• Modify/update as things change• Full rebuild

• Query-driven tools• Specify match criteria

Scanning Visit (Protocol)

• Acquisition Site/Scanner • Anatomical – 4D NIFTI• Functional Paradigm 1

• “Working Memory Task”• Multiple “Runs” or “Series” (4D NIFTI)

• Stimulus schedule (text file)• Functional Paradigm 2

• Breathhold• …

• ASL (4D NIFTI)• B0 Field Map (4D NIFTI)

Properties of an fMRI Data Set

• 4D NIFTI file with pixel data • Where was data acquired? Acquisition Site• What subject is it? BIRN Id, Project Name • Which visit? Visit Id• What was collected? Paradigm, Run Number• How analyzed? Analysis Name• Analysis Level? Analysis Level

• When was data acquired? • TR=?, Slice Timing = ?, Skip=?• Stimulus schedule?• When analyzed? • Currently being analyzed? Error in analysis?• fBIRN Hierarchy

} DefineLocation

InHierarchy

fips-process.xml<FIPS> <BIRNID>001811344398</BIRNID> <acquisitionSiteID>0018</acquisitionSiteID> <project> <name>FBIRNCoilComparison</name> </project> <visit> <ID>2</ID> <visitDate>2006-09-21</visitDate> </visit> <series> <paradigm>working_memory</paradigm> <number>1</number> <analysislevel> <level>none</level> <name>NIFTI4D</name> </analysislevel> </series></FIPS>

Note: FIPS maintains all other nodes in the fips-process.xml.Allows for subsequent upload of derived data.

FIPS Hierarchy (Unix FS)$FIPS_ROOT/ Project/ [eg, FBIRNSubject2007TE] BIRNID/ AcqSiteID/ [eg, 0006] vVisitNo/ [eg, v0002] ParadigmID/ [eg, working_memory] rSeriesNo/ [eg, r006] raw/f.nii.gz, fips-process.xml, ImageWrapper.xml ScheduleFiles/ (or use global)

$FIPS_ROOT/FBIRNSubject2007TE/0018123456/0006/v0002/working_memory/r006/raw

4-digit, zero-padded

3-digit, zero-padded

$FIPS_ROOT • is a unix environment variable • all data are under $FIPS_ROOT• $FIPS_ALT_ROOTS – for disk space managment• … more later

FIPS 2.0 Hierarchy$FIPS_ROOT

FBIRNSubject2007TE FBIRNCoilComparison

0018123456 0018789012

0003 0006 MGH-Bay3

bert

v0002 v0017

working_memory

r001 r002 r0009

breath_hold

1. Project

2. BIRNID

3. Scanner (AcqSite)

4. Visit

5. Paradigm

6. Run/Series

Zero-padded

xRunFFx

xVisitFFx

raw 7. Analysis

bert

May beSame or Diff Subject

FIPS 2.0 Hierarchyr0003

raw sm5.preprocScheduleFiles Outliers gamma+dt.feat

f.nii.gzfips-process.xmlImageWrapper.xml

f.nii.gztemplate.nii.gzfips-process.xmlImageWrapper.xml

6. Run/Series

7. Analysis

FIPS 2.0 Database# ColId 1 Nth – Row number# ColId 2 BIRNID# ColId 3 RecruitSite# ColId 4 AcqSite# ColId 5 Project # ColId 6 StudyName# ColId 7 StudyId# ColId 8 VisitName# ColId 9 VisitId treated as an int# ColId 10 VisitDate# ColId 11 SeriesType# ColId 12 Paradgim# ColId 13 SeriesNumber treated as an int# ColId 14 AnalysisLevel# ColId 15 AnalysisName# ColId 16 AnalysisDate# ColId 17 AnalysisTime# ColId 18 ErrorStatus 0 or 1# ColId 19 IsRunning 0 or 1# ColId 20 XMLDir Hierarchy path

• Flat File, Simple ASCII• Not in the hierarchy• $FIPS_DB_DIR/fips.database.dat•20 columns• Modified Frequently

• IsRunning• Error Status

• fips-db –rebuild• Rebuild from scratch • Searches under $FIPS_ROOT• Slow

• fips-db –synch• Just finds and reloads fips-process.xml files that it knows about

• fips-db –prune• Removes fips-process.xml files that do not exist anymore

• fips-locate

Hierarchy

Analysis

BIRN Hierarchy

Browsing the Data

• fips-browse –all

Can add any matchcriteria to constrain items in browser.Eg: --level FirstLevel --visit 3 --run 2

Queries and Matches

fips-locate –birnid 0018123456 –project FBIRNSubject2007TE

Prints the location of all data for subject 0018123456 in project FBIRNSubject2007TE (func, B0, anat, raw, preproc, analyzed)

fips-locate –birnid 0018123456 –project FBIRNSubject2007TE --paradigm working_memory –run 3

Prints the location of only those data sets that are in the 3rd run of the working_memory paradigm (all visits, raw, analyzed, etc).

Matching Criteria: --project, --birnid, --visit, --paradigm, --run,--analysis (same as –flac), --level, --error, --no-error, --is-running, --is-not-running

Project Management Tools• XML file accompanies each data set (fips-process.xml)

• imported with data• maintained, modified by fips

• Hierarchy• Directory Structure• Naming convention

• Database (fips.database.dat)• Outside of hierarchy • Modify/update as things change• Full rebuild

• Query-driven tools• Specify match criteria

Outline


Importing• Assign each data set (ie, NIFTI file):

• fips-process.xml and ImageWrapper.xml files• SRB: Created when uploading, “In-Place “ Upload

• fips-import –srcdir directory• Directory is the root of everything you want to import

• Multiple subdirectory layers OK• fips-import:

• worms through all subdirs• finds fips-process.xml files• checks whether this data set already exists• copies data and xml into $FIPS_ROOT if not• updates FIPS DB

Importing

fips-srb-visits – downloads and imports visits from the SRB.

Example:fips-srb-visits –project FBIRNSubject2007TE

• Will download and import all visits for given project• Updates DB• Will select only those that have not been downloaded already• Automatic disk space management ($FIPS_ALT_ROOTS)• Can also select Acquisition Site, BIRNID, Recruit Site

Outline


First Level (Run) Analysis (FLA)• Motion Correction• Slice-timing Correction• B0 Correction• Spatial Smoothing• Intensity Normalization (Grand Mean)• Fit General Linear Model (GLM)

• Regression Coefficients• Residual Variance• Contrasts of Regression Coefficients (Var)• t, F, z maps of contrasts• Clusters (GRF)

• Registration to MNI152

Higher Level Analyses• Cross-Run Fixed-Effects (xRunFFx)• Cross-Visit• Cross-Subject• Cross-Scanner• … } Typically:

• Random or Mixed Fx• Require Demographics• Export• QDEC (FreeSurfer)

Analysis Configurations

Scanner

Human Upload

Hierarchy

ConfigureFirst-LevelAnalysis

FLAC Library

First-LevelAnalysis

FIPSDatabase

StimulusTiming

Import

PixelDataImageWrapper.xmlfips-process.xmlStimulus Schedules

Pixel-Data Independent

Pixel-Data Dependent

FLAC – First-Level Analysis Configuration

• All pre-processing options (MC, B0, slice timing, smoothing)

• Hemodynamic model (SPM HRF, delay, dispersion, derivative)

• Stimulus identity and timing Nuisance regressors• Contrasts• Data set independent!• Define ONCE!

FLAC – First-Level Analysis Configuration

fips-fla-config –project FBIRNSubject2007TE –paradigm working_memory –flac myflac

myflac – is the name that this will be referred to in future FIPS commands. Also determine name in hierarchy. Eg, fb3-wm-sm5. Cannot already exist.

• FLAC Library – Outside of Hierarchy $FIPS_FLAC_DIR/ProjectName/Paradigm• Launches the FSL FEAT program.

• YOU MUST KNOW HOW TO USE FSL. FIPS IS NOT A REPLACMENT FOR KNOWING HOW TO USE FSL!

Configuring the First-Level Analysis using FEAT




When you are done:• Hit “Save”• Save as “design.fsf”• Hit “Exit”


$FIPS_FLAC_DIR/FBIRNSubject2007TE/working_memory/myflac

The design.fsf will be storedIn your FLAC library as:

fips-fla-config –project FBIRNSubject2007TE –paradigm working_memory –flac myflac

Analyze the Data with FSL/FEAT

fips-fla –project FBIRNSubject2007TE –paradigm working_memory –flac myflac

Match Criteria: –project FBIRNSubject2007TE –paradigm working_memory (Analysis level = none is implied)Analyzes ALL Subjects, Visits, Runs from All SitesNote: --flac is NOT a match criteria in this case

More match constraints:• --birnid BIRNID (only analyze this subject)• --site SiteID (only analyze data collected at this site)• etc

FIPS Hierarchy$FIPS_ROOT/ Project/ [eg, FBIRNSubject2007TE] BIRNID/ AcqSiteID/ [eg, 0006] vVisitNo/ [eg, v0002] ParadigmID/ [eg, working_memory] rSeriesNo/ [eg, r006] raw/f.nii.gz, fips-process.xml, ImageWrapper.xml ScheduleFiles/ (or use global) myflac.feat/ FLAC2.feat/ xRunFFx/ (cross-run, within-visit) myflac.gfeat xVisitFFx/ (cross-visit, within-subject) ParadigmID/ myflac.gfeat

$FIPS_ROOT/FBIRNSubject2007TE/0018123456/0006/v0002/working_memory/r006/myflac.feat

During the Analysisfips-fla –project FBIRNSubject2007TE –paradigm working_memory –flac myflac

For each data set (run/series):• Has data been analyzed?

• Was there an error in the analysis? Re-runs• Do you want it to re-do the analysis? --overwrite

• Is it currently being analyzed? Skips (parallel)• Is there enough space on disk?

• Immediately marks as being analyzed (“IsRunning”)• Commences analysis• If an error occurs, marks error (“ErrorStatus”), marks as NOT being analyzed, moves onto next data set (unless –exit-on-error, or $FIPS_EXIT_ON_ERROR)• When finished successfully, marks as no error and not running

Outline


Browsing the Data

• fips-browse –all

Can add any matchcriteria to constrain items in browser.Eg: --level FirstLevel --visit 3 --run 2

Browsing the Data: FSL FEAT Report

Interactive Viewing

fips-view –project FBIRNSubject2007TE –paradigm working_memory –flac myflac --birnid 0018123456 –visit 2 –run 3 --mni152

Note: --flac is a Match Criteria

• Runs FreeSurfer tkmedit• View is in MNI152 Space• Other spaces include: --native, --anat, --hemi (surface)

Outline


Exporting

• Exporting data is the process of collecting several data sets that are distributed in the hierarchy and putting them in one place outside of the hierarchy for processing outside of FIPS• The data that are collected are both pixel data and information needed to cross-reference into a subject data base (eg, BIRNID, Site, Visit, Run, etc).• Pixel data can be exported in MNI152 or surface-based spaces.

Exportingfips-concat --outdir myexportdir –project FBIRNSubject2007TE –paradigm working_memory –flac myflac

• Creates output directory myexportdir• Exports all 1st Level Analyses from given proj/par/flac• MNI152 • All copes, varcopes, mean_funcs, masks• All data sets concatenated into a 4D NIFTI file• Creates a demo.dat file with a list of BIRNIDs, etc, for each data set. This is a text file spreadsheet.

Integrating with FreeSurfer

• Create an FSBIRNID.dat file:• ASCII file• List of BIRNID-FreeSurferID pairs• $FIPS_FLAC_DIR/Project/FSBIRNID.dat

• Why under the Project?

fips-fsreg –project FBIRNSubject2007TE …• Registers FEAT example_func to anatomical with FLIRT• Creates a FreeSurfer registration matrix

Visually Inspect:fips-fsreg –project FBIRNSubject2007TE --manual

Applying FreeSurfer Tools to FSL fMRI Analysis 44

Manual Registration

tkregister2 --help

• Visually inspect registration• Manually edit registration (6 DOF)• Cf Manual Talairach registration

Quality Control

fips-qa-sum –project FBIRNSubject2007TE –paradigm working_memory –flac myflac --qa qa.dat –mc mc.dat –reg reg.dat

• QA reports: FSNR, FWHM after analysis, and more• MC report: max relative and absolute motion over the run• Reg: registration to standard space• All are ASCII spread sheets• All have BIRN, Visit, Run, and Visit Date• Future: automatically detect outliers and create summary plots and

Managing the Database

# ColId 1 Nth – Row number# ColId 2 BIRNID# ColId 3 RecruitSite# ColId 4 AcqSite# ColId 5 Project # ColId 6 StudyName# ColId 7 StudyId# ColId 8 VisitName# ColId 9 VisitId treated as an int# ColId 10 VisitDate# ColId 11 SeriesType# ColId 12 Paradgim# ColId 13 SeriesNumber treated as an int# ColId 14 AnalysisLevel# ColId 15 AnalysisName# ColId 16 AnalysisDate# ColId 17 AnalysisTime# ColId 18 ErrorStatus 0 or 1# ColId 19 IsRunning 0 or 1# ColId 20 XMLDir Hierarchy path

• Flat File, Simple ASCII• $FIPS_DB_DIR/fips.database.dat• Modified Frequently

• IsRunning• Error Status

• fips-db –rebuild• Rebuild from scratch • Searches under $FIPS_ROOT• Slow

• fips-db –synch• Just finds and reloads fips-process.xml files that it knows about. Faster than rebuild.

• fips-db –prune• Removes fips-process.xml files that do not exist anymore

• Don’t do other processing!

Hierarchy

Analysis

BIRN Hierarchy

Managing Permissions

• UNIX Permissions:• Read, Write: User, Group, All• Ownership: User, Group• unix command ls –l (to see perms on a file)• unix command id (to see what groups you belong to)

• Change/Set Permissions:• chmod –R g+w directory• chown –R greve:birn directory

• FIPS Environment:• $FIPS_UMASK (eg, 002)• $FIPS_DEFAULT_GROUP (eg, birn)

Space Management: AltRoots$FIPS_ROOT

FBIRNSubject2007TE

0018789012

0006

v0002 v0017

Project

Subject

Scanner (AcqSite)

Visit

AltRoot1

FBIRNSubject2007TE

0018789012

0006

v0002 v0017

Project

Subject

Scanner (AcqSite)

Visit

working_memory

r0001 r0002 r0009

Symbolic Links at the Visit Level

Space Management: AltRootssetenv FIPS_ALT_ROOTS /space/1/fips-alt-root:/space/2/fips-alt-root

fips-importfips-srb-visitsfips-mv-data• Examines all AltRoots• Finds the one with the most space• Imports or Moves data therefips-alt-roots –reportfips-df

Process Tracking

fips-ps• Prints summary of currently running processes • Something like unix ps command• Command (eg, fips-fla)• User• Host• Date and Time Started• Log File associated with process

• $FIPS_PROC_DIR• $FIPS_LOG_DIR• fips-locate –is-running

EnvironmentFIPSDIR – directory under which FIPS software is locatedFIPS_ROOT – root of FIPS hierarchyFIPS_ALT_ROOTS – Alternative RootsFIPS_FLAC_DIR - local FLAC and PPC rootFIPS_DB_DIR - database locationFIPS_TEMP_DIR – place with lots of space (don’t use /tmp)FIPS_LOG_DIR – place log filesFIPS_PROC_DIR – place for process control filesFIPS_GLOBAL_SCHEDULES_DIR

FIPS_EXIT_ON_ERROR – 0 (don’t exit) or 1 (exit). Default is 0.FIPS_DEFAULT_PROJECTFIPS_DEFAULT_PARADIGMFIPS_DEFAULT_FLAC

FIPS_DEFAULT_UMASK – unix file permission maskFIPS_DEFAULT_GROUP – unix group

FIPS_OUTPUT_TYPE – default is NIFTI_GZ (can also be NIFTI)FIPS_FSL_VERSION – require this version of FSL (eg, 3.3)

Add $FIPSDIR/fipspy to PYTHONPATH

Stimulus Schedules

1 73 42 5 6 8

HAPPY

Happy.stf:

10.0 2 1

30.0 2 1

28.7 6 1

43.1 2 1

SAD

Sad.stf:

3.0 2 1

13.0 2 1

15.0 2 1

37.1 2 1

Timet=0 t=3.0 t=10.0

t=13.0

t=15.0 t=30.0 t=28.7 t=37.1t=43.1

FSL 3-ColumnCustom file:Columns:1. Onset Time2. Duration3. Weight

Example: Happy andSad Faces as defined by the experimenter.

Documents

FBIRN Image Processing Stream (FIPS 2.0) Douglas N. Greve