Cosmic Microwave Background Data Analysis At NERSC

Julian Borrillwith

Christopher CantalupoTheodore Kisner

What Is The CMB ?

A snapshot of the Universe when it first became neutral 400,000 years after the Big Bang.

Cosmic - filling all of space.

Microwave - redshifted by the expansion of the Universe from 3000K to 3K.

Background - primordial photons coming from “behind” all astrophysical sources.

Why Do We Care About The CMB ?

The CMB is a unique probe of the very early Universe.Its tiny (1:105-8) fluctuations carry information about - the fundamental parameters of cosmology - ultra-high energy physics beyond the Standard

Model

What Does The CMB Look Like ?

CMB Work At NERSC

• Started in 1997:– 2 separate allocations for Maxima & Boomerang– together 5 users & 30,000 CPU-hours

• Developed into premier world center for CMB analysis:– single allocation shared by O(10) experiments– O(100) users & O(1,000,000) MPP-hrs/year

• Now includes "Big Science" satellite mission– split into two allocations

• mp107 - 13 sub-orbital experiments 40 users & 500,000 MPP-hrs

• planck: Planck satellite 60 users & 2,000,000 MPP-hrs*

The Planck Satellite

•The primary driver for current NERSC CMB work.

•A joint ESA/NASA mission due to launch in the fall of 2008.

•An 18+ month all-sky survey at 9 microwave frequencies from 30 to 857 GHz.

•O(1012) observations, O(108) sky pixels,O(104) spectral multipoles.

Data Management

• Dominated by time-ordered data– O(1-10) TB, O(10,000- 100,000) files

• Each data set must be analyzed as a whole.• Each data analysis needs O(100x) storage.• Each data set may have its own format/distribution.• Each data set must be selectively shared.

• Requires– Pre-fetching & active disk quota management– Efficient & abstracted run-time reading– Project account

Task Management

• Any member of a team must be able to– Access all the data– Access all the general & project-specific codes– Generate and execute standard analyses– Share the results with the team

• Some members of a team must be able to– Control the overall team work-load/-distribution– Manage software versioning and access

• Requires– Project account with individual user certification– Limited capability for most; full capability for

some.– Synchronized data-for-task management

A Framework for CMB Analysis At NERSC

Data Management

Data staging

Run-time IO

Memory

Task Management

User

Project

Critical Components & Issues

• NERSC Global Filesystem– access from Franklin

• Storage Resource Manager– optimal transfer protocols

• Project quotas– separation from UNIX groups

• Project accounts– appropriate queue limits

• User accounts– maintain (unique) accessibility

• Modules– work just fine

Conclusions

• NERSC has developed into the world's leading center for HPC for CMB data analysis– Recognized as such by the recent NASA/NSF/DOE

Weiss report on the future of CMB research.

• This reflects the NERSC resources'– capacity and capability,– accessibility,– long-range development plan.

• Long may it continue !