CHAP Meeting 4 October 2007 CISL Update Operations and Services CISL HPC Advisory Panel Meeting 4...
26
CHAP Meeting 4 October 2007 CISL Update Operations and Services CISL HPC Advisory Panel Meeting 4 October 2007 Tom Bettge Tom Bettge Director of Operations and Services Director of Operations and Services Computational and Information Systems Laboratory Computational and Information Systems Laboratory
CHAP Meeting 4 October 2007 CISL Update Operations and Services CISL HPC Advisory Panel Meeting 4 October 2007 Tom Bettge Director of Operations and Services
CHAP Meeting 4 October 2007 CISL Update Operations and Services
CISL HPC Advisory Panel Meeting 4 October 2007 Tom Bettge Director
of Operations and Services Computational and Information Systems
Laboratory
Slide 2
CHAP Meeting 4 October 2007..during the past six months..
blueice bluevista
Slide 3
CHAP Meeting 4 October 2007 Resource Utilization
Slide 4
CHAP Meeting 4 October 2007 Challenges for CISL and CHAP (a
good thing!!) l Witness: 1.7M GAUs available, 1.0M GAUs requested l
Witness: NCAR Resources Increasing (another bump in Sept 2008)
University use could fall behind NCAR University use could fall
behind NCAR l Tapping Latent GAUs? (CISLs challenge) Special queue
access by request for large university users Consider requiring
projects with large allocations to use in a paced manner, monthly,
quarterly, etc. l Filling the surplus gap for university PIs
Awareness (CISL and NSF) Mirror the NCC n Only one project selected
for BTS in Nov, 5 were submitted Leave well enough alone Build it
and they will come. from April CHAP Meeting
Slide 5
CHAP Meeting 4 October 2007 Large Community Requests (10 6
GAUs)
Slide 6
CHAP Meeting 4 October 2007 Community (non-CSL) GAUs Used
Slide 7
CHAP Meeting 4 October 2007 Breakthrough Science (BTS)
Initiative Spring 2007
Slide 8
CHAP Meeting 4 October 2007 Breakthrough Science Projects Usage
PIAffiliation NSF Prog Title GAUs Used Eric Chassignet Florida
State OCE Building a New Version of CCSM3 with HYCOM as the Ocean
Model 452,000 Dave Randall CSUATM Global Climate Modeling with
Super Parameterization 325,000 Brian Savage University of Rhode
Island EAR 3-D Tomography of the Crust and Upper Mantle Beneath the
Gulf Extensional Province and Baja California 1000 Bill Smyth
Oregon State OCE Instability and Turbulence in a Sheared,
Diffusively Unstable Fluid 279,000 Ron Cohen Carnegie Inst. of
Washington EAR Quantum Monte Carlo on Geophysical Materials 521,000
David Yuen University of Minnesota EAR (1) Poro-elastic Wave
Propagation in Oil Reservoirs (2) 3-D Tsunami Wave Simulations
39,000 Annick Pouquet NCARNCAR Small Scale Structures in MHD
Turbulence: Toward a Better Understanding of the Magnetosphere
233,000 Rich Rotunno NCARNCAR High Resolution Hurricane Simulations
726,000
Slide 9
CHAP Meeting 4 October 2007 University Capability Computing
NCAR Capability Computing PIAffiliation NSF Prog Title GAUs Used
David Straus COLAATM Detection and Attribution of Climate Change:
The Interactive Ensemble 424,000 Chris Davis NCARNCAR Advanced
Hurricane WRF Model Trials 227,000
Slide 10
CHAP Meeting 4 October 2007 Program Distribution
Slide 11
CHAP Meeting 4 October 2007 BTS Outcomes l CISL was able to
provide large amounts of computing time through the use of special
queues and the fair share scheduler l Three university projects
have submitted large requests to CHAP because of BTS or UCC
simulations Straus, 423K Cohen, 250K Smyth, 70K l Very likely to
provide early Capability Computing Opportunity in 2008 with
availability of bluefire (some caveats here) community eligibility
TBD start solicitation earlier require more detailed proposals
users provide NCAR with more precise benchmarks (NCAR will help in
advance) projects which need resources and are ready-to-go
Slide 12
CHAP Meeting 4 October 2007 ICESS Computing Status
Slide 13
CHAP Meeting 4 October 2007 ICESS System Phase II l Jan 2007 to
~Jun 2008: 1.9 GHz POWER5+ p575 SMP nodes dual-link Federation
Switch 4 TB memory, 150 TBytes disk 1600 batch processors 16
processors/node 4 bluesky equivalents, 12 TFLOPs peak l ~Jun 2008
to ~Jun 2011: 4.7 GHz POWER6 p575+ SMP nodes quad-link Federation
Switch 9.2 TB memory, 150 TBytes disk ~3200 batch processors 32
processors/node 15.5 bluesky equivalents, 57.6 TFLOPs peak Phase I
Phase II blueice bluefire
Slide 14
CHAP Meeting 4 October 2007 Sustained Capacity with ICESS
Slide 15
CHAP Meeting 4 October 2007 ICESS Subcontract Performance
Commitments l blueice is 4 bluesky-equivalents l bluefire is to be
15.5 bluesky-equivalents The following table shows committed
average code speedups (each code is run on the indicated range of
processor counts). Average blueice speedup over bluesky Average
estd bluefire speedup over bluesky Estd bluefire speedup over
blueice CAM (64,256) 3.485.571.60 HD3D (8,16,32,64,128)
4.417.501.70 POP (8,16,24,32,48,64,128) 3.576.571.84 WRF
(1,2,4,8,16,32,64,128,256) 3.525.531.57 Weighted Average*
3.605.941.65 *Weighted Average: 45% CAM, 10% HD3D, 20% POP, 25%
WRF
Slide 16
CHAP Meeting 4 October 2007 Facility Constraints
Slide 17
CHAP Meeting 4 October 2007 Liquid Cooling !!!! l Power6 uses
even more direct methods of heat transfer and reduces reaction time
for mechanical systems
Slide 18
CHAP Meeting 4 October 2007 bluefire Schedule l April
2008delivery Power up enough bluefire to match blueice production
workload l May 2008ATP completed on partial bluefire l June
2008blueice decommissioned l July 2008ATP completed on entire
bluefire l Aug-Nov 2008Capability Computing Opportunity UCC and NCC
details TBD l October 2008bluevista decommissioned l Dec 2008Full
availability of bluefire e.g., CSL not allocated fully until
December 1
Slide 19
CHAP Meeting 4 October 2007 MSS Plans
Slide 20
CHAP Meeting 4 October 2007 MSS Growth
Slide 21
CHAP Meeting 4 October 2007 MSS Observations l Current growth
rate is 130-150 TB/month l Capacity of current archive (6 PB) will
be exhausted June-Sept 2008 l Growth rate will be higher with
bluefire availability l Estimate capacity of 20 PB needed prior to
move to NSC l Near Term Plan Engage vendors with estimated
requirements Request proposals for 4 year contract to partner with
NCAR for MSS expansion Award, install, and begin production by Sept
2008 Phased decommission of current silos
Slide 22
CHAP Meeting 4 October 2007 CAS2K7 September 9-13, 2007 l
Keynote Speakers emphasis on petascale computing and
interdisciplinary applications and data access l Trends
Power/performance impacts on facilities huge issues Grids becoming
increasingly useful Important problems are interdisciplinary
Increasing parallellism, not faster clocks Presentations:
www.cisl.ucar.edu/dir/CAS2K7
Slide 23
CHAP Meeting 4 October 2007 Questions and Discussion
Slide 24
CHAP Meeting 4 October 2007 Main HPC Systems at NCAR l IBM
Power5 p575 l 624 1.9 GHz P5pes l 4.7 Tflop Peak l 210 kW l IBM
Power6 p575+ l 3200 4.7 GHz P6pes l 57.6 Tflop Peak l 700 kW l IBM
Power5+ p575 l 1600 1.9 GHz P5pes l 12.0 Tflop Peak l 290 kW IBM
Power5 IBM Power6 IBM Power5+ bluevistablueicebluefire
Slide 25
CHAP Meeting 4 October 2007 2007-2008 HPC Resource Split NCAR
Capability Computing
Slide 26
CHAP Meeting 4 October 2007 Breakthrough Science (BTS)
Initiative Spring 2007 l First four months of blueice production
use l 75% of blueice devoted to BTS NSF selected five projects CHAP
selected one projects NCAR selected two projects l Consultant
assigned to each project l Special queues: bts_prm, bts_reg,
bts_sby, bts_ded provided prototype access for capability
computing
