New Data Center at BNL– New Data Center at BNL– Status UpdateStatus Update

HEPIX – CERNHEPIX – CERN

May 6, 2008May 6, 2008

Tony Chan - BNLTony Chan - BNL

BackgroundBackground

RACF (RHIC/ATLAS Computing Facility) provides RACF (RHIC/ATLAS Computing Facility) provides support for RHIC experiments and ATLASsupport for RHIC experiments and ATLAS

Rapid growth in the past few years caused space, Rapid growth in the past few years caused space, power and cooling shortagespower and cooling shortages

Steep capacity ramp-up for ATLAS cannot be Steep capacity ramp-up for ATLAS cannot be accommodated with current facility resources accommodated with current facility resources

Phased increase in facility infrastructure over Phased increase in facility infrastructure over many yearsmany years

RACF Current CapacitiesRACF Current Capacities

Nearly 15 PB of tape storage capacityNearly 15 PB of tape storage capacity

Nearly 5 PB of disk storage capacityNearly 5 PB of disk storage capacity

Almost 13 million SI2K in computing capacityAlmost 13 million SI2K in computing capacity

Approximately 160 racks of equipment on (5000 Approximately 160 racks of equipment on (5000 ftft22) 465 m) 465 m22 of floor space of floor space

The Growth of the Linux FarmThe Growth of the Linux Farm

0

10000

20000

30000

40000

50000

60000

70000

KSpecInt2000

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

(es

t.)

2009

(es

t.)

2010

(es

t.)

2011

(es

t.)

2012

(es

t.)

Total Distributed Storage Capacity Total Distributed Storage Capacity

0

5000

10000

15000

20000

25000

30000

35000

Sto

rag

e C

apac

ity

(TB

)

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

(es

t.)

2009

(es

t.)

2010

(es

t.)

2011

(es

t.)

2012

(es

t.)

Expected Computing Capacity Expected Computing Capacity Evolution Evolution

0

10000

20000

30000

40000

50000

60000

70000FY

' 06

FY

' 07

FY

' 08

FY

' 09

FY

' 10

FY

' 11

FY

' 12

KSI

2K RHIC

USATLAS

Expected Storage Capacity Expected Storage Capacity EvolutionEvolution

0

5000

10000

15000

20000

25000

30000

35000

40000FY' 0

6

FY' 0

7

FY' 0

8

FY' 0

9

FY' 1

0

FY' 1

1

FY' 1

2

TB o

f dis

k st

orag

e

RHIC

USATLAS

Evolution of Space Usage Evolution of Space Usage

0

1000

2000

3000

4000

5000

6000

7000

8000

Us

ag

e o

f D

ata

Ce

nte

r (s

q.

ft.)

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008(est.)

2009(est.)

2010(est.)

2011(est.)

2012(est.)

Capacity of data center

Intel dual and quad-core deployed

Evolution of Power Usage Evolution of Power Usage

0

500

1000

1500

2000

2500

3000

Po

we

r U

sa

ge

(k

W)

2003 2004 2005 2006 2007 2008(est.)

2009(est.)

2010(est.)

2011(est.)

2012(est.)

Existing UPS Capacity

What are we doing about it?What are we doing about it?

Short-Term Short-Term (next 12 months)(next 12 months)• More efficient use of facility resourcesMore efficient use of facility resources• Auxiliary cooling units in existing facilityAuxiliary cooling units in existing facility• Renovation of (2000 ftRenovation of (2000 ft22) 185 m) 185 m22 of unused floor and adding 300 of unused floor and adding 300

KW of power underwayKW of power underway• Enough for ~ 40 additional racksEnough for ~ 40 additional racks

Medium-Term Medium-Term (12 to 24 months)(12 to 24 months)• New building with (6600 ftNew building with (6600 ft22) 622 m) 622 m22 and 1.5 MW of power and 1.5 MW of power• Mission-specific facility (redundant cooling, deep raised floors, Mission-specific facility (redundant cooling, deep raised floors,

etc)etc)• Enough for ~ 120 rack and 4 robotic silosEnough for ~ 120 rack and 4 robotic silos

Long-Term Long-Term (beyond 24 months)(beyond 24 months)• New BNL data center with 25000 ftNew BNL data center with 25000 ft22 (2300 m (2300 m22) after 2015) after 2015

Rack-Top Cooling UnitsRack-Top Cooling Units

Data Center Layout in 2007Data Center Layout in 2007

Data Center Layout by 2009Data Center Layout by 2009

COMPUTER DATA CENTER ADDITION

BUILDING 515

BROOKHAVEN NATIONAL LABORATORY

UPTON, NEW YORK

BROOKHAVEN NATIONAL LABORATORY

COMPUTER DATA CENTER ADDITION BUILDING 515

New Data Center Layout New Data Center Layout

Building TimelineBuilding Timeline Finalize new building design – May 2008Finalize new building design – May 2008 Award contract for new building – June 2008Award contract for new building – June 2008 Start construction – July 2008Start construction – July 2008 Renovated floor space occupancy – October 2008Renovated floor space occupancy – October 2008 New building construction complete – May 2009New building construction complete – May 2009 New building occupancy – July 2009New building occupancy – July 2009

New buildings of this size requires at least 2½ years from New buildings of this size requires at least 2½ years from first proposal to occupancy – this one done in 1½ yearsfirst proposal to occupancy – this one done in 1½ years

New building cost under US $5 million – so far still on New building cost under US $5 million – so far still on schedule and within budgetschedule and within budget

Reason for optimism – cost to renovate 2000 ftReason for optimism – cost to renovate 2000 ft22 floor space floor space is under budget by 30%!is under budget by 30%!

Suggested Floor PlanSuggested Floor Plan

SummarySummary The growth of the RACF in the past few years exposed The growth of the RACF in the past few years exposed

severe infrastructure problemssevere infrastructure problems

Learned some important lessons (maximize use of existing Learned some important lessons (maximize use of existing space, better planning, etc)space, better planning, etc)

Renovated space and additional power available in October Renovated space and additional power available in October 2008 – some breathing room2008 – some breathing room

New data center dedicated to RACF will be available in New data center dedicated to RACF will be available in summer 2009summer 2009

New facility in long-range plan after 2015New facility in long-range plan after 2015