A Plan for HEP Data Grid Project in Korea
Kihyeon Cho
Center for High Energy Physics (CHEP)Kyungpook National University
CDF/D0 Grid Meeting August 5, 2002
2 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
HEP Data Grid Project in Korea
Current Status and Plan Inside of CHEP CHEP and EU Data Grid CHEP and Fermilab
Conclusions
Contents
HEP Data Grid Project in HEP Data Grid Project in KoreaKorea
4 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
In Korea, HEP Data Grid Project has started on March 2002 funded by Ministry of Information and Communication.
Five year project (2002-2006)
30 High Energy Physicists (Ph.D level) from 5 Experiments (CDF, AMS, CMS, K2K, PHENIX) are involved in this project.
This project includes to make Regional Data Center (Tier-1) for CMS, AMS, CDF(?) in Asia. The Regional Data Center will be located at CHEP (Center for High Energy Physics) at Kyungpook National University.
HEP Data Grid in Korea
HEP Data Grid in Korea
5 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Center for High Energy Physics ( C H E P )
Kyungpook National University Daegu, Korea
1. CMSTier-1
Regional Center(CERN)
4. BelleExp
(Japan)
5. K2KExp
(Japan)
3. CDFGrid
(USA)
2. AMSRegional Center(CERN)
6. PHENIX
Grid(USA)
Data Grid
Cluster
Korea U
Sungkyunkwan U
Chonnam N U
Gyeongsang N U
Konkuk U
Ewha W U
Dongshin U
KBSI
…Other users
CHEPKyungpook N U
Seoul N U
Yonsei U
• Established in July 1, 2000.
• A national center designated by the Korean Ministry of Science and Technology and supported by the Korean Science and Engineering Foundation (KOSEF)
• 47 HEP physicists with doctoral degrees and 120 graduate students from 12 institutes inside Korea participate in the center.
Participation of Institutions in the HEP Data Grid Project
6 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Europe CERN (CMS)
US FNAL (CDF)
US BNL (PHENIX)
Space Station (AMS)
Japan KEK (K2K)
Korea CHEP
Experiments for HEP Data Grid Project in Korea
7 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Networking Multi-leveled hierarchy (both for data and for computation) Tier-1 : CHEP Tier-2 : SNU, Sungkyunkwan University (SKKU)
Network Services Videoconferencing OO Database access
Data Storage Capability in Data Center Storage : 1100 TB Raid Type Disk Tape Drive : 60 Tape Drives of IBM 6590 TSM Server/HPSS : 3Pbyte
Computing powers (1000 CPU Clusters) Tier-1 Institute (CHEP) : 600 CPU linux clusters Tier-2 Institute (SNU, SKKU) : each 200 CPU linux clusters
Final Goal
Project (2002-2006)
8 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Processing Fabric1000 CPUs
Commodity PCs
3 PBytes storage
Gbit Networks
KOREN
Supercomputers
National network
LANs
(1) For Data Grid abroad Network between CERN(Tier-0) and
CHEP(Tier-1), Korea TEIN ( Gbps)
Network between Fermilab, USA and CHEP(Tier-1), Korea APII (1 Gbps)
Network between KEK, Japan and CHEP(Tier-1), Korea APII and Hyunhae (1 Gbps)
(2) For Domestic Data Grid Network between CHEP(Tier-1) and Tier-
2 155 Mbps ~ 1 Gbps
Network between CHEP(Tier-1) and Other 45 ~ 155 Mbps
The Goal of HEP Data Grid Network
Project (2002-2006)
9 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
KOREN/TEIN and APII Test beds
APIIJAPAN(KEK)
APIIUSA
(Chicago)TEIN
(CERN)
APIIChina(IHEP)
USAStarTap, ESNET
Fermilab, BNL etc.
KOREN Topology
CHEPDaegu
1 Gbps
Seoul
Daejeon
Daegu
Current Status at CHEP Current Status at CHEP
Korea CHEP
11 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Current Resources
CPU : 40 CPU (1.7 GHz) Clusters
HDD □about 3 Tbyte (80 GB IDE X 40)
□100Gbyte Raid + 600 Gbyte Tape library
Network CHEP ----- KNU ---- KOREN ---- Star tap ---- Fermilab
1Gbps 1 Gbps 45Mbps ?
The actual network performance between CHEP and FNAL is 3~5 Mbps (30~50 Gbyte/day).
Hardware
12 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICSServers
…
IBM 8271
PCs
L3 Switch (3군 )
Gigabit Ethernet
Gigabit Ethernet
C6509
KOREN
Gigabit Switch
…Servers
Gigabit Ethernet
FNAL
CERN
APII45Mbps
TEIN10Mbps
Gigabit E thernet/ F ast E thernet
Linux C lustersC luster Servers
Hub
Tape Library(0.6 TB)
Disc S torage(NAS) 100GB
DB Server
Current Network between CHEP and FNAL (Actual 3~5Mbps)
Star Tap
40
KyungpookNan’l Univ.
CHEP
13 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Object Item QuantityFile Server Pentium III 866MHz
(dual)
2
Login and Compile Server Pentium IV 1.7 GHz
AMD 1.47 GHz
1
1
Main CPU
Pentium III 1 GHz (dual)
Pentium IV 1.7 GHz
AMD 1.47 GHz
AMD 1 GHz
1
16
14
5UPS 10 KW (30 min) 1
NAS
100 GB HDD (RAID)+
600 GB Tape library 1
Network Switch 24x100Mbps +
2 x 1 Gbps
1
PC Clusters
Current Status at CHEP
14 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Installed Globus 2.0 on 12 PCs.
(10 @CHEP, 1@SNU, 1@Fermilab)
Constructed Private CA (Certificate Authority).
Installed MDS (metacomputing directory service).
Installed GridFTP, Replica Catalog, Replica Management.
Test the Grid test-bed
□ between CHEP(Tier-1) and SNU(Tier-2)
□ between CHEP(Tier-1) and Fermilab
Current Status at CHEP
Data Grid Software and Middleware
CHEP and EDG (EU Data Grid)CHEP and EDG (EU Data Grid)
Europe CERN
Korea CHEP
TEIN (Tran Eurasia
Information Network )
EDG (EU Data Grid)
16 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Actual network topology (map) of the TEIN Pilot Project
The bandwidth of the TEIN
□ Current :10 Mbps with 20M PCR
□ At the end of this year : 45 Mbps
Proportion of traffic exchanged with other Asian and European countries
□ This project will also include connectivity with KEK in Japan.
Possible future extensions of the TEIN Pilot Project
□ The ultimate goal of the connectivity between CHEP and CERN is more than Gbps (Lambda).
CERN 10 Mbps 1 GbpsCHEP
GEANT
TEIN
KOREN
TEIN (Trans Eurasia Information Network)
17 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Three researchers have visited CERN for EDG test bed between July 14~24, 2002.
PKI was created for Korean test bed.□ C=KR,O=KCHEPDG,OU=SNU : Seoul National University□ C=KR,O=KCHEPDG,OU=KNU : Kyungpook National University
Representatives for institute:□ Bockjoo Kim : Seoul National University□ Kihyeon Cho : Kyungpook National University
We got the certificates from France.
We have learned basic components of EDG.□ UI : User Interface□ CE : Computing Element□ SE : Storage Element□ WN : Worker Node□ RB : Resource Broker, Gate Keeper□ LCFG : Server for installing EDG software for each component
The Status of EDG test bed in Korea
18 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
User submits job to UI and requests data to SE from UI
Check resources for given job
Submit job toWorker node and send output to user
EDG Jobs Life Cycle
19 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Future plan for EDG test bed
CHEP will join VO (virtual Organization) as CMS.
To install basic components at CHEP.
To register the server for CA inside of Korea.
To maintain the test bed by running CMS simulation code.
We will have better network between Europe and Korea. □ At the end of this year the bandwidth of TEIN will be 45 Mbps.
CHEP and Fermilab GridCHEP and Fermilab Grid
Korea CHEP
Fermilab in USA
KCAF (DCAF in Korea)
21 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
CHEP has a plan to make a cron of CAF.
CAF (Central Analysis Farm)– Limited resources and spaces at FCC– At Run IIb, data size is 6 times more than now.– In case of network problems at FCC, it is dangerous.
DCAF– Users around regional area and/or around the world– Korea, Toronto, Karstrule, ….
Let us call DCAF in Korea as “KCAF”.
Why DCAF/Grid in the future?
22 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
DCAF DCAF in Korea
(KCAF)….
Central Analysis Farm (CAF)
users
Resource Broker
Proposed schemes
23 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
CAF in USA
Center for HEP (CHEP) atKyungpook Nat’l
UniversityDaegu, Korea
DCAF in Korea (KCAF)
APII (Asia Pacific
Information Infrastructure)DCAF in Karstrule
DCAF in Toronto
Where is KCAF?
24 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Step 1. To make MC production farm using KCAF– First, we start to construct 20/40 CPU test bed for KCAF. – After policy inside of CHEP (another test bed for EDG) and
between CDF, we will decide how many CPUs for actual MC generation farm will be used among this year’s planed 100 CPUs.
Step 2. To handle real data– To extend the KCAF to the real data handling system using
SAM, Gridftp, etc after settling down real data handling system.
Step 3. Final goal of CDF Grid– A gridification for KCAF related with EDG and CDF Grid
The Step for the Goal of KCAF
25 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Current CAF at FCC
26 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Cron of CAF (Central Analysis Farm)
Users– Korean Group – Other Asian Group– Other around the world
Technical Request– KDC for Kerborse– Data Handling System– To connect between 1 Tbyte buffer and fcdfsam at FCC
A Design of KCAF
27 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
1TB buffer (FCC)
CAFGUI
ICAFGUI SAM Station (KCAF)
User Desktop
Remote Desktop
KCAF Head Node
rcp,
GridFTP
bbftp
cp
KCAF Cluster
FCC (CAF)
CHEP
(KCAF)
ICAF FTP server
CDFen(Raw Data)
dCache
rcp
fcdfsam
Smaster
FSS, Stager
stager
stager
TechnicalRequest
STKen
Calibration
Data
dCache
rcp
28 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
100 PC clusters will be constructed at the end of this year.
6 Tbyte HPSS system is supposed to come.
Contribute 1 T byte hard disk (now) + 4 T byte hard disk (December) to CDF for the network buffer between CHEP and FCC.
On early November, 2002, International HEP Data Grid Work Shop will be held at CHEP, Kyungpook National University in Korea.
Future Plan (This year)
29 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Conclusions
Grid is one of world trend at High Energy Physics area. Someday we should use it.
Our government is very interested in Grid project.
HEP Data Grid project in Korea is very active for both EU and USA side.
Any comments?