A Design for KCAF for CDF Experiment
Kihyeon Cho
(CHEP, Kyungpook National University)
and
Jysoo Lee
(KISTI, Supercomputing Center)
The International Workshop on HEP Data Grid November 8-9, 2002
2 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
CHEP and Fermilab CDF
CDF Data/Analysis Flow
CDF Grid RoadMap (bottom –up method)□ Step 1. CAF (Central Analysis Farm)□ Step 2. DCAF (DeCentralized Analysis Farm)□ Step 3. Grid
KCAF (DeCenteralized Analysis Farm in Korea)
Future Plan
Contents
CHEP and Fermilab CDF GridCHEP and Fermilab CDF Grid
Korea CHEP
Fermilab in USA
KCAF (DCAF in Korea)
4 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICSServers
…
IBM 8271
PCs
L3 Switch (3군 )
Gigabit Ethernet
Gigabit Ethernet
C6509
KOREN
Gigabit Switch (CHEP)
…Servers
Gigabit Ethernet
FNAL
KEK
APII45Mbps
APII2 X 1Gbps
Gigabit E thernet/ F ast E thernet
Linux C lustersC luster Servers
Hub
Tape Library(0.6 TB)
Disc S torage(NAS) 100GB
DB Server
CERN
TEIN10Mbps
Network between CHEP and overseasNetwork between CHEP and overseas
5 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Overview of FermilabOverview of Fermilab
Main Injectorand Recycler
p source
Booster
CDF
D0
Fixed Target
Experiment
KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Totals 11 countries 52 institutions 525 physicists
North America Europe Asia3 Natl. Labs25 Universities
2 Universities
1 Research Lab6 Universities1 University
4 Universities
2 Research Labs
1 University
1 University
1 Research Lab4 Universities
1 University
KOREA
Center for High Energy Physics:Kyungpook National UniversitySeoul National UniversitySungKyunKwan University
CDF Collaboration
7 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Production Farm
8 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
CDF Computer Resource CAF Stage 1CDF Computer Resource CAF Stage 1
Current configuration: Stage 1
– 63 dual worker nodes.
– Roughly 160 GHz total, compared to 38 GHz for fcdfsgi2, and 8 GHz for cdfsga (Run I).
– 7 fileservers for physics (~50% full), 5 for DH, 2 for development, 2 for user scratch.
FCC (Feynman Computer Center)
9 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
CAF design considerations:
– Submit jobs from 'anywhere'
– Job output can be:sent directly to desktop
or stored on CAF for later retrieval or
– input to subsequent job
CAF DesignInteger
10 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Requirement in 2005– 200 simultaneous users will analyze 107 events of secondary dat
a set in a day.– Need ~700 TB of disk and ~5THz of CPU by end of FY’05
Current CAF (Central Analysis Farm) is not enough:– Limited resources and spaces at FCC– In case of network problems at FCC, it is dangerous.
We need DCAF (DeCentralized Analysis Farm) and Grid
DCAF (DeCentralized Analysis Farm)– Users around regional area and/or around the world– Korea, Toronto, Karlsruhe, ….– Let us call DCAF in Korea as “KCAF”
DCAF/Grid at CDF Experiment
11 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
CAF in USA
Center for HEP (CHEP) Kyungpook National U. D
aegu, Korea
KCAF (DeCentralized Analysis Farm in Korea)
APII (Asia Pacific
Information Infrastructure)DCAF in KarlsruheDCAF in Toronto
DCAF for CDF Experiment
12 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Proposed Schemes for Final Goal
DCAF in Korea (KCAF)
CAF or
13 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Step 1. To make MC production farm using KCAF– First, we start to construct 12 CPU test bed for KCAF. – After policy inside of CHEP (another test bed for EDG and iVDG
L), we will decide how many CPUs for actual MC generation farm will be used among this year’s planed 140 CPUs.
Step 2. To handle real data– To extend the KCAF to the real data handling system using SAM
(Sequential data Access via Meta-data), Gridftp, etc after settling down real data handling system.
Step 3. Final goal of CDF Grid– A gridification for KCAF related with EDG, iVDGL and CDF Grid
The Road Map for Goal
14 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
1. Software CDF Software• FBSNG for batch job
2. Kerborse • Client : login from Korea to Fermilab• Server : login from Fermilab to Korea• KDC : Client & Server Trust
3. Data Handling System• Glasgow, Italy and Yale use Data Handling System.• SAM station needs to be installed on 1 PC work node.
4. OS for Fermilab Red Hat Linux 7.3 with Kernel 2.4.185. Fundamental license problem for DB
The Road Map in Detail
15 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
1TB buffer (FCC)
CAFGUI
ICAFGUI SAM Station (KCAF)
User Desktop
Remote Desktop
KCAF Head Node
rcp,
GridFTP
bbftp
cp
KCAF Cluster
FCC (CAF)
CHEP
(KCAF)
ICAF FTP server
CDFen(Raw Data)
dCache
rcp
fcdfsam
Smaster
FSS, Stager
stager
stager
TechnicalRequest
STKen
Calibration
Data
dCache
rcp
A Design of KCAF for CDF (Fermilab) Grid
16 KIHYEON CHOCENTER FOR HIGH ENERGY PHYSICS
Around 140 CPU linux clusters will be constructed at the end of this year at CHEP.
We are constructing KCAF for CDF at CHEP, Kyungpook National University.
Contribute 1 T byte hard disk (now) + 4 T byte hard disk (December) to CDF for the network buffer between CHEP and Fermilab.
The DCAF demonstration will be shown at SC2002 on November 16, 2002.
Future Plan (This year)