CHEP as

an AMS Remote Data Center

International HEP DataGrid Workshop

CHEP, KNU2002. 11. 8-9

G.N. Kim, H.B. Park, K.H. Cho, S. Ro, Y.D. Oh, D. Son (Kyungpook) J. Yang (Ewha), Jysoo Lee (KISTI)

AMS (Alpha Magnetic Spectrometer)

PHYSICS GOALS :

• High Energy Experiment in the International Space Station (ISS)

• Installation of AMS detector at ISS in 2005 and run for 3 or 4 years

To search for Antimatter (He,C) in space with a sensitivity of 103 to 104 better than current limits(<1.110-6). To search for dark matter

High statistics precision measurements of e, and p spectrum.

To study Astrophysics.High statistics precision measurements of D, 3He, 4He, B, C, 9Be, 10Be spectrum

B/C: to understand CR propagation in the Galaxy (parameters of galactic wind). 10Be/9Be: to determine CR confinement time in the Galaxy.

Tracker

TRD

TOF (S1, S2)

Ma

gn

et

He

Ve

sse

l

RICH

ECAL

TOF (S3, S4)

Veto Counter

USS

Radiators

Electronics

AMS 02 In Cargo Bay

separate e from p,p up to 300 GeV

8 Layers

e ,to 1000 GeV

He ,He ,B,C,...3

4

AMS on ISS for 3 years

AMS 02 on ISS for 3 years

AMS 02

In Cargo Bay

AMS Crew Operation Post

Data Flow from ISS to Remote AMS Centers

White Sands, NM

Data Flow from ISS to Remote AMS Centers

Internet

Distribute Locally & Conduct Science

JSC

0.9 meter dish

CommercialSatellite Service

A Dedicated Service/Circuit

Or

MSFCPOIC

TelemetryVoice

PlanningCommanding

Five Downlin

k Video Feed

s & Tele

metry Back

up

Remote User Facility

Space Station

Data Flow from ISS to Remote AMS Centers

ACOP

AMS

High RateFrame MUX

White Sand, NM facility

PayloadOperationsControlCenter

ScienceOperationsCenter

Monitoring&

sciencedata

Stored data

Real-timeDataH&S

Real-time & “Dump”

data

Real-time, “Dump”, & White Sand’s LOR playback

Exte

rnal

Com

mu

nic

atio

ns

Payload DataService system

LongTerm

ShortTerm

GSC

TelesciencecentersISS

NASAGroundInfrastructure

RemoteAMS Sites

H&SMonitoringScienceFlight ancillarydata

Real-time & “dump”

NearReal-time

File transferplaybac

k

MSFC, Al

AMS Ground Centers

Science Center

POCCPOCCPOIC@MSFC AL

AMS Remotecenter

RT data CommandingMonitoringNRT Analysis

NRT Data Processing Primary storage Archiving DistributionScience Analysis

MC productionData mirror archiving

Exte

rnal

Com

mu

nic

ati

on

s

ScienceOperationsCenter

XTermHOSC Web Server and xterm

TReK WS

commandsMonitoring, H&S dataFlight Ancillary dataAMS science data (selected)

TReK WS“voice”loop

Video distribution

Production Farm

AnalysisFacilities

PC Farm

Data Server

AnalysisFacilities

GSC D S

A eT rA v e r

GSCBuffer dataRetransmitTo SDC

AMS Remote Station

AMS RemoteStation

AMS Station

GSC

MC production

commands archive

AMS Data, NASA data,

metadata

CHEP

Internet

Internet

AMS Science Data Center (SDC)

Data processing and Science Analysis• receive the complete copy of data• science analysis• primary data storage• data archiving• data distribution to AMS Universities and Laboratories• MC production

The SDC will provide all functioning and will give the possibility to process and analyze all data.

SDC computing facilities should be enough to provide dataaccess and analysis for all members of the collaboration.

Data Processing Farm of SDC

A farm of Pentiums (AMD) based systems running Linux is proposed. Depending on the processor clock speed the farm will contain 25 to 30 nodes.

-Processing node: * Processor:Dual-CPU 1.5+GHz or single-CPU 2+GHz Pentium/AMD * Memory: 1 GB RAM * Mother Board Chip Set: Intel or AMD * Disk: EIDE 0.5-1.0 Tbyte, 3-ware Escalade Raid controller * Ethernet Adapter: 3x100 Mbit/s - 1Gbit/s * Linux OS

-Server node: * dual-CPU 1.5+GHz Pentium/AMD * 2 Gbyte RAM * 3 Tbyte of disk space with SCSI UW RAID external Tower * 3x100 Mbit/s (or 1Gbit/s) network controllers * Linux OS

batchphysicsanalysis

batchphysicsanalysis

event

summary

datarawdata

eventreconstruction

eventreconstruction

eventsimulation

eventsimulation

interactivephysicsanalysis

analysis objects(extracted by physics topic)

event filter(selection &

reconstruction)

event filter(selection &

reconstruction)processeddata

Analysis Chain: Farms

Table. AMS02 data transmitted to SDC from POIC

Table. AMS02 Data Volumes

Total AMS02 data volume is about 200 Tbyte

Stream Band Width Data Category Volume (TB/year)High Rate 3-4 Mbit/s Scientific 11 – 15 Calibration 0.01Slow Rate 16 kbit/s Housing Keeping 0.06NASA Auxillary Data 0.01

Origin Data Category Volume (TB)Beam Calibrations Calibration 0.3Preflight Tests Calibration 0.4 3 years flight Scientific 33 –45 3 years flight Calibration 0.03 3 years flight House Keeping 0.18Data Summary Files (DST) Ntuples or ROOT files 165-260 Catalogs Flat files or ROOT files or ORACLE 0.05 Event Tages Flat files or ROOT files or ORACLE 0.2 TDV files Flat files or ROOT files or ORACLE 0.5

Data Storage of SDC

Purposes: - Detector verification studies; - Calibration - Alignment - Event visualization - Data processing by the general reconstruction program - Data reprocessing

Requirement: - Tag information for all events during the whole period of data taking must be kept on direct access disks - Raw data taken during last 9 months and 30 % of all ESD should be on direct access disks 20TB - All taken and reconstructed data must be archived 200TB

ORACLE Data Base Organization

• Organization of database by machine, server, database, table - flexibility to load, locking data volume - the loading of machines A and B should be balanced• most probably both machines will be LINUX Pentiums• backup and replication of database

Machine AServer A

Database A Task A

Task B

Task C

Database B

Server B

Machine BServer A

Database A Task A

Task B

Task C

Database B

Server B

AMS Remote Center(s)• Monte-Carlo Production

• Data Storage and data access for the remote

stations

AMS Remote Stations(s) and Center(s)• Access to the SDC data storage * for the detector verification studies * for detector calibration purposes * for alignment * event visualization• Access to the SDC to get the detector and production status• Access to SDC computing facilities for Science Analysis• Science Analysis using local computing facilities of Universities and Laboratories.

producers

producers

Raw data

server

server

Oracle RDBMS

Conditions DB

Tag DB

Active Tables :

Hosts, Interfaces,

Producers, Servers

Catalogues

server

server

server

server

server

server

Nominal Tables

Hosts, InterfacesProducers, Servers…

ESD

ESD

ESDESD

ESD

server

Raw data

{I}{II}

{III}

{IV}

{V}

{VI}

{VI}

• {I} submit 1st server•{II} “cold” start•{III} read “active” tables (available hosts, number of servers, producers, jobs/host)•{IV} submit servers•{V} get “run”info (runs to be processed, ESD output path)• {VI} submit producers (LILO, LIRO,RIRO…)•Notify servers

AMS Data Production Flow

Connectivity from AMS02 on ISS to CHEP

JSC

MSFCPOIC/POCC

TelemetryVoice

PlanningCommanding

MITLANs

NISNB4207

vBNS MIT

vBNS

ChicagoNAP

CERN(SDC)

CHEP

ISP-1 ISP-2

무궁화위성

ISP-3

RC RS

Five Downlink Video Feeds &

Telemetry Backup

White Sands, NM

ISP :Internet Service ProviderNAP - Network Access PointvBNS - very high Broadband Network ServiceNISN - NASA Integrated Network ServicesPOCC - Payload Operations Control CenterPOIC- Payload Operations Integration Center

CommercialSatellite Service

International Space Station (ISS)

Tape Library(~200 TB)

Disk Storage (20TB)

DB ServerGigabit Ethernet

Linux ClustersCluster Servers

Hub

Internet Server

AMS RC

CHEP

Analysis Facility200 cpu

Data Storage(20-200 TB)Display Facility

Ewha

AMS RS

무궁화위성

Network Configuration (July-Aug, 2002)

Servers

L3 Switch

C6509

KOREN

Gigabit Switches(CHEP)

…

IBM 8271

PCs…

Servers

Gigabit Ethernet

CERN

GEANT-TEIN(EU)10~45Mbps

(2002) 10 Gbps

Gigabit E thernet/ Fast Ethernet

Linux C lustersC luster Servers

Hub

Tape Library(0.6 TB)

Disc S torage(NAS) 100GB

DB Server

Fermilab

KREONET

Gigabit Ethernet

APII(US)45Mbps(2002)

KEKAPII(Japan)

8Mbps(2002)

45Mbps1 Gbps

ResearchTraffics

KORNETBoranet

otherTraffics

(total 145 Mbps)

Gigabit Ethernet/ATM155 (Physics

Department)

Connectivity to the Outside from CHEP

APIIJAPAN(KEK)

APIIUSA

(Chicago)

TEIN(CERN)

APIIChina(IHEP)

USAStarTap, ESNET

KOREN Topology

CHEP(Regional Cent

er)Daegu

1 Gbps

Seoul

Daejeon

Daegu

■ Singapore (1)

SIngAREN through APII (2Mbps)

■ China (1, Preplanned)

CSTNET (APII) thru APII

Europe CERNTEIN

APII-TransPac

Hyunhai/Genkai

US FNAL