Upload
mauli
View
32
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The Data and Storage Services Group and CASTOR. Alberto Pace. DSS group mandate. Ensure a coherent development and operation of storage services at CERN for all aspects of physics data The technologies currently used to deliver these services are CASTOR AFS TSM - PowerPoint PPT Presentation
Citation preview
Data & Storage Services
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
DSS
The Data and Storage Services Group and CASTOR
Alberto Pace
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
2
DSS group mandate
• Ensure a coherent development and operation of storage services at CERN for all aspects of physics data
• The technologies currently used to deliver these services are– CASTOR– AFS– TSM
• We have the responsibility to constantly understand and consider alternatives to these solutions– This is a very complex cost / benefit assessment– The cost and the risk of a change are high. So must be the
expected benefits
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
4
DSS organization: 3 sections
• TAB – Tape Archive and Backup– Design, operate and support the archive and backup services– This includes the tape-based software back-end for CASTOR,
tape robotics, drive and media for physics, infrastructure for backup and restore of file servers and databases
– 7 staff members• FDO – File and Disk operations
– Operate and support the storage and file system services for physics
– This includes the CASTOR and AFS services– 7 staff members
• DT – Design and Transition– Design and develop central storage services and their evolution.– This includes CASTOR and XROOT components as well as
protocols for optimal access to physics data– 6 staff members
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
5
Castor data growth
Source: Miguel Marques Coelho Dos Santos
12 million files / month
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
6
Tier-0 export
Source: Miguel Marques Coelho Dos Santos
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
7
Castor Usage (Last 2 months)
Disk Servers (Gbytes/s)
Data written to tape (Gbytes/s)
Source: Miguel Marques Coelho Dos Santos, German Cancio Melia
• 45K tape cartridges, 29K of which full• 26PB of data, 130 drives, 7 libraries
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
8
Castor Role
LHC Experiments
Tier-1s datareplication CASTOR
Disk Pools
tape servers
ASGC
BNL
FNAL
FZK
IN2P3
CNAF
NDGF
NIKHEF
PIC
RAL
TRIUMF
Analysis CPU ClustersData Reprocessing End-user analysis
ANALYSIS
AREA OFCONCERN
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
9
Areas of research & Development
LHC Experiments
Tier-1s datareplication CASTOR
tape servers
ASGC
BNL
FNAL
FZK
IN2P3
CNAF
NDGF
NIKHEF
PIC
RAL
TRIUMF
ANALYSIS
Managedon demandreplication
ScalableSecureAccountableGlobally accessibleManageableMultiple level of services-Arbitrary availability-Arbitrary reliability-Arbitrary performanceDecoupled from HW
Disk Pools
Areas of R & D
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
10
Current strategy
• Stability of service is required during the LHC operation
• Keep Castor for what it was designed for and for what it is good at– Limit developments to consolidation. Continue improving
tape reliability and efficiency for reads+writes (tape scrubbing, minimise tape recalls, developments for buffered tape marks).
• We have the responsibility to constantly understand and consider alternatives– This is a very complex cost / benefit assessment– The cost and the risk of a change are high. So must be the
expected benefits– Investigations (“Demonstrators”) are done independently
from Castor production service
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
11
Areas of developments
• In CASTOR– Consolidation in the area of Stager, Scheduler, SRM– Monitoring – Tape subsystem
• improved efficiency for reads+writes, tape scrubbing, minimise tape recalls, buffered tape marks
• “Demonstrator” Requirements– Scalable– Secure– Accountable– Globally accessible– Manageable– Multiple level of services
• Arbitrary availability, Arbitrary reliability, Arbitrary performance– Decoupled from HW
CERN IT Department
CH-1211 Genève 23
Switzerlandwww.cern.ch/
it
InternetServices
DSS
12
The Castor review agenda
• Presentations– The April 2010 incident (German)– Change and release management (Sebastien) – Operation, deployment and upgrade processes
(Miguel) – Tape operation (Vlado) – Monitoring (Dirk)
• Reviewer discussion