Upload
elmo-barnett
View
31
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Cluster distributed dynamic storage. BACKGROUND Manchester University 1000 nodes new cluster (2x1000 processors) 2x250 GB disks equivalent to ~400 TB of available disk space excluding the space reserved for OS and scratch directories. There is no tape storage behind. - PowerPoint PPT Presentation
Citation preview
A. Forti A. McNab
Cluster distributed dynamic storage
BACKGROUND
• Manchester University 1000 nodes new cluster (2x1000 processors)
• 2x250 GB disks equivalent to ~400 TB of available disk space excluding the space reserved for OS and scratch directories.
• There is no tape storage behind.• Cluster internal connectivity is 1 Gb/s and
is connected to the external world by the 2.5 Gb/s university production network. It will be replaced by a 10Gb/s dedicated link.
• High rate data transfers from Tier1 is possible possible. 350 Mb/s achieved on the university production network.
• The aim is to use the space as a cache.
CONFIGURATION
• dcache has been chosen as software• There will be 2 types of nodes: external and
internal– External nodes will be front end nodes
with an active gridftp door for transfer from the outside world. They’ll have a dedicated cpu to handle the transfers.
– Internal will accessible only from other nodes
• On each node:– 2 dcache partitions 1 on each disk– 2 pools, 1 for each partition– /pnfs mounted– Gsidcapdoor active
• No raid arrays will be created it spoils the independence of the discs. In this way
– Disk 2 can be assigned to a particular VO– Disk 1 can be used for resilience. – Disk 1 can be wiped out during installation
if needed– Disk 2 can be left untouched.
• Data will have 2 copies on the system using resilience.
Single external node internals
gridftpDoor
Externalworld
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Headnode
PnfsSingle external node internals
gridftpDoor
Externalworld
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Headnode
Pnfs
Internal nodes access.
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Headnode
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
applicationapplication
application
Internal nodes access.
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Headnode
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
Disk 2250GB
Disk 1150GB
pool
OS/Home100GB
gsidcapDoor
Pnfs
pool
applicationapplication
application
APPLICATIONS
• Applications will access the data through /pnfs or gsidap doors
– Gsidcap doors might be turned off in the future
OTHER SOLUTIONS
• Solution based on http technology is being developed in Manchester:
• Uses HTTP or HTTPS for data transfer– Access control via XML policies and
X.509 or VOMS certificates– File location using Hypertext Cache
Protocol and multicast queries– Avoids need for central database, by
enforcing atomic file operations and using status of files on disk to respond to queries
• Experiments using root have asked to support Xrootd as a possible alternative solution
– Plain root APIs to access dcache are also being evaluated. This turns out useful in the final stages of analysis when the experiment framework becomes irrelevant.