Keith RochfordDublin Institute for Advanced Studies
HEAnet National Networking Conference 2009
Preparing for the Data Deluge - the e-INIS National Data Store
•National collaborative project
•8 Partners
•23 funded personnel
•Objective:
•A Sustainable National e-Infrastructure
•Support advanced research
•Let researchers focus on the research
Project Overview
High Performance Computing
National Data Storage & Services
Advanced Network Services
Expert User Support
ICHEC NUI,G
TCHPC
Pilot data services Federated architecture - DIAS, TCD, UCC
Dedicated support scientists Specialist software developers
ROADM Network Grid-Computing Federated Access Management (EduGate)
Big Science = Big DataSKA 6.5 Pb/s EISCAT-3D 4x10^13 b/s
(25TB/day)ESS 1TB / day (25k
files)1000 Genomes 1.5 PBClarin 100 TB / yearElixir 40 TB ? (x2 /
yr)LSST 30 TB / night
Domestic Data Volumes
National Survey1) Data re-use & Sharing
55 %
2) Double every 12 months
44 %
3) Curation 46 %
Capacity requirements for 1
298 TB
Capacity requirements for 1 & 2
295 TB
Capacity requirements for 1 & 2 & 3
290 TBICHEC User Survey 2009
•Lacking on the Irish research landscape
•A coordinated effort of existing infrastructure partners.
•Pilot project (move towards production)
•Already benefitting new project proposals
•Ongoing capital investment for duration of project
•Hardware is the easy part
•Data management is true added value
National Data Store
•Objectives
•Increase research capacity
•Support sharing, re-use & open access
•Foster collaboration
•Encourage best practice
•High-quality data service (not raw storage)
•Capitalise on federated architecture
National Data Store
TCD Grid-Ireland Ops Centre
•Dell MD3000/1000375 TB•HP ExDS 232 TB
DIAS School of Cosmic Physics
•Nexsan SATAbeast144 TB
Sites and Equipment
UCC Centre for Unified Computing
•Dell Equalogic 45 TB•IBM DS3200 45 TB
840 TB
DIAS Equipment
•Density•Power Management
•Connectivity•Manageability
Nexsan SATAbeast + i400
•Integrated Rule-Oriented Data System
•Flexible and extensible
•Built-in metadata support
•Supports numerous front-ends:
•iRODS WebClient
•WebDav (Davis)
•Fedora Commons Repository
Middleware
•Available on equitable basis to all Irish research groups
•Application classification: C, B or A
•Evaluation criteria:
•Application on behalf of community
•National Dimension
•Must include a strategy for data management and access control
•Should include an outreach and education component
Application and Allocation
•Digital Humanities Observatory
•EC-Earth
•National Next-gen sequence repository
•National Biophotonics and Imaging Platform
•National Bioinformatics Portal
•Neonatal Brain Research UCC
•Systems Biology Ireland*
Early Adopters
QuickTime™ and a decompressor
are needed to see this picture.
Advanced Optical Networking
•Data management capability of increasing importance
•Domain experts should manage data
•e-Infrastructure should provide tools and encourage best practice.
•Future directions:
•digital archives and curation?
•Integration with identity management federation
Summary