Upload
inside-bigdatacom
View
1.738
Download
0
Embed Size (px)
Citation preview
2
• Major expansion and upgrade of Seagate HPC product portfolio • Including ClusterStor HPC HDD designed for Big Data applications
• ClusterStor L300 - Engineered Solution for Lustre • Upgraded platform expands performance leadership for Lustre storage
• ClusterStor G200 - Engineered Solution for IBM Spectrum Scale (GPFS) • ClusterStor platform extended to support all major HPC file systems
• ClusterStor A200 - Engineered Solution for Tiered Archives • Tier 2 storage expressly designed for active archive and HSM applications
Seagate Announcements at Supercomputing 2015 Overview
3
The Complete Portfolio for HPC, & Big Data
ClusterStor Manager
SAS SATA SSD
CP-2584 › 504 TB › Dual Controllers
CP-2524 › 12 TB › Dual Controllers
Spectrum Scale
CS-G200 with ISS › 8 GB/s › IBM SS 4.1
CS-A200 CS-L300 with Lustre › 9 GB/s › Lustre 2.5
Object Solutions
Ceph
Scality
OpenStack Swift
› HPC Drive › 4TB › 10K RPM
› SMR Drive › 8TB
› SAS SSD › 1.3 TB
› Tiered Archive › Up to 5 PB per rack
Cleversafe
4
Vertical Markets and User Environments served by ClusterStor
Weather Healthcare Finance Pharmacology Engineering Academic Defense Energy
Data rich analytic analysis that benefits from increased resolution, predictive models and recursive operations to enhance accuracy
Numerically complex, computationally intensive applications
Multi-disciplinary collaboration to overcome extreme problem-space, social and economic complexity
High fidelity simulation, modeling and visualization of results
High performance storage, networks and computational assets
6
Next Generation Architecture Providing the Best Investment and Productivity Value
Introducing the ClusterStor L300
The Performance and Scale of Lustre
The Power of the Clusterstor Engineered Solution
Industry’s Fastest
Converged Scale-Out Platform
Highest performance
throughput per hard disk drive
Industry’s highest quality disk drives with
lowest disk failure rate
Robust Management and Support
NEW!
Performance Increments of 12 to 16GB/sec per SSU
Intel Omni-Path or Mellanox IB EDR Network
Mixed I/O Workload Options
Improved RAS Features
Designed for Parallel Access to HPC and Big Data
Robust Management and Support
World’s Fastest File System
Extremely Scalable Volume and Namespace
7
ClusterStor L300 Key Components
› 5U84 Enclosure – CS2584 › SSU Haswell-class High Availability
Controller Pair › ESU / JBOD – Up to 3 per Single
SSU (1+1 or 1+3 option) › Mellanox Connect-X 4 Infiniband
EDR/FDR/QDR or /25/40/50/100Gb › Intel Omni Path
› Two (2) trays x (42) 3.5” drive slots
› Dual-ported 3.5” Nearline SAS & SSD HDD Support
› Pair of H/A Embedded NSD Storage Servers
› 7.2K RPM, 10K RPM HDDs, SSDs
SSU & ESU/JBOD
› Segate 2U24 Enclosure – SP2224 › Intel Haswell-class High Availability Controller Pair › Mellanox Conect-x4 Infiniband EDR/FDR/QDR or 10/25/40/50/100Gb › Intel Omni Path › 24 2.5” drive slots › Dual-ported 3.5” Nearline SAS & SSD HDD Support
L300 System and Metadata Management Unit
8
ClusterStor HPC Drive: 4TB SAS HDD
0
100
200
300
400
500
600
Random writes (4K IOPS, WCD)
Random reads (4KQ16 IOPS)
Sequential data rate (MB/s)
Performance Leader World-beating performance over other 3.5in HDDs: Speeding data ingest, extraction and access
Capacity Strong 4TB of storage for big data applications
Reliable Workhorse 2M hour MTBF and 750TB/year ratings for reliability under the toughest workloads your users throw at it
Power Efficient Seagate’s PowerBalance feature provides significant power benefits for minimal performance tradeoffs
CS HPC HDD
CS HPC HDD
NL 7.2K RPM HDD
CS HPC HDD
NL 7.2K RPM HDD
NL 7.2K RPM HDD
HPC Industry First; Best Mixed Application Workload Value
9
Seagate Nytro XD Cache Management Software - Linux Filter Driver per OSS - Monitors Writes Block Stripe Size
- Admin Definable Threshold - Eg; 32kb Block Stipe Size or less to SSD
- Small Blocks Write to SSDs - Data Flush/Writes to HDDs
- Large Blocks Write to HDDs
ClusterStor L300 SSU Small Block Accelerator
› Small Block Sizes are Written to the GridRaid HDD storage pool
› The Last Accessed Small Block Stripe is Written to the HDD OST in a Continuous “Cache Flush” Cycle
› Small Block Sizes are Written to the GridRaid ISS HDD storage pool
› The Last Accessed Small Block Stripe is Written to the HDD OST in a Continuous “Cache Flush” Cycle
ClusterStor Scalable Storage Unit
Object Storage Server #1
SSD Disk Pools are Configured as 1+1 / RAID 10 w/OSS High Availability
Small Block Stripe Sizes are Cached to a
SDD Disk Pool
Object Storage Server #2
Small Block Stripe Sizes are Cached to a
SDD Disk Pool
Large Block Stripe Sizes are written to
HDD
Large Block Stripe Sizes are written to
HDD
New ClusterStor L300 Embedded Application Sever
Empty PCI Slots With Intel Omni-Path or Mellanox EDR HBA
New L300 Object Storage Server
- Haswell-class CPU - PCI Slot for Network HBA - Intel Omni-Path or Mellanox EDR
CS L300: New Intel Network Support “Omni-Path”
11
Top of Rack Switches
Omni Path HBA/HFI Inside the Laguna Seca Controller
Omni Path Top of Rack Switches (eliminates single points of failure)
CS L300: New Mellanox Infiniband Network Support
12
CS 9000 CS L300
Mellanox IB EDR HBA Inside the Laguna Seca Controller
Mellanox IB EDR Top of Rack Switches (eliminates single points of failure)
13
Block vs. File • SFA14K 60GB/s = Raw Block • ~45 – 50 GB/s through a PFS
Footprint • Requires ~ 840 – 1200 drives • = 40 – 57 RU
File Servers • Requires ~ 8 – 12 external File Servers
ClusterStor • Benchmarks = 1 Rack of 7 SSUs = 35RU
The Register: 11/10/15 datadirect_updates_core_storage_array; DDN slide 17: http://www.ddn.com/download/Full%20Throttle%20September%202015-DDN%20and%20Intel.pdf
Missed the ClusterStor comparison… DDN has also announced a new product
CS 9000
63 GB/s
CS L300
112 GB/s
SFA14Ke
35 GB/s
DDN
Introducing ClusterStor Secure Lustre 200 (SL200)
ClusterStor CS9000 HW Platform
Red Hat Linux Secure Edition Features ClusterStor Client & MDS Label Checking
Multi-Level Security for Lustre Storage • Certified Multilevel Security (MLS) • Intelligence Community Cross Domain • Complete and explicit audit trails
• Know who is doing what on the system • Kerberos network authentication
• protects against insider threats
16
Introducing ClusterStor G200 Spectrum Scale
The Enterprise Features and Reliability of IBM’s Spectrum Scale File System
The Power of the Clusterstor Architecture
Proven at Scale across
global organizations
Easy to Use, get up and running in a few hours
Enterprise ready: data protection, management,
security and more
Industry’s Fastest Converged Scale-Out Platform
Highest performance
throughput per hard disk drive
Industry’s highest quality disk drives with
lowest disk failure rate
Robust Management and Support
Designed for the world’s most data intensive workflows
Pre-integrated, tested, tuned, ready to deploy
Accelerates workflows and removes bottlenecks
Ensures end-to-end data availability, reliability & integrity
Drastically lowers Total Cost of Ownership
Automated tiering and ILM from HDDs
to Flash to Tape
17
ClusterStor Spectrum Scale – Standard Configuration
SSU - NSD (MD) Server x 2 (SATI) Large File Sequential Performance
- 9GB/sec per 5U84 (Clustered) - - 5GB/sec per 5U84 (Scatter)
Meta Data Performance 26K File Creates per Second Average
2 Billion Files Capacity per 5U84
Metadata SSD Pool ~13K File Creates / sec
~ 1Billion files, 800 GB SSD x 2
User Data Pool ~4GB/sec
HDD x qty (40)
Metadata SSD Pool ~13K File Creates / sec
~ 1Billion files, 800 GB SSD x 2
User Data Pool ~4GB/sec
HDD x qty (40)
NSD (MD) Server #1 NSD (MD) Server #2
Management Server
18 Single Line Definition of GRIDRAID with Benefit
Fastest Rebuild for failed drive ClusterStor GRIDRAID
Parity Rebuild Disk Pool #1
Parity Rebuild Disk Pool #2
Parity Rebuild Disk Pool #3
NSD Server
Parity Rebuild Disk Pool #4
Traditional RAID
Parity Rebuild Disk Pool #1
NSD Server
GridRAID
De-clustered RAID 6: Up to 400% faster to repair Rebuild of 6TB drive – MD RAID ~ 33.3 hours, GridRAID ~ 9.5 hours Recover from a disk failure and return to full data protection faster
Repeal Amdahl’s Law: speed of a parallel system is gated by the performance of the slowest component Minimizes application impact to widely striped file performance
Minimize file system fragmentation Improved allocation and layout maximizes sequential data placement
4 to1 Reduction in NSDs Simplifies scalability challenges
ClusterStor Integrated Management CLI and GUI configuration, monitoring and management reduces Opex
Feature Benefit
19 Best of Most Widely Adopted Parallel File System Solutions
Choice in High Performance Storage
Performance Efficiency Highest performance throughput
per hard disk drive
Engineered Solution Pre-integrated, tested,
tuned, and shipped ready to deploy – built-in
rock solid productivity
Reliability Industry’s highest quality disk drives with lowest disk
failure rate, mitigates root cause to
degraded performance
Scalability Sustained linear performance,
maximize production-level uptime, keep HPC initiatives on
schedule and on budget
Robust Management and Support
Comprehensive file system management , RAS/Phone home, holistic hardware monitoring with
health alerts
Five Unique
Solution Values
1
Lustre® IBM® Spectrum Scale®
2
3 4
5
Seagate Confidential 21
ClusterStor A200 Active Archive Product Overview
Active archive object storage tier for ClusterStor Lustre platform
Combined with Clusterstor HSM to provide automatic policy-driven data migration & retrieval
Optimised for a balance of cost, performance & density
Utilizes network erasure coding to provide high levels of data availability and data durability
No single points of failure, no single points of maintenance
10Gb Ethernet node connectivity Lustre 2.5.x
ClusterStor 9000 HSM
Packaged as upgrade to ClusterStor
CS A200
ClusterStor A200
Seagate Confidential 22
ClusterStor A200 $/GB Relative usable storage costs vs Object & Tape
ClusterStor A200 Archive Tier Value Proposition › Cost effective, deep & fast
o Single Controller per SSU o 8TB SMR SATA HDDs o Increase research productivity
› Improves data availability via network erasure coding
Value Proposition vs tape- CS A200 as target for active data › Much better accessibility (<1s vs many minutes) › Better data availability › Better data durability › Better performance For active archiving, CS A200 accessibility & performance benefits more than justify the additional cost vs tape
ClusterStor CS A200 Tape
Relative Price per GB
Seagate Confidential 23
CS-A200 Economics, Performance, & Availability Getting Object Storage Right
Economics
› Single controller per SSU › Utilize most cost effective
drives - 8TB+ SMR SATA hdds
› Cost efficient erasure coding
Performance
› Parallel read/write access paths › Reduced filesystem
fragmentation increases research productivity
› Read/write throughput of 10GB/s per rack (7ssus)
Availability/ Durability
› 11 “9’s” availability (12+SSUs) › No single points of failure, no
single points of maintenance › Improve data availability &
durability via network erasure coding
24/7
Seagate Confidential 24
ClusterStor A200 Data Layouts
10Gbe Networks
Data Mover
Archive Software Library
Customer Application
Archive Software Library
8 Data Chunks 2 Parity Chunks
› 8+2 Declustered Erasure Coding • Objects are sharded up into 8+2 data+parity chunks
& written across 10 networked SSU’s • Data resiliency with up to two concurrent SSU
failures
› Highly parallelized data access • 2x 10GigE ports per SSU • Up to 10 GB/s per rack throughput
› Highly available and durable configuration • Fast data rebuilds (8TB data rebuild in less than 1 hr)
› Seamlessly scalable • Performance, capacity, & object counts
› Upcoming Capabilities › x+y erasure coding options › Multiple, flexible data layouts
CS A200 Network Erasure Coding
Seagate Confidential 25
Key HSM v1 features
Policy-driven migration from primary storage platform / tier to lower tier/target storage
› Age, last read, size etc
Policy-driven stubbing/release of payload to free up storage
Automatic retrieval/rehydration of file when stub is accessed › Rehydration of complete object
HSM v1 supports ClusterStor Lustre filesystem/ platform › Spectrum Storage Platform (GPFS) planned for next release
ClusterStor HSM v1 Functionality Overview
ClusterStor A200
Seagate Confidential 26
Clusterstor A200 Tiered Archive Features
ClusterStor A200
Storage optimized for cost per GB
Global namespace “infinite” numbers of files
HSM API - support for Lustre, GPFS, others TBA
Performance much faster than tape (up to 10GB/s per rack)
High density storage (up to 3.6PB usable per rack)
Can achieve 5 “nines” system availability
Rapid drive rebuild (<1hr for 8TB in a large system)
Integrated management and consensus based HA
X+y network erasure coding