Upload
lamkiet
View
229
Download
3
Embed Size (px)
Citation preview
Redefining Storage Economics
Ethernet SANComparison of FC (and FCoE), iSCSI, and AoE
Robert Przykucki, Director of Product Management
November 2010
Redefining Storage Economics
Unique Market Window: Ethernet SAN
40%IT Budget
Unsustainable storage costsin uncertain economy
Legacy SAN is a poor fit withCloud and Virtualization
Storage growth is unstoppableEthernet always wins
Redefining Storage Economics
Storage Challenges
Legacy SAN 1:1 Mainframe to storage
Today’s SAN Many:1 Servers to storageMany:1 Virtual to physical servers
Redefining Storage Economics
Ethernet SAN Bare metal performance No controller bottleneck Eliminates complex topologies No more complex multipathing Ethernet simplicity Lightweight protocol wins!
Scale-out Ethernet SAN
Redefining Storage Economics
Evolution of SAN
Protocol Comparison
Management Comparison
Q&A
Agenda
Redefining Storage Economics
Complexity
Price-Performance
FICON/ESCON
Fibre Channel
FCoE
Ethernet
iSCSI
The drive to Ethernet
Redefining Storage Economics
Protocol Stack ComparisonH
eavy
Wei
ght
Pro
toco
l
Block
Session
Transport
Network
Data Link
FibreChannel iSCSI
TCP
FCoE
FCP iSCSI FCP
AoE
SCSI SCSI SCSI ATA
IP FCoE AoE
FC Ethernet DCB Ethernet
Redefining Storage Economics
Protocol Stack Comparison
FibreChannel iSCSI FCoE AoE
Block SCSI SCSI SCSI ATA
Hea
vy W
eigh
tP
roto
col
Session
Transport
Network
Data Link
FCP
FC
iSCSI
TCP
IP
Ethernet
FCP
FCoE
DCB
AoE
Ethernet
MPIO Multi-Path DriverRequired for FC, iSCSI and FCoE
Redefining Storage Economics
iSCSI data path
iSCSI: Connection-based, serial delivery
iSCSI 64K IO Transfer CRCSegment Segment Segment Segment Segment Segment Segment Segment Segment
64K IO Transfer
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #1
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #2
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #3
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #4
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #5
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #6
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #7
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #8
Initiator(IO is bound to
iSCSI connections)
Port 0
Port 1
Port 2
Port 3
Port 0
Port 1
Port 2
Port 3
Array
Path0
Path1
Path2
Path3
Verify CRC, Remove Ethernet Layer
Remove IP Header
TCP: Re-assemble Byte Segments
iSCSI: Verify IO Integrity
64K IO: Send to disk
TCP Data (IO Segment) CS
IP Data
Ethernet Data (Jumbo) CRC
Frame #9
Frame #1 Frame #2 Frame #3 Frame #4 Frame #5 Frame #6 Frame #7 Frame #8 Frame #9
iSCSI Verify
64K IO Transfer
Disk
ACK
ACKACK
ACK
Redefining Storage Economics
Port 0
Port 1
Port 2
Port 3
AoE data path
AoE: Connectionless, parallel delivery
AoE Initiator Array
AoE Mapping
64K IO Transfer1 2 3 4 5 6 7 8
Ethernet AoE Data Segment CRC
Port 0
Port 1
Port 2
Port 3
Path0
Path1
Path2
Path3
Write 1A
Write 1B
Write 1C
Write 1D
Write 2A
Write 2B
Write 2C
Write 2D
Write ACK
Write ACK
Disk
ACK IOAoE 8K block (data) x4
Redefining Storage Economics
Protocol Summary
iSCSI/FC/FCoE AoE
Topology Connection Connectionless
Transport IO Session/Serial delivery
IO TCP scatter/gather
Not required
Disk IO, Ethernet Frames
Multi-Path Multipath Driver
Manual Setup
Not required
Automatic
Dropped Data
Recovery
iSCSI: TCP with Retransmit
FC: Link Flow Control, SCSI timeout
AoE Correlation Timeout
AoE Retransmit
Out of Order
Data
iSCSI: TCP scatter/gather
FC: Prevented via Link Level Flow
Control
In order arrival not
mandated
Overhead Fibre Channel provides 2-3x higher performance than iSCSI
AoE is the most direct and fastest SAN protocol
Redefining Storage Economics
Management Comparison
Redefining Storage Economics
Provisioning Fibre Channel
60 Minutes
60 Minutes
60 Minutes
30 Minutes
210 Minutes
FC Stack
Redefining Storage Economics
Provisioning Fibre Channel
FC Stack
Redefining Storage Economics
Provisioning AoE
AoE Stack
5 Minutes
<1 Minute
21 Minutes
15 Minutes
Redefining Storage Economics
“ESG Lab has tested nearly every major storage system in the industry, and we found the ease of implementation and management of Coraid EtherDrive storage shockingly simple compared to most Fibre Channel and iSCSI systems. The simplicity of the system is brilliant, and the price-performance is staggering. It's fair to say we were enormously impressed.”
- Tony Palmer, Senior Engineer at ESG Lab
ESG Lab Validation
• Higher performance than Fibre Channel at 20% the cost
• Performance: 1200+ MB/sec per shelf• ESG was able to provision EtherDrive in
less than 2 minutes from power on
Redefining Storage Economics
Attendee Questions
Redefining Storage Economics
What is FCoE?
Host• Requires CNA• Additional Configuration• Reduces cabling
Network• Requires TOR• Requires DCB, not Ethernet• Extra network protocols
Storage• Does not replace FC• Complexities of FC remain• Still a separate network• FCoE is a bridge technology
Redefining Storage Economics
Cabling from your server is the biggest problem in your datacenter
You have lots invested in FC
FCoE or AoE?
Use FCoE if?
Things to Consider FCoE requires a CNA adapter Requires ToR switch to bridge FC
from FCoE to FC storage island Still requires FC to deliver data to
storage Requires lossless datacenter
Ethernet
Price/Performance and simplicity are high priority
Moving to virtualization and cloud architectures in datacenter
Standardizing on Ethernet
Use AoE if?
Things to Consider No multi-path software Requires jumbo frames (Up to 9K) Use standard Ethernet Works seamlessly with VMware
Redefining Storage Economics
Q&A
Thank youSend questions/comments to:
Join us for our next webinar:Coraid/VMware: The Secret to Virtualizing Mission Critical
ApplicationsSign up at www.coraid.com
Redefining Storage Economics
Extra Slides
Redefining Storage Economics
ESG Review of Ethernet SAN
Redefining Storage Economics
How does FCoE compare to AoE?
FCoE StackMulti-Path
Software
Redefining Storage Economics
Where in the stack is AoE relative to iSCSI?
Protocol Use Case
iSCSI Maps the SCSI protocol onto TCP
Assumes that initiators are talking over the “Internet”
Assumes a lossless link to transport data
Must append a CRC to detect bit errors at destination
TCP First conceived by Ham Radio Operators to transfer data
Goal: Maximize throughput on high latency links
“Creeps up” onto the right throughput
Many standards RFCs describe different TCP implementations
Not designed for low latency environments (storage)
Does not detect bit errors
Guarantees in order byte delivery of data
IP Designed to interconnect Ethernet Networks… “Internetwork”
Very sophisticated automated topology optimization
Hundreds of RFC’s
AoE Maps ATA disk commands onto Ethernet
Assumes multiple paths between initiator and target
Designed to operate on a datagram best effort protocol
Ethernet Very efficient connectionless datagram delivery service
Detects bit errors from source to destination
Redefining Storage Economics
Fibre Channel iSCSI AoE
Wedge Driver
(Multi-Path)
Not Required
Host Bus Adapter
Switch Ethernet Ethernet
Storage
Why is Coraid easier to manage?
MPIOPower
PathHDLM
SAN
Path
Veritas
DMP
Redefining Storage Economics
FCoE Summary
Host• Requires CNA• Additional Configuration• Reduces cabling
Network• Requires TOR• Requires DCB, not Ethernet• Extra network protocols
Storage• Does not replace FC• Complexities of FC remain• Still a separate network• FCoE is a bridge technology
Redefining Storage Economics
FC Ports
FC traffic
LAN
FC traffic
IP traffic
DCB Ports
CNAs
Access Layer Servers
DCB
FC SAN
FCoE Summary
RequiresThese
…AndThese
To SustainThis?
Redefining Storage Economics
View 1000-user Block on EtherDrive
SRX3500 x457.6TB
10k RPM SATA
View / ESX
Cluster
x2
VMware
Management
View Connection
Manager x2
vCenter x1
Users
512
vDTs512
vDTs
SRX420072TB
User Data Storage
Redefining Storage Economics
Is iSCSI really slower than FC?
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
Brocade 1020 Emulex OCe10102-F Qlogic QLE8142
FCoE
iSCSI
0
100,000
200,000
300,000
400,000
500,000
600,000
700,000
Brocade 1020 Emulex OCe10102-F Qlogic QLE8142
FCoE
iSCSI
FC is up to 3x faster than iSCSI TCP offload does not help Storage arrays don’t use TCP offload engines. How do they
cope with the additional iSCSI/TCP overhead?
See ITBrand Pulse Test Report: Server I/O with Real World ApplicationsCompares vendor CNA IO performance with a RAM storage device, thus exposing CNA bottlenecks
Oracle 8K Block Exchange 4K Block
74%
drop