View
6.189
Download
2
Category
Tags:
Preview:
DESCRIPTION
Joe Gervais,
Citation preview
Enabling Converged Networks with iSCSI over Data Center Bridging
Joe GervaisSenior Director, Product Marketing, Emulex
Dell Storage Forum 2011
Storage2
Agenda
• iSCSI over Data Center Bridging (DCB) Defined• Enabling Convergence• Companion Technologies For 10G iSCSI over DCB• Sample Topology
Dell Storage Forum 2011
Storage3
iSCSI over DCB Defined
Dell Storage Forum 2011
Storage4
iSCSI over DCB
• iSCSI over Data Center Bridging (DCB) protocols a.k.a. Enterprise iSCSI
– Data Center Bridging Capabilities eXchange Protocol (DCBX)› Supports centralized configuration of DCB and related protocols› Initiated by endpoints (hosts/storage) announce themselves with iSCSI
Type/Length Value (TLV)
– Enhanced Transmission Selection› Provides priority groups with bandwidth controls
– Priority Flow Control (PFC)› Enable lossless Ethernet operation to provide deterministic performance
• What’s needed– iSCSI HBA supporting iSCSI over DCB
› Could be a NIC – some limitations– 10G Ethernet Switch supporting DCB– 10G Storage Array supporting iSCSI over DCB
Dell Storage Forum 2011
Storage
Data Center Bridging Exchange Protocol (DCBX)
Automatic “handshake” to allow device to exchange Enhanced Ethernet capabilities• Simplifies configuration• Identify misconfigurations• Allows switch to override server parameters
Based on Link Layer Discovery Protocol (LLDP)
Dell Storage Forum 2011
Storage
Benefit Hardware-offload iSCSI
Software iSCSI
Performance iSCSI uses separate I/O stack than NIC traffic. Maintains consistent storage performance under varying LAN traffic conditions.
LAN congestion leads to PFC pause frames affecting iSCSI traffic.
• Essential to have lossless, consistent, reliable storage when sharing the same network regardless of LAN traffic conditions
• DCB designed to address this concern using PFC for different traffic types
– Uses VLAN priorities - each can be paused
– Ensures zero data loss when receive queues are full
• Implementation specifics may limit efficiency of PFC
– Hardware iSCSI vs. Software iSCSI
Lossless Ethernet
• PAUSE
• Ethernet Link• Receive
Queues• Transmit Queues
• Zero
• One
• Two
• Three
• Four
• Five
• Six
• Seven
• Priority
• Zero
• One
• Two
• Three
• Four
• Five
• Six
• Seven
• Priority
• STOP
Dell Storage Forum 2011
Storage
Server Hardware
OS
Hypervisor
App
OS
App
OS
App
OS
App
OS
iSCSI Target
LAN
NIC Buffer Pool
App
OS
App
OS
CN
A P
ort
Adapter Hardware
LAN app trafficHardware iSCSI
iSCSI Buffer Pool
Initiates iSCSI transactions, instructs OS to pre-allocate buffers for transaction
PG1PG2
Lossless Ethernet - Hardware iSCSI
Dell Storage Forum 2011
Storage
Server Hardware
OS
Hypervisor
App
OS
App
OS
App
OS
App
OS
No iSCSI session awareness, hence cannot pre-allocate buffers in shared OS buffer pool. Buffer over-run results in upstream Pause Frames QOS is
compromised
iSCSI Target
LAN
NIC Buffer PoolShared between LAN and iSCSI traffic
App
OS
App
OS
CN
A P
ort
Adapter Hardware
PriorityPauseFrame
Software iSCSI LAN app traffic
PG1PG2
iSCSI traffic suffers from unnecessary pause frames especially with concurrent TCP/UDP traffic… therefore not eliminating jitter
Lossless Ethernet - Software iSCSI
Dell Storage Forum 2011
Storage9
Enabling Convergence
Dell Storage Forum 2011
Storage10
Barriers to Convergence
• iSCSI on Gigabit Ethernet best practices architected separate Ethernet network for iSCSI traffic– Servers - iSCSI on separate NICs– Switches – iSCSI on separate switches
• Rational– Congestion causes packet loss, creating non-deterministic performance– TCP/IP error recovery expensive from a predictable latency perspective
• In virtualized environments, best practices employ multiple NICs for traffic isolation (iSCSI, vMotion, production, management, etc.)– Enables effective bandwidth caps between services, ensures available
bandwidth at server
Dell Storage Forum 2011
StorageEmulex Confidential
• Large Enterprise– Provide Deterministic
Performance› Requires Lossless Ethernet
(DCB)– Support Convergence
› DCB support› Multichannel Support
– Enterprise class management
• Small / Medium Enterprise– Support 1 -> 10 migration
› Multichannel support› DCB support
– Support 1G backwards compatibility› 10GBASE-T support
– Easy-to-use tools for configuration/management› DCBX based network
configuration
Enabling Converged iSCSI
User Experience
Lossless Ethernet
Multichannel
10GBaseT
Dell Storage Forum 2011
StorageEmulex Confidential
User Experience
• Hardware iSCSI enhances other aspects of the user experience
Benefit Hardware-offload iSCSI Software iSCSIScalability Support of more VMs in virtualized
environmentsNot performance-optimized
Reliability Attachments to iSCSI targets are maintained in hardware
Disruption in the host could lead to the loss of sessions
Manageability Emulex OneCommand™ Manager provides a single, uniform management interface; procedures for attaching to iSCSI targets are the same on all supported host operating systems. Graphical and command-line interfaces are provided.
The software iSCSI initiator runs on top of the TCP/IP stack and is managed differently for every operating system.
OneConnect™ iSCSI adapters can be managed across the enterprise from a single console.
Software iSCSI is managed locally on each host.
Dell Storage Forum 2011
StorageEmulex Confidential
User Experience - Scalability and Reliability
• Hardware iSCSI demonstrated support for more VM’s per server, given performance objective of 5k IOPs per VM
• Reduced need for physical servers translate to power and cost savings• Attachments to iSCSI targets maintained in adapter hardware – immunity to host disruptions that
could lead to session loss in software initiators
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS
App
OS… …
iSCSI storage
iSCSI storage
EthernetSwitch
EthernetSwitch
VM1 VM2 VM3 VM26 VM1 VM2 VM3 VM17
Hardware iSCSI Software iSCSI
53% more
Software iSCSI
Hardware iSCSI
0
5
10
15
20
25
30
VMs
Dell Storage Forum 2011
Storage
• OneCommand Manager provides a single, uniform management interface
– Procedures for attaching to iSCSI targets are the same across supported OS
• Alternatively, can use VMware vSphere Client to manage OneConnect iSCSI adapter
– SCSI Management API (IMA) plugin provided for VMware
– Hardware iSCSI device viewed in vSphere Client under Storage Adapters in the Configuration tab
• Less complicated than software iSCSI, which requires configuring a Vmkernel connection to ESX, then binding to the physical NIC
User Experience - Manageability
Dell Storage Forum 2011
StorageEmulex Confidential
Boot from SAN
• OneConnect iSCSI adapter can attach to an iSCSI target at boot time• iSCSISelect Utility is used to specify and configure a boot device on a
particular LUN– Invoked at boot-time by pressing CTRL-S– Configures iSCSI initiator to attach to desired target– LUN can be configured as boot device
Dell Storage Forum 2011
Storage
• Current versions of vSphere don’t directly support DCB protocols– No APIs for a NIC to support
DCBX and PFC.
• Since HBAs/CNAs offload iSCSI, DCB transparent to OS– DCBX and PFC all handled by
device– Host to Device interaction all at
SCSI layer
iSCSI over DCB with VMware
App
OS
App
OS
App
OS
App
OS…
iSCSI storage
EthernetSwitch
VM1 VM2 VM3 VM26
Hardware iSCSI
Dell Storage Forum 2011
Storage17
Companion Technologies For 10G iSCSI over DCB
Dell Storage Forum 2011
Storage
• Technology– Universal Multi-Channel (UMC)
presents multiple (typically 4) PCI functions per port to the OS
› Partitions the port into independent channels of user-defined bandwidth
– Switch-agnostic› Multiple channels can be
segmented based on MAC address and/or customer-provided VLAN ID (C-VLAN)
• Benefits– Retains networking best practices
in 1G to 10G migration› Physical 1G port becomes logical
10G port– Transparent to operating systems
› OS still sees dedicated NIC devices
– Complementary to DCB traffic shaping with ETS
› Bandwidth controls at both logical NIC and priority group level
Multi-Channel
Physical Port Channels
MultifunctionAdapters
Network/IP
Storage/iSCSI
Network/IP
Network/IP
Unique PCI function per logical port
Dell Storage Forum 2011
StorageEmulex Confidential
Low Cost Media Options for iSCSI over DCB
• 10GBASE-T– Backwards compatible with
Gigabit Ethernet– Uses Cat6/Cat6A unshielded
twisted pair cabling with RJ-45 connectors
– Supports 100m distances› Top of Rack, End of Row (EOR)
• SFP+ Direct Attach Copper– iSCSI adapter with SFP cage– Twinaxial Cable with SFP
modules on ends – Supports 5-7m distances
› Top of Rack› Optical versions available
supporting 100m or more
Dell Storage Forum 2011
Storage20
Sample iSCSI over DCB Topology
Dell Storage Forum 2011
StorageEmulex Confidential
Enterprise iSCSI Solution Example
• Sample Solution Overview– PowerEdge servers (R710)
in VMware cluster– EqualLogic array
(PS6010XVS)– PowerConnect switch (B-
8000e)– Emulex Enterprise iSCSI
HBAs› vCenter iSCSI IMA Plug-in
IMAPlugin
iSCSI Configuration today
Dell Storage Forum 2011
Storage
Enterprise iSCSI Solution Example
DCBX iSCSI TLV
DCBX iSCSI TLV
DCBX Enable PFCDCBX Enable 9000B MTUDCBX Enable ETS PG2 BW 1G
DCBX Enable PFCDCBX Enable 9000B MTUDCBX Enable ETS PG2 BW 1G
Dell Storage Forum 2011
Storage
Emulex OneConnect
• 10 Gigabit Ethernet products for Dell PowerEdge Servers– NIC, iSCSI and FCoE models
• iSCSI over DCB support with OCe11000 family of products– Built in support for DCB with Microsoft Windows,
VMware vSphere, Red Hat and Novell SUSE Linux operating systems
• SFP+ and 10GBASE-T models
Dell Storage Forum 2011
Storage24
iSCSI over DCBBenefits Summary
• Enables convergence of LAN and SAN traffic– DCB enables mixed traffic– Multichannel enables multiple logical NICs in one 10G link
• Deterministic performance in converged, mixed traffic environment– Lossless Ethernet using Priority Flow Control enables support
• Ease of use– Centralized configuration via switch with DCBX support
› Hosts, storage announce capabilities with iSCSI TLV› Switch enables PFC, Jumboframes, other parameters via DCBX
• Easy server integration– iSCSI HBAs such as Emulex OneConnect provide transparent support
Dell Storage Forum 2011
Storage
• See http://www.emulex-dell.com/
– To register for the Emulex quarterly Dell newsletter
– For information on Emulex OneConnect products supporting iSCSI over DCB
– For information about other Emulex solutions for Dell
– For iSCSI over DCB white paper
Resources & Call to Action
Dell Storage Forum 2011
Storage26
Thank you
Recommended