Cisco Community - Unified Computing System …...Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. 2Cisco Public UCS QoS System Components QoS Philosophy User

Session ID – BRKCOM 2004

Unified Computing System (UCS) Quality of Service (QoS) and Security

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation_ID 2

UCS QoS

System Components

QoS Philosophy

User Configurable Parameters

QoS Tools

Congestion Management

Packet Flow Details

Agenda


System Components

Fabric Interconnect

(40 or 20 10GE ports) + (2 or 1 Expansion Slots)

Blade Chassis

Upto 8 half width blades or 4 full width blades

Fabric Extender

Host to uplink traffic engineering

Up to 8:1 Flexible oversubscription possible

Adapter

Virtualized adapter for single OS and hypervisor systems

Compute Blade

Compute Chassis

x86 Computer x86 Computer

X

I I

x8 x8 x8 x8

B

MGMT

S S

B

X X X X X

C C

A

G G

G G

SAN

G

R

A

G

G G

G

R

G

P M P

SAN LAN

Fabric

Switch

Fabric

Switch

Fabric

Extender

Fabric

Extender

Compute Blade

(Half slot)

Adapter

Compute Blade

(Full slot)

Adapter Adapter


Fabric

Switch

Compute Chassis Compute Chassis

Compute Chassis

Fabric

Switch

System Components - Details

Compute Blade

(Half slot)

x86 Computer

Compute Blade

(Full slot)

x86 Computer

Adapter

X

Fabric

Extender I

Fabric

Extender I

x8 x8 x8 x8

C

MGMT

S S

Fabric ASIC

Port ASIC

10GE/FC/1GE Media Access Controllers

Forwarding - Ethernet, Fibre Channel, Multipath

Policy Engine

Packet Buffering

FEXLink ASIC

Host to uplink traffic engineering

Connectivity detection & management portal

Virtual Interface Card ASIC

Virtualized adapter for both single OS and hypervisor systems

Ethernet and FC vNICs

CNA

FCOE couples Industry standard NICs and HBAs

Adapter Adapter C

X X X X X

C C

F

P P

P P

SAN

P

L

F

P

P P

P

L

P

V C V

SAN LAN


QoS Architecture

Compute Chassis

x86 Computer x86 Computer

X

I I

x8 x8 x8 x8

B

MGMT

S S

B

X X X X X

C C

A

G G

G G

SAN

G

R

A

G

G G

G

R

G

P M P

SAN LAN

Fabric

Switch

Fabric

Switch

Fabric

Extender

Fabric

Extender

Compute Blade

(Half slot)

Adapter

Compute Blade

(Full slot)

Adapter Adapter

No packet drops within the array

Largest buffers are on switch and host memory, so congestion pushed to edges

Priority Flow Control (PFC) used to ensure packet drops are at vNIC or Switch

All traffic in a CA system belongs to 1 of 6 System Classes

Four are user configurable while the other two are for FCoE and standard Ethernet

QoS parameters can be configured at a per system class level, or a per vNIC level.


User Configuration

Globally for each System Class

COS value for packets in this class

Drop/No-drop behavior

Strict Priority

Bandwidth/Weight

Users configure QoS parameters at two levels

Class Name FC Bronze

COSValue 3 0

Drop/No-Drop No-Drop Drop

Strict Priority No No

Bandwidth/Weight 20% 30%

Example


User Configuration

For each vNIC (Egress properties)

System Class for traffic from this vNIC

Rate limit (Mbps)

Burst Size (Kbytes)

Users configure QoS parameters at two levels

vNIC1 vNIC2 vNIC3

Class FC FC Bronze

Rate 4000 4000 5000

Burst 300 400 100

Example: Logical Server A


User Configuration – Example

Class Name FC Gold Ethernet BE

COS Value 3 1 0

Drop/No-Drop No-Drop Drop Drop

Strict Priority No No No

Bandwidth/Weight 1 (20%) 3 (60%) 1 (20%)

vNIC1 vNIC2 vNIC3

Class FC FC Eth. BE

Rate 4000 4000 5000

Burst 300 400 100

Logical Server A

Global System Class Definitions

vNIC1 vNIC2

Class Gold Eth. BE

Rate 600 4000

Burst 100 300

Logical Server B

FC Traffic High Priority

Ethernet

Best Effort

Ethernet


QoS Tools

Priority Flow Control

• Enables lossless Fabrics

for each class of service

• PAUSE sent per virtual lane

when buffers limit exceeded

Transmit Queues Ethernet Link

Receive Buffers

Eight

Virtual

Lanes

One One

Two Two

Three Three

Four Four

Five Five

Seven Seven

Eight Eight

Six Six STOP PAUSE

COS based

Bandwidth Management

• Enables Intelligent sharing of

bandwidth between traffic classes

control of bandwidth

•802.1Qaz Enhanced Transmission

10 GE Link Realized Traffic Utilization

3G/s HPC Traffic

3G/s

2G/s

3G/s Storage Traffic

3G/s

3G/s

LAN Traffic

4G/s

5G/s 3G/s

t1 t2 t3

Offered Traffic

t1 t2 t3

3G/s 3G/s

3G/s 3G/s 3G/s

2G/s

3G/s 4G/s 6G/s

Among the tools used are aggregate shapers at the vNICs (VIC Adapter), ETS, Policers at the switch for each vNIC.


Traffic Engineering

FI-1

Class-A Class-B Class-C

vNIC-1

vNIC-2

vNIC-3

FEX-1

FI-2

Blade-2, VIC-1

vNIC-1

vNIC-2

vNIC-3

FEX-2

VIC with 3

vNICs

2 Fabric Extenders

in chassis, each with

1 link to the switch.

2 FI, both with 1

connection to

each FEX

Blade-1, VIC-1

vNICs can be pinned to specific switches when created (with configurable failover to other switch)

Depending on requirements, vNICs could be pinned to one switch or distributed evenly

vNICs in System Class C pinned to one interconnect


Traffic Engineering

FI-1


vNIC-1

vNIC-2

vNIC-3

FEX-1

FI-2

Blade-2, VIC-1

vNIC-1

vNIC-2

vNIC-3

FEX-2

Blade-1, VIC-1

vNICs can be pinned to specific switches when created (with configurable failover to other switch)

Depending on requirements, vNICs could be pinned to one interconnect or distributed evenly

vNICs in System Class C distributed across interconnects


Congestion Spread

FI-1


vNIC-1

vNIC-2

vNIC-3

FEX-1

FI-2

Blade-2, VIC-1

vNIC-1

vNIC-2

vNIC-3

FEX-2

VICwith 3

vNICs

2 Fabric Extenders

in chassis, each with

1 link to the FI.

2 FI, both with 1

connection to

each FEX vNIC1in class C

generating large

amount of traffic

Pause packet

sent out for

Class-C

Class-C

congestion,

going into

switch on this

port

vNICs 1 and 2

in Class-C

backup in host

memory

Blade-1, VIC-1


QoS Tools

Priority Flow Control (aka Per Priority Pause)

Ability to assert pause per IEEE 802.1p lane

All internal UCS links support PFC

Shaping and WRR

Adapters can shape traffic generated by hosts. Available in VIC

WRR in Switch, FEX and Adapters

Documents

Cisco Community - Unified Computing System …...Presentation_ID © 2010 Cisco and/or its affiliates. All rights reserved. 2Cisco Public UCS QoS System Components QoS Philosophy User