91
Cisco Support Community Expert Series Webcast: Vishal Mehta Customer Support TAC Engineer February 19, 2014 Troubleshooting Common Features of UCS and Nexus 1000v

UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

  • Upload
    others

  • View
    11

  • Download
    0

Embed Size (px)

Citation preview

Page 1: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Support Community Expert Series Webcast:

Vishal Mehta Customer Support TAC Engineer

February 19, 2014

Troubleshooting Common Features of UCS and Nexus 1000v

Page 2: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

2 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

• Today’s featured expert is Cisco Support TAC Engineer Vishal Mehta

• Ask him questions now about UCS and Nexus 1000v

Vishal Mehta

Customer Support TAC Engineer

Page 3: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

3 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Panel of Experts

February 19, 2014

Ali Haider

Customer Support

Engineer

Gunjan Patel

Customer Support

Engineer

Page 4: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

4 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Today’s presentation will include audience polling questions

We encourage you to participate!

Page 5: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

5 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

If you would like a copy of the presentation slides, click the PDF link in the chat box on the right or go to:

https://supportforums.cisco.com/community/netpro/data-center

Or, https://supportforums.cisco.com/docs/DOC-40057

Page 6: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

6 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Do you have Nexus 1000v installed on a UCS environment?

a) I have UCS but no Nexus 1000v on it

b) I have Nexus 1000v but not on UCS

c) I am planning to have Nexus 1000v on UCS

d) I am using Nexus 1000v with UCS

Page 7: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Use the Q & A panel to submit your questions and panel of experts will respond.

Page 8: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Troubleshooting Common Features of UCS and Nexus 1000v

Page 9: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 9 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

UCS – End Host Mode

Nexus 1000v

Compatibility

Configurations

Port-Channeling

Quality of Service

Private VLAN

MAC Trace

Multicast

MS-NLB

Traffic Flow

TAC Cases

Q & A

Page 10: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 10 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 11: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

11 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Cisco recommends End-host Mode

No Spanning Tree Protocol

Reduces CPU load on upstream switches

Simplified upstream connectivity

UCS connects to the LAN like a Server, not like a Switch

Maintains MAC table for Southbound servers only

Prevents Loops by preventing Uplink-to-Uplink switching

Southbound traffic on same VLAN switched locally

Active/Active for Data Traffic and Active/Standby for Control Traffic

Page 12: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

12 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Traffic received from servers going to unknown destination, send it out on a pinned uplink

Unicast traffic from an uplink going to unknown destination then just drop the traffic

Consider only broadcasts received from servers or on designated broadcast uplinks

Broadcast traffic received on designated Broadcast Link that originated from one of its own servers, then just drop the traffic

Broadcast traffic received on designated Broadcast Link is flooded to all the servers and its not passed onto the uplinks

Page 13: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

13 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Server to server traffic on the same VLAN is locally switched

Uplink port to Uplink port traffic not switched

Each server link is pinned to an uplink port / port-channel

Network to server unicast traffic is forwarded to the server only if it arrives on pinned uplink port. This is termed as the Reverse Path Forwarding – (RPF) check

Packet with source MAC belonging to a server received on an uplink port is dropped (Deja-Vu Check)

Page 14: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

14 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Broadcast traffic is pinned on exactly one uplink port (or port-channel) i.e., it is dropped when received on other uplinks (Designated Receiver)

All multicast groups are pinned to same uplink port (port-channel)

Server to server multicast traffic is locally switched

RPF & deja-vu check also applies for multicast traffic

Page 15: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

15 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 16: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

16 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 17: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

17 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 18: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

18 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 19: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

19 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

N1k VSM is for control plane functions only & is not in path of data traffic

If Nexus 1010 or VSM goes down, it does not disrupt existing traffic from VMs on VEM, VEM continues to operate in headless mode, but new traffic & vMotion will FAIL

Nexus 1000v has no knowledge of FCoE

Layer 3 is recommended mode for VSM-VEM connectivity

N1k does not participate in Spanning-Tree

LACP negotiations are offloaded to VEM from VSM

Do not configure a dynamic vNIC policy for ESXi host service profiles VSM-VEM wont work

Do not enable fabric failover on the adapters in UCSM service profiles when using Nexus 1000v

Page 20: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

20 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

NTP Time synchronization across VEMs is a must

L3 mode uses UDP port 4785 for both source and destination (make sure that is not blocked in ACL/FW)

Don’t use VMware snapshots to take backup on VSM – Not Supported and causes Flapping of VEM

Using all switches – vSwitch, vDS and Nexus1000v ? – make sure cabling is correct (for C-series servers)

You can use same VLAN for control/packet/management (avoid VLAN 1)

Ensure control, management and packet VLANs are allowed in upstream devices

Page 21: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

21 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Used for critical communication

Bad for security reasons

Brings port online faster

Ports get straight access to the kernel even if the DVS is not up

Once a system profile is in use by at least one interface

Can add to the list of system VLANs

Cannot delete any VLANs from the list

Must be used for

Management VLAN

Storage VLAN (iSCSI/ NFS)

For L2 – make sure control VLAN is a system

For L3 – make sure vmk port-profile set with system VLAN

Page 22: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

22 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

port-profile type ethernet UPLINK

vmware port-group

switchport mode trunk

switchport trunk allowed vlan 10,18,19,25,90

channel-group auto mode on mac-pinning

no shutdown

system vlan 18,25,90

state enabled

port-profile type vethernet Layer3

capability l3control

vmware port-group

switchport mode access

switchport access vlan 25

no shutdown

system vlan 25

max-ports 1024

state enabled

port-profile type vethernet VAPP

vmware port-group

switchport mode access

switchport access vlan 90

no shutdown

system vlan 90

max-ports 1024

state enabled

Page 23: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

23 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

L2 mode • Needs L2 connectivity

• VSM uses BOTH Control and Packet vNICS

• Ctrl & pkt VLAN MUST be present in the

each segment of the data path

• More broadcasts

L3 mode • VEM-VSM should have L3 Connectivity (ping)

• VSM uses either mgmt0 or ctrl0 for control

• Requires a vmk in the VEM for encapsulation

• VSMs (active-standby) still require L2

• Less broadcasts

Page 24: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

24 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Can the same N1KV manage both ESXi & Hyper-V?

Answer: No. Separate N1KV switches should be deployed for different hypervisor

Page 25: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

25 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 26: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

26 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

When upgrading Nexus 1000v & VMware infrastructure, follow below tool for procedure

http://www.cisco.com/web/techdoc/n1kv/upgrade/utility/n1kvmatrix.html

Page 27: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

27 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Prior to any firmware upgrades or installation refer to below tool for UCS

http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html

Page 28: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

28 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Always upgrade drivers after UCS firmware upgrade

Page 29: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

29 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Do you find configuring these common features of UCS & Nexus1000v to be a complicated task?

a) Yes, I do

b) No, I do not

c) I’m not sure

Page 30: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

30 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 31: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

31 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Check below links for new configuration and troubleshooting documents

www.cisco.com/go/ucs

www.cisco.com/go/nexus1000v

MAC Trace

http://www.cisco.com/en/US/products/ps10477/products_tech_note09186a0080c17094.shtml

L2 Multicast

http://www.cisco.com/en/US/partner/products/ps10477/products_configuration_example09186a0080c20af4.shtml

Private VLAN

http://www.cisco.com/en/US/products/ps10280/products_configuration_example09186a0080c1e923.shtml

More externalization from Cisco TAC on its way

Page 32: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

32 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

N1k on UCS-C series – use LACP

When connecting to upstream switches with multiple uplinks or vPC/VSS

Supports up to 16 physical links in the bundle

LACP uses native vlan for PDU exchange

Check for LACP counters on Upstream Switch

Page 33: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

33 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 34: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

34 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

N1k on UCS-B series – use MAC-Pinning

MAC-Pinning is only supported port-channeling option for N1k and UCS-B

No “port-channel” configuration needed on upstream switches

Used when upstream switches cannot be clustered using vPC or VSS

Page 35: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

35 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

UCS Global QoS

Only CoS Values 0-6 can be assigned. CoS 7 is reserved for Control

6 class of traffic – Platinum, Gold, Silver, Bronze, Best Effort & Fibre Ch.

The only required classes are Best Effort & Fibre Channel (default)

Page 36: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

36 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

UCS QoS – vNIC

Cisco VIC card by default will overwrite the CoS value set by the host

Select Full Host Control to trust the markings done by the host / software switch, like Nexus 1000v

Page 37: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

37 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

QoS policy applied to the vNIC of the adapter

Page 38: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

38 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Nexus 1000v QoS Basics

N1k has 8 pre-defined traffic protocol classes

Per VEM, only 16 classes of traffic (8 CoS values & 8 Traffic Protocols)

The sum of bandwidth percentages for all classes in a policy-map should equal 100

Infrastructure traffic (such as vMotion, VMware Mgmt & N1k Control/Mgmt/Packet) should always have bandwidth reserved for times of congestion

If the N1k Control/Mgmt/Packet traffic does not have the proper resources allocated, VEM-to-VSM communications can suffer during congestion

Traffic

Protocol

N1k Control

N1k Packet

N1k Mgmt

vMotion

VMware

Mgmt

FT Logging

NFS

Storage

iSCSI

Storage

Page 39: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

39 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Nexus 1000v CBWFQ

Page 40: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

40 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Nexus 1000v Configuration

Page 41: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

41 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Nexus 1000v Verification

Page 42: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

42 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Following are important points to remember for PVLAN in UCS-N1k setup

Only Isolated Ports are supported in UCS

With N1k installed – Community VLANs can be used but promiscuous port must be on N1k

A server vNIC cannot carry both regular & isolated VLANs

No support for Promiscuous Ports/Trunks, Community Ports/Trunks or Isolated Trunks

Promiscuous ports need to be outside of the UCS domain

A vNIC can only have one Isolated VLAN

For Bare Metal OS (Windows/Linux) – create vNIC with Isolated VLAN and set as native VLAN

For ESXi – Separate vNIC needs to be created just to carry the Isolated VLAN

Page 43: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

43 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Link below explains all possible configurable options for Private VLAN on UCS and Nexus 1000v

http://www.cisco.com/en/US/products/ps10280/products_configuration_example09186a0080c1e923.shtml

Isolated PVLAN with promiscuous port on an upstream device

Isolated PVLAN on N1K with promiscuous port on an upstream device

Isolated PVLAN on N1K with promiscuous port on N1K uplink port−profile

Community PVLAN on N1K with promiscuous port on N1K uplink port−profile

Isolated PVLAN on VMware DVS with promiscuous port on the DVS.

Community PVLAN on VMware DVS switch with promiscuous port on the DVS

Page 44: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

44 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

http://www.cisco.com/en/US/products/ps10477/products_tech_note09186a0080c17094.shtml

MAC:A MAC:B

Page 45: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

45 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

IGMP snooping allows a Layer 2 switch to dynamically determine which hosts connected to a particular VLAN in the switch need to receive a particular multicast transmission. This is done by snooping or listening & inspecting IGMP membership reports

In the absence of an upstream multicast router, enabling IGMP snooping querier allows a Layer 2 switch to send IGMP queries to the host (in place of the Multicast Router) so that the multicast membership is kept up-to-date on the switch

The aim of UCS release 2.1 is to provide this flexibility in configuring/modifying

IGMP snooping

IGMP querier

IGMP Snooping configuration is only available and configurable on a per VLAN basis. You cannot enable/disable IGMP Snooping globally

Disabling of IGMP snooping is supported in both End Host Mode (EHM) and Switch Mode

Page 46: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

46 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

L2 Multicast configuration on UCS, Nexus 5000 and Nexus 1000v at below link:

http://www.cisco.com/en/US/partner/products/ps10477/products_configuration_example09186a0080c20af4.shtml

Page 47: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

47 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

UCS supports all 3 modes of MS-NLB – Unicast, Multicast, Multicast+IGMP

Configuration requirements if NLB is Multicast or Multicast IGMP:

Static ARP entries with the Virtual NLB IP & MAC Address on edge router

An IGMP Querier in the environment for Multicast to work (Multicast+IGMP)

Static multicast MAC entries on all the upstream switches in the local segment that needs to communicate with the NLB IPs (Multicast mode without IGMP)

IGMP Snooping needs to be enabled on all switches, including UCS (Multicast+IGMP)

Traffic behavior with MS NLB:

The source MAC on traffic outbound from the NLB nodes is unicast always

The destination MAC on traffic outbound from the NLB nodes is unicast always

In IGMP/Multicast mode, the destination MAC on traffic inbound to NLB is multicast

In Unicast mode, the destination MAC on traffic inbound to NLB is unicast

N1k supports only Unicast Mode from version 4.2(1)SV1(5.1) onwards

Page 48: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

48 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Configuration on Cisco Layer 2 Switches for MS-NLB

www.cisco.com/en/US/products/hw/switches/ps708/products_configuration_example09186a0080a07203.shtml - IOS 6500

http://www.cisco.com/en/US/tech/tk828/technologies_configuration_example09186a0080c13ba0.shtml - Nexus 7000

MS-NLB with VMware

http://deinoscloud.wordpress.com/2010/02/10/microsoft-network-load-balancing-nlb-on-vmware-esx/

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006778

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1556

Cisco Forums on UCS

https://supportforums.cisco.com/thread/2112862

https://communities.cisco.com/message/109945

Page 49: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

49 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 50: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

50 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 51: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

51 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 52: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

52 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 53: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

53 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 54: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

54 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 55: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 55 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

How often do you contact TAC for UCS-N1k troubleshooting?

a) I work with TAC on a regular basis

b) I sometimes contact TAC

c) I rarely contact TAC

d) I have never worked with TAC

Page 56: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 56 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 57: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 57 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Setup:

vMotion across FI via N7k fails or extremely slow

vMotion traffic switched locally within same FI works smoothly

vMotion traffic from UCS is tagged with CoS value 2

Default QoS setting on N7k was changed

N7k interface output discards increments during vMotion

CoS 1 & 2 were not mapped to any class-maps on N7k

This was causing packets with CoS 1 & 2 to be dropped

FIX:

class-map type queuing match-any 1p7q4t-out-q-default

match cos 1-2, 6

Page 58: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 58 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Customer Network has VLAN ID 8 as Native VLAN

All Management IP addresses including that of ESXi are on native VLAN

After changing Native VLAN to 100 – cannot ping the ESXi Management

Need to change ESXi settings as shown below:

Page 59: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 59 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

With bare metal installation of an OS (Windows/Linux) on a UCS blade, take into consideration the fact that UCS vethernet ports are always trunk interfaces

To essentially make a vNIC into an "access" port, configure one VLAN per vNIC, with the Native VLAN checkbox checked so that packets sent across that trunk are untagged

Page 60: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 60 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 61: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 61 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Problem: Orlando hosts (VEM) can be added into N1k but not those in San Jose.

Errors:

1. vCenter reports “Operation timeout” when trying to add VEMs to VSM

2. Following error on VSM 2013 Mar 10 15:31:15.875 N1k-ORL VMS-5-DVS_HOSTMEMBER_INFO A host with name= [CiscoLive.a.sanjos.ca] & uuid=[789abc3d-1234-45ab-9b56-1w123456789f] is added to the dvs. The host is not found as a module in the VSM configuration.

Issue: VEM MAC address was not getting learned on upstream switches

Page 62: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 62 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Troubleshooting:

1. Initially suspected long distance between Data Centers

2. Checked VSM-vCenter connectivity – healthy connection

3. Checked opaque data on VSM and VEM is consistent

4. Verified the configuration across Data Centers devices – UCS, N7k, N1k

5. Checked if VSM & VEM MAC addresses are learned on upstream switches for

the control VLAN

Solution:

Broadcast from VEMs on San Jose Site were not getting programmed correctly on

the Fabric Path interfaces of Chicago site

VEM MAC not learned on 4948 switch in Orlando

Issue on Nexus 7000 fabric-path port

Page 63: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 63 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Make sure it is defined in UCS

Defined & allowed on upstream switches

Added into vNIC of service profile

Defined on N1k

Allowed on port-profile of vEths on N1k

Case: Cannot add all the VEMs (from same UCS chassis) to VSM

Symptom: on vCenter – it shows hosts being added to N1k. however, same host not found under ‘show module’, hosts appears on ‘show module vem missing’

Solution: The 3 VEMs which were having issues were un-bounded from service profile template and vNIC had missing VLANs, one of them being control VLAN

Page 64: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 64 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

VMs cannot ping Default Gateway:

Is Default Gateway a Firewall? Check the settings

Check the IP subnet & mask

ARP Failing – Proxy ARP?

Firewall responding with its own MAC address instead that of the blades?

Disable proxy-arp on ASA example:

http://www.cisco.com/en/US/docs/security/asa/asa84/configuration/guide/route_overview.html#wp1106863

Troubleshooting Tip:

Create SVI (interface-vlan) on Upstream Switch in same vlan and try to ping both VM & Default Gateway. This helps to narrow down the scope of the problem.

Page 65: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 65 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

When you install ESXi, vmk0 gets assigned the MAC address of the first NIC in the server

This is not a problem in traditional server environments where the MAC address is burnt into the NIC

Page 66: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 66 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

How it happens in UCS ?

Existing Service Profile MAC Addresses:

vNIC0 MAC address of 00:25:B5:00:00:0E

vNIC1 MAC address of 00:25:B5:00:00:0F

Install ESXi on the blade & MAC address of vmk0 is now 00:25:B5:00:00:0E

Later move new SP to blade with following MAC Addresses:

vNIC0 MAC address of 00:25:B5:00:00:0A vNIC1 MAC address of 00:25:B5:00:00:0B

Even though the original server has new mac address for its vNICs, vmk0's MAC is still 00:25:B5:00:00:0E

Duplicate MAC address has now been introduced

This can cause an issue with intermittent connectivity and loss of around 30-40% of ping packets

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031111

Page 67: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 67 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 68: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 68 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Best Practice:

1. For bare metal installations of OS (Microsoft, RHEL, SUSE) we suggest using fabric failover and no teaming in the OS. This saves loading drivers & extra configuration on the OS side

2. For VMware hypervisor with UCS running 1.4 or newer, we suggest not to use fabric failover vSwitch or DVS or N1k will automatically load balance & failover

3. For Microsoft hypervisor with UCS running 1.4 or newer, we suggest using fabric failover

4. With iSCSI, vNICs should not use Failover feature as the best practice is to use host-based multi-pathing drivers for all load balancing & failure handling

Recommendation:

Design for redundancy by using vNIC Failover or NIC teaming/bonding, but never both

Page 69: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 69 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Does Adapter support it ?

http://www.cisco.com/en/US/products/ps10277/prod_models_comparison.html#~tab-a

Page 70: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 70 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Error on UCSM

Page 71: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 71 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 72: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 72 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Page 73: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

73 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

To complete the evaluation, please click on link provided in the chat or in the pop-up once the event is closed.

Those who fill out the Evaluation Survey will enter a raffle to win:

Cisco Press eBook

(value up to $80 USD)

Page 75: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

75 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

A. The UCS system was implemented at the billion dollar Epic Game’s company to assist in production of some of the top selling video games for the Xbox 360, iPad, iPhone and iPod.

B. The UCS and Nexus 1000v system was implemented in the making of the new motion picture “Need for Speed” based on the famous video game series.

C. Electronic Arts used the UCS and Nexus 1000v in assisting with the infrastructure necessary to produce the new FIFA 14 game.

What do UCS and video games have in common?

Page 76: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

76 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: IOS-XR: Installing and Upgrading Software and Introduction to Cisco Software Manager

Tuesday, February 25

9:00 a.m. Mexico City

10:30 a.m. Caracas

12:00 p.m. Buenos Aires

5:00 p.m. Madrid

Join Cisco Expert:

Sergio Barragan

During this live event, Sergio will present Turboboot methods, CLI installation and IOS SR system upgrade.

Registration for this live webcast:

http://tools.cisco.com/gems/cust/customerSite.do?METHOD=E&LANGUAGE_ID=S&SEMINAR_CODE=S19871&PRIORITY_CODE

Page 77: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

77 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: OSPF Introduction Tuesday, March 4

10:00 a.m. JST Tokyo

or

Monday, March 3

5:00 p.m. San Francisco

Join Cisco Expert:

Tsuyoshi Suzuki

During this live event, Tsuyoshi will provide an overview of OSPF, how it works, terminologies and features to learn, and noteworthy points needed to learn when pursuing CCNA certification..

Registration for this live webcast:

http://tools.cisco.com/gems/cust/customerSite.do?METHOD=E&LANGUAGE_ID=J&SEMINAR_CODE=S19864&PRIORITY_CODE

Page 78: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

78 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: Deploying and Troubleshooting Cisco Unified Contact Center Express Scripts

Tuesday, March 18

12:00 p.m. Moscow Time

9:00 a.m. Brussels Time

Join Cisco Expert:

Sergey Oliferov

During this live event, Sergey Oliferov discuss how to deploy and troubleshoot scripts for Cisco Unified Contact Center Express.

Registration for this live webcast:

http://tools.cisco.com/gems/cust/customerSite.do?METHOD=E&LANGUAGE_ID=R&SEMINAR_CODE=S19966&PRIORITY_CODE=

Page 79: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

79 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: Enterprise IPv6 Deployment

Join Cisco Expert: Chip Nielsen Learn and ask questions about enterprise IPv6 deployment.

Ends February 28

Join the discussion for these Ask The Expert Events:

https://supportforums.cisco.com/community/netpro/expert-corner#view=ask-the-experts

Topic: Cisco Unified Communications Manager (CUCM) Version 10.0

Join Cisco Experts: Raees Shaikh & Amit Singh Learn and ask questions install, upgrade requirements along with call routing enhancements of Cisco Unified Communications Manager (CUCM) Version 10.0.

Ends February 28

Page 80: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

80 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: FSPF Concept and Troubleshooting in Cisco SAN Environment

Join Cisco Expert: Upinder Sujlana Learn and ask questions about FSPF, VSAN interaction, load balancing and troubleshooting.

Starts March 3

Topic: SP Mobility

Join Cisco Expert: Rahul Pal, Arpit Menaria & Krishna Kishore D. V. Learn and ask questions about SP Mobility

Starts March 3

Join the discussion for these Ask The Expert Events at:

https://supportforums.cisco.com/community/netpro/expert-corner#view=ask-the-experts

Page 81: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

81 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: Latest Release of Cisco Unified Computing System

Manager (UCSM) 2.2

Thursday, February 27, 2014

3:30AM – 6:00PM San Clara, CA

Join Cisco Experts:

Krishna Sivakumar, Product Manager, Cisco

Sameh Zakhary, Product Manager, Cisco

Jason Shaw, Technical Marketing Engineer, Cisco

Join Cisco UCS Product managers and other data center management experts for a discussion on the UCS Manager 2.2 Release. We will raffle out one Beats by Dr. Dre earphones, at the end of the demo. You will have a chance to win Cisco Polo T-shirt by answering questions on site.

RSVP at:

http://www.meetup.com/csc-sv/events/162154302/

Page 82: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

82 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Topic: Working with Compliance Management and Configuration

Services (CMCS)

Thursday, March 13, 2014

12:00PM – 2:00PM San Clara, CA

Join Cisco Expert:

Mark Doering, Technical Marketing Engineer, Cisco

Join us to learn how Cisco Compliance Management and Configuration Service (CMCS) can help you to proactively manage regulatory, commercial and organization compliance. Lunch will be provided.

RSVP at:

http://www.meetup.com/csc-sv/events/160605382/

Page 83: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

83 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

https://supportforums.cisco.com

Page 85: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 85 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

https://supportforums.cisco.com

http://www.facebook.com/CiscoSupportCommunity

http://twitter.com/#!/cisco_support

http://www.youtube.com/user/ciscosupportchannel

https://plus.google.com/110418616513822966153?prsrc=3#110418616513822966

153/posts

http://itunes.apple.com/us/app/cisco-technical-support/id398104252?mt=8

https://play.google.com/store/apps/details?id=com.cisco.swtg_android

http://www.linkedin.com/groups/CSC-Cisco-Support-Community-3210019

Newsletter Subscription: https://tools.cisco.com/gdrp/coiga/showsurvey.do?surveyCode=589&keyCode=146298_2&PH

YSICAL%20FULFILLMENT%20Y/N=NO&SUBSCRIPTION%20CENTER=YES

Page 86: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 86 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

• Spanish https://supportforums.cisco.com/community/spanish

• Portuguese https://supportforums.cisco.com/community/portuguese

• Japanese https://supportforums.cisco.com/community/csc-japan

• Russian https://supportforums.cisco.com/community/russian

Page 87: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 87 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

https://supportforums.cisco.com/community/netpro/online-tools/mobile-

technical-support

Global community members can collaborate with colleagues and other support professionals with easy, on-the-go access to the community’s breadth of technical resources in their local language.

With the latest version of the mobile app, you can now access the Spanish, Portuguese, Japanese and Russians communities.

Page 88: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 88 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

https://supportforums.cisco.com/community/netpro/idea-center/cafe/blog/2013/06/07/ratings-

extended-to-documents-blogs-and-videos

Now your ratings on documents videos and blogs count give points to the authors!!!

So, when you contribute and get ratings you now get the points in your profile.

Help us recognize the good quality content in the community and make your searches easier.

Rate content in the community.

Page 89: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Cisco Confidential 89 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

A. The UCS system was implemented at the billion dollar Epic Game’s company to assist in production of some of the top selling video games for the Xbox 360 and iPad, iPhone and iPod.

B. The UCS and Nexus 1000v system was implemented in the making of the new motion picture “Need for Speed” based on the famous video game series.

C. Electronic Arts used the UCS and Nexus 1000v in assisting with the infrastructure necessary to produce the new FIFA 14 game.

For the Gears of War 3 launch, we boosted both our web presence and online forums for the growing number of fans and users. With Cisco UCS, we’re able to spin up new virtual machines on demand”. Under the old environment, Nilsson’s team would need two to three days to get a new physical server provisioned. “Today, I can get a new virtual machine up in minutes.” In addition to speedy, consistent, no-touch provisioning, Epic Games also benefits from a sharp decrease in operation costs with Cisco UCS.

What does UCS and video games have in common?

Page 90: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Thank you for Your Time!

Please take a moment to complete the evaluation

Page 91: UCS and Nexus 1000v Vishal Mehta - Cisco Support Community

Thank you!