Download pdf - Cisco ACI Security in Action - …d2zmdbbm9feqrf.cloudfront.net/2016/usa/pdf/BRKACI-2320.pdf · Hardware Security Module (ACT2 HSM) • Validates the FPGA software, ... DHCP and ISIS

Cisco ACI Security in Action

Jason Gmitter, CCIE 12030

Technical Solutions Architect

BRKACI-2320

• Introduction

• Secure Fabric

• A Little About Policy

• Micro-Segmentation

• Service-Graphs

• ACI-TrustSec Integration

• Micro-Services

Agenda

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public

Trends Impacting Datacenter Security

EVOLVING

THREATS

NEW APPLICATIONS

(PHYSICAL, VIRTUAL

AND CLOUD)

NEW TRAFFIC TRENDS

Source: Cisco Global Cloud Index, 2012

BRKACI-2320 4


Secure Network Segmentation and Auditing

• Policy defined in simple language

• Group policy based secure multi-

tenant network segmentation

• L4-7 Service insertion to stateful

NGFW, IPS

• Centralized RBAC and two factor

Authentication

• Centralized Auditing and Security

Monitoring

SECURE

NETWORK

ACCESS

CONTROL

SECURITY

POLICYCENTRALIZED

AUDIT

MONITORING

ACCESS

BRKACI-2320 5


Outside

(Tenant

VRF)

App DBWeb

QoS

Filter

QoS

Service

QoS

Filter

ACI Fabric

Application Policy

Infrastructure Controller

Integrated GBP VXLAN Overlay

APIC

Introducing: Application Centric Infrastructure (ACI)

BRKACI-2320 6

• Introduction

• Secure Fabric



• Service-Graphs


• Micro-Services

Agenda


ACI SecurityAutomated Security With Built In Multi-Tenancy

Security AutomationEmbedded Security

• White-list Firewall Policy Model

• RBAC rules

• Hardened CentOS 7.2

• Authenticated Northbound API (X.509)

• Encrypted Intra-VLAN (TLS 1.2)

• Secure Key-store for Image Verification

• Dynamic Service Insertion and Chaining

• Closed Loop Feedback for Remediation

• Centralized Security Provisioning & Visibility

• Security Policy Follows Workloads

Distributed Stateless Firewall

Line Rate Security Enforcement

Open: Integrate Any Security Device

PCI, FIPS, CC, UC-APL, USG-v6

ACI Services

Graph

Micro-Segmentation

• Hypervisor Agnostic (ESX, Hyper-V, KVM*)

• Physical, Virtual Machine, Container

• Attribute Based Isolation/Quarantine

• Point and Click Micro-segmentation

• TrustSec-ACI Integration

Encryption

• Link MACSEC

• INS-SEC Overlay Encryption

• MKA, SAP

• GCM-AES-256/128-XPN

• GCM-AES-256/128

BRKACI-2320 8


APIC & ACI – A Crypto Based Platform

APIC

Same SSL Certificate

presented by all APICs to

External HTTPS

connections

• User and Orchestration access to APIC

• Web-Token or X.509 based certs

• APIC to Switch - SSL connection leveraging public key certificates

• APIC ISO is encrypted and keys are stored on APIC TPM

• Anti Counterfeit Technology-2 Hardware Security Module (ACT2 HSM)

• Validates the FPGA software, ROMMON software, switch preboot image and the switch full image

Cisco Signed Certificates

(shipped with switch and APIC)SSL

SSL

BRKACI-2320 9


Secure Fabric Overview Permissive Mode – Default mode of operation

Allows any existing fabrics with invalid SSL certs to operate normally

APICs to Switch communication is encrypted

No serial number based authorization

Strict Mode

Enforces serial number based authorization

Controllers and switches are manually authorized to join the fabric

Only Nodes with SSL cert with authorized Serial number are allowed

Strict Mode is allowed only when all the nodes in the existing fabric have valid SSL certificates

All communication between Switches and APICs are encrypted except LLDP, DHCP and ISIS

All switches need to have valid SSL certificate All Controllers need to have valid SSL certificateApproving/Rejecting Controller

BRKACI-2320 10


APIC Hardening

CentOS

6.3

CentOS

7.2Hardening

Surface Analysis: Nessus, Nmap, Nexpose, Qualys, etc …

Web Analysis: WebInspect , AppScan, BurpSuite, etc …

OS Network Config Testing: IPv6, IPv4, Firewalls, Listening services, Vulnerability scans, NFS permissions & TPS review

API: REST, SOAP, XML & JSON APIs Injection

CLI: CLI Injection

DB: Imperva's Scuba Scanner, password stored hash, encrypted data store

Manual: Security Penetration Testing

Not EC’edTarget Q3-CY16

BRKACI-2320 11


System AccessAuthentication, Authorization, RBAC

Universe

Tenant: Pepsi

App Profile

EPGs

L3 Networks

Tenant: Coke

App Profile

EPGs

L3 Networks

Fabric

Switch

Line Cards

Ports

• Local & External AAA (TACACS+, RADIUS, LDAP) Authentication & Authorisation

• RBAC to control READ and WRITE for ALL Managed Objects

• RBAC to enforce Fabric Admin and per-Tenant Admin separation

• Authentication for all Management Interfaces

• Roles: What user can do?

• Domain: Which Subtree role apply

APIC

BRKACI-2320 12


Will provide the ability to

prevent an admin from

adding a server to the wrong

Zone.

Security Domain

BRKACI-2320 13


To prevent Network admin from

creating contracts and VRFs.

Security admin responsible for

contract and VRF creation only.

Roles Based Access Control

BRKACI-2320 14


Capacity Dashboard

Troubleshooting Wizards

Drag and Drop Configuration

APIC provides full FCAPSAutomation and Operations

BRKACI-2320 15


Audit for all Changes• Audit-logs are native to

the object model, the aaaModLR type objects is an element of the subtree for each MO

• These contain:

• The object that was affected by a change

• What changed, Time stamp, user who made the change, the trigger,

etc.

BRKACI-2320 16


ACI Endpoint Tracker Application Enables Compliance and Auditing

• Tracks all attachment, detachment, movement of Endpoints

• Timestamps all end-point attach/detach events for auditing

• Stores full historical data in MySQL database for forensics

• Supports open visualization and query tools

• Built on top of open source ACI ToolkitBRKACI-2320 17


Configuring ACL Logging

• At contract-subject-

filter

• Between two end-

point groups

• Enabled on both

points

• Logging default value

• Enabled for deny

• Disabled for

permit

9300 (-EX) Required for Permit Logging

APIC Syslog

Splunk

SIEM Tools

BRKACI-2320 18


A whiteboard diagram of how network is configured to secure applications.

http://blog.esquilax.org/2015/01/14/generating-aci-diagrams-with-acitoolkit/

https://github.com/cgascoig/aci-diagram

ACI Diagrams

BRKACI-2320 19


• A python script that you can import the .json config of a tenant and have it converted into a word doc with explanation of how tenant is configured:

https://github.com/erjosito/stuff/blob/master/json2doc.py

JSON2DOC

BRKACI-2320 20


MAC-Limit > Maximum MACs, the following actions will happen on the port:

Learning is disabled

Exceeded mac-address are not added into CAM table

Exceeded mac-address traffic is dropped

Generate 1 syslog entry for violation action

MAC Limit would be supported only Per Port

FEX would not be supported

MAC Limit would enforce only on MAC and would not enforce on MAC & IP (this means IP learning will continue on learnt MAC)

Port-Security for MAC-Table: Protect Mode Behavior Details

Leaf Switch SW Release Timeline

9300 and 9300(-E) 2.0 Q3-CY-16

9300 (-EX) 2.X Q4-CY-16

BRKACI-2320 21


ACI Security Certifications

Certification ACI

Done

Target Q4 CY 16

Target Q3 CY 16

Target Q4 CY 16

Planning

BRKACI-2320 22

• Introduction

• Secure Fabric



• Service-Graphs


• Micro-Services

Agenda


The ACI Policy Model

Tenant ≈ GlobalWealth

VRF ≈ VRF

Bridge Domain ≈ Subnet/SVI

End Point Group ≈ Broadcast Domain/VLAN

Private VLAN

Contracts≈ Access Lists

L2 External EPG≈ 802.1q Trunk

EPG1 EPG2

L3 External EPG≈ L3 Routed Link

Any-Any

Replicates a

Traditional Switch

Security Domain ≈ Secure

BRKACI-2320 24


The ACI Policy Model – Network Centric Configuration

Tenant

Global VRF/Routing Table and Protocol

VLAN 30 BD10.10.30.1/24

VLAN 30 EPG

VLAN 20 BD10.10.20.1/24

VLAN 20 EPG

Any-Any Contract Any-Any Contract

VLAN 10 BD10.10.10.1/24

VLAN 10 EPG


BRKACI-2320 25



The ACI Policy Model – Network Centric Configuration

Tenant

Global VRF/Routing Table and Protocol

VLAN 30 BD10.10.30.1/24

VLAN 30 EPG

VLAN 20 BD10.10.20.1/24

VLAN 20 EPG

Any-Any Contract Any-Any Contract

VLAN 10 BD10.10.10.1/24

VLAN 10 EPG

L2 External (802.1q Trunk)

L3 External (Routed Interface)

Connect

To External

Switch

BRKACI-2320 26


DBAppWeb

Pieces and Parts of an Application Profile

Clients

End Points:

The things that actually make up the

application, such as: containers,

VMs, physical servers, etc.

VMs Physical Servers

BRKACI-2320 27


DBAppWeb


Clients

End Point Groups (EPG):

Grouping of like objects/services

Policy Enforcement Boundary

BRKACI-2320 28


DBAppWeb


Clients

Contracts:

These are the “services” provided by

or consumed by an EPG.

Describes what is allowed in/out of

an EPG, such as: Filters (ex: TCP

port 80 & 443), Service Graphs (ex:

FW, SLB, IDS/IPS), etc.

BRKACI-2320 29


Clients

Web App DB

A 3-tier app, aka “The Unicorn”

BRKACI-2320 30


There’s Always More Beneath the Surface

Clients

Web App

Common Services

DB

BRKACI-2320 31



Clients

Web App

Common Services

Content Mgmt

DB

BRKACI-2320 32



Clients

Web App

Common Services

Scan & Remediation Content Mgmt

DB

BRKACI-2320 33



Clients

Web App

Common Services


DB

Backup Service

BRKACI-2320 34



Clients

Web App

Common Services


Partner Services

$

DB

Backup Service

Partner

Staging

BRKACI-2320 35


And There are Many Layers of an Onion

Clients

Web App

Common Services


Partner Services

$

DB

Backup Service

Partner

Staging

BRKACI-2320 36


Infrastructure Policy Abstraction

Clients

Web App

Common Services


Partner Services

$

DB

Backup Service

Partner

Staging

BRKACI-2320 37


Partner

Staging

Abstract Policy Objects in ACI

Clients

Web App

Common Services


Partner Services

$

DB

Backup Service

BRKACI-2320 38


Abstract Object Relationships

Relationship Relationship Relationship Relation

shipEntity

En

tity

En

tity

En

tity

En

tity

Entity Entity Entity

Entity Entity

BRKACI-2320 39


Example of a “Common Service” Relationship (DNS)

Relationship Relationship Relationship Relation

shipEntity

En

tity

En

tity

En

tity

En

tity


Common Entity

(DNS) Entity

BRKACI-2320 40


Policy Reuse – Define Once, Use Many

Relationship Relationship Relations

hip

Rela

tions

hipEntity

En

tity

En

tity

En

tity

En

tity


Common Entity

(DNS) EntityD1:Ux

Service (P)

C C C

BRKACI-2320 41


Policy Reuse – Define Once, Use Many++

Relationship Relationship Relations

hip

Rela

tions

hipEntity

En

tity

En

tity

En

tity

En

tity


Common Entity

(DNS) EntityD1:Ux

Service (P)

C C C

C C C

C

C

C

BRKACI-2320 42


How do you built a Zero Trust Policy? Enforcing a policy is cool, but defining the policy is the hard part

• What is the Policy?

• Who knows the policy?

• Who is creating the policy?

• What tools can they use?

• Netstat

• PS –EF | grep listen

• Sniffer captures

• What is “good” traffic

• How real time is the data and how long is that information valid for?

BRKACI-2320 43


Cisco Tetration Analytics Architecture

Analytics Engine

Cisco Tetration

Analytics™

Platform

Visualization and

Reporting

Web GUI

REST API

Push Events

Cisco Confidential-NDA Required

Data Collection

Host Sensors

Network Sensors

Third-Party

Metadata Sources

Tetration

Telemetry

Configuration

Data

Cisco Nexus®

92160YC-X

Cisco Nexus

93180YC-EX

VM

BRKACI-2320 44


Bare Metal

NIC

92160YC-X

93180YC-EX

Spine with

X9732C-EX LC*

92160YC-X

93180YC-EX

HyperVisor

NIC

Where do they sit?

Software Sensor

What does the OS see?

What processes are running?

What sockets do I see?

Hardware Sensor

What flows does the switch see?

How are the flows performing?

What is the buffer status?

Cloud

* Future

BRKACI-2320 45


Application

Insight

Policy

Simulation

and Impact

Assessment

Automated

Whitelist

Policy

Generation

Forensics:

Every Packet,

Every Flow,

Every Speed

Policy

Compliance

and

Auditability

Cisco Tetration Analytics

BRKACI-2320 46


Application Discovery and Endpoint Grouping

Cisco Tetration

Analytics™

Platform

BM VM VM BM

BM VM VM BM

Brownfield

BM VM VM VM BM

Cisco Nexus® 9000 Series

Bare-metal, VM, & switch telemetry

VM telemetry (AMI …)

Bare-metal & VM telemetry

BM VM

BMVM

VM BM

VMVM

VM BM

BMVM

BM

Network-only sensors, host-only sensors, or both (preferred)

Bare metal and VM

On-premises and cloud workloads (AWS)

Unsupervised machine learning

Behavior analysis

BRKACI-2320 47


What is really running on my network ?Tetration Analytics outcome

Using Tetration Analytics

outcome and linking it to our

Services and Application CMDB

• (Service Owner)

• Service Category

• Service

• Service Offering

• Application

Dependencies

Security

Internet

DB Proxy

BRKACI-2320 48


Export Clusters and Policies

in JSON/XML format

Import Policy using ACI Toolkit

Automatic creation of

EPGs and Contracts

APIC

ACI

Toolkit

Data Network

Policy

Application PolicyTetration

Analytics

UCS

Nexus 9K

UCS

Get To Zero-Trust Model

python apic_tool.py -l admin -p <password> -u https://172.31.216.51 --tenant tetration --app default --config

whitelistdemo.json

BRKACI-2320 49

• Introduction

• Secure Fabric



• Service-Graphs


• Micro-Services

Agenda


The ACI Micro Segmentation Toolbox

Intra-EPG

isolation

Micro-segmented

EPGs

with attributes

Integration with

L4/L7 Services

ecosystem

BRKACI-2320 51

EPGs &

Contracts

ACI Policy Model


Cisco ACI Delivers Micro-SegmentationFlexible, Granular, Consistent

Attributes Based Intra-EPG BasedEPG Based

PROD

PODDMZ

SHARED

SERVICES

Basic DC Segmentation

DEV

TEST

PROD

Application Lifecycle

Segmentation

WEB

APP

DB

Service Level

Segmentation

Network-Centric

Segmentation

VLAN 1 VXLAN 2

VLAN 3

FW

OS

‘Linux’

IP

‘1.1.1.1’

FW

Name

‘Video’

Intra-EPG Isolation

All Workloads Can Communicate

Application Tier Policy Group

Isolate Workloads within Application

Tier

Application Tier Policy Group

Quarantine Compromised Workloads

Isolate

VMware VDS Microsoft Hyper-V KVM* Cisco AVS

Policy Driven Micro-Segmentation for Any Workload

Physical*Future

Segmentation

BRKACI-2320 52


Management tools for every organization

APIC GUI

API - Automation

vCenter Plugin

NX-OS

Style CLI

Choose the

right one!

BRKACI-2320 53


WAN

Micro-segmenting One Big Flat Subnet

WAN

http://172.16.1.100

VIP - 172.16.1.100

Web Application protected

by a NGFW at the perimeter

172.16.10.0/24

Web

VM1DB

VM

Web

VM2 Single Subnet to

simplify IPAM

tcp/3306

tcp/80

tcp/80

Web

VM1

Web

VM2VMMySQL

Joomla Web Application

Web Database Production

Environment

Load Balancer can reach

web servers, but not the

DB

Web Servers reach DB

via NGFW, but do not

need to talk to each other

BRKACI-2320 54

http://172.16.1.100/


Use case Intra-EPG: Shared Management/Backup• Set “Intra EPG isolation” to “Enforced”

EPG:

Web-Intra isolation

VMware vDS created by APIC

192.168.1.31 192.168.1.32 192.168.2.20

Inter-EPG communication is

permitted if there is a contract.

EPG: MGMT

• Intra-EPG Isolation makes “ALL” endpoints in an EPG isolated

• Can isolate mix of Physical and Virtual endpoints in same EPG

• Each VM may belong to different Tenant and their own context

• All VMs have one mgmt interface in Mgmt Ctx

Endpoints in the same EPG

can’t talk each other.

BRKACI-2320 55


ACI Attribute-based Micro-Segmentation

Quarantine Infected

VMs With

VM Name = VDI-MARKET*

Hypervisor

Virtual Switch (any)

Attributes Based Micro-Segments

(DVS, AVS, Hyper-V Switch, OVS*)

FW

VM Name = VDI Name = Finance-*IP = 1.1.1.x

FW

Attributec Type

MAC Address Filter Q1CY17

IP Address Filter Network

VNic Dn (vNIC domain name) VM

VM Identifier VM

VM Name VM

Hypervisor Identifier VM

VMM Domain VM

Datacenter VM

Custom Attribute

(VMWare AVS/vDS only)

VM

Operating System VM

BRKACI-2320 56


Web Database Dev

WAN

Acme Co. – Web App for online Shop

tcp/80

tcp/80

Web

VM1

Web

VM2VMMySQL

Joomla Web Application

Web Database Prodtcp/3306

http://172.16.1.100/acme

172.16.1.100

Test vDesktops

VMMySQL

Web Database Test

VMMySQLHAProxy

172.16.1.200

Test Site:

http://172.16.1.200/acme

Web

VM3

Pool automatically updated

by APIC when VM moves

into uEPG

New VM added to NGFW

rules allowing DB access

BRKACI-2320 57

http://172.16.1.100/acme

http://172.16.1.100/acme


Micro-Segmentation HW Support

N9300 N9300 (-E) 9300 (-EX)

AVS Useg (VM, IP, MAC) Yes Yes Yes

Microsoft Useg (VM, IP, MAC) Yes Yes Yes

vDS Useg (VM, IP, MAC) No No Yes

Bare-Metal (IP-EPG) No Yes* Yes

Bare-Metal (MAC-EPG) N/A Future (Q1 CY17) Future (Q1 CY17)

Openstack (ML2, GBP) No Future Future

Container No Future Future

* Caveat: 2

BRKACI-2320 58


Why secure the server infrastructure?

• CIMC/iLO: BMC/IPMI components are full-fledged micro-servers running their own OS/apps and have their own attack surface.

• Hypervisor management interface(s): Hypervisors can be compromised through security vulnerabilities within their kernel or application/services

• Compromising the BMC or the hypervisor could lead to complete control of the virtual infrastructure

Reduce attack surface as much as possible

Restrict lateral movement if compromised

BRKACI-2320 59


Infrastructure Physical Setup

ESXi 1

acme-dvsacme-avs

CIMC

vSwitch0

cimc

vmotion

storage

data

ESXi 2

acme-dvsacme-avs

CIMC

vSwitch0

cimc

vmotion

storage

data

Leaf 103 Leaf 104

mgmt mgmt

BRKACI-2320 60


Logical Setup – CIMC/iLO

Application Network Profile: SERVER_MGMT

EPG with physical domain.

Uses intra-EPG isolation

for preventing CIMC to

CIMC communication.

Contracts for shared

services access (DHCP,

NTP, DNS & ping)

Contract for accessing

CIMC (HTTPS, SSH &

KVM)

BRKACI-2320 61


Logical Setup – ESXi mgmt vmknics

Application Network Profile: VSPHERE_INFRA

Contracts for accessing

ESXi mgmt vmknics (SSH &

vSphere agent & console)

EPG with physical domain.

Uses intra-EPG isolation for

preventing ESXi to ESXi

communication.



PXE, NTP, DNS & ping)

BRKACI-2320 62


Logical Setup – Vmotion+Storage vmknics

Application Network Profile: VSPHERE_INFRA



DNS & ping)

EPG with VMM domain for DVS.

Intra-EPG communication is

allowed for VMotion traffic to

occur.

EPG with VMM domain for DVS.

Uses intra-EPG isolation for

preventing ESXi to ESXi

communication.

Contract for accessing

NFS storage

BRKACI-2320 63


Logical Setup – Tying it all together

BRKACI-2320 64

• Introduction

• Secure Fabric



• Service-Graphs


• Micro-Services

Agenda


• APIC defines Tenants

• EPG is VLAN/Subnet

• Fabric GW/Routing

• No Device Package

• ‘Happier’ SecOps

• Orchestrate it ALL!

• Vendor Device Package

EPG

Web

EPG

App

EPG

DB

EPG

Web

EPG

App

EPG

DB

Unmanaged Service Graphs

EPG

Web

EPG

App

EPG

DB

Managed Service Graphs

ACI L2 Fabric ACI No Package ACI by Design

APIC in Control

BRKACI-2320 66


SECURITY

Firewalls managed

separately from APIC

by security team.

Service attaches to EPG

VLANs/PGs and serves as

a host gateway to steer

traffic between VLANs.

ACI L2 Fabric

EPG

Web

EPG

App

EPG

DB

Allow flexibility to enable ACI fabric for EPG management, and attach security directly into EPGs.

Endpoint Group (EPG):

Creation of EPG segments still

done on APIC, EPs are virtual

machines or physical servers.

Contract:

Not implemented yet.

Firewalls control traffic

flows between EPGs.

Service Chain:

Not implemented yet.

Firewalls are GWs and

peer with external routers.

Programmability:

Northbound API to script

full Tenant network

creation.

EPG

Web

EPG

App

EPG

DB

EPG

Out

BRKACI-2320 67


ACI No Device Package

EPG

Web

EPG

App

EPG

DB

SECURITY

Customers enable full ACI fabric benefits with out forcing a device package.

Endpoint Group (EPG):

Creation of EPG segments still

done on APIC, EPs are virtual

machines or physical servers.

Contract:

Is between EPGs and

adds unmanaged Service

Graphs (no device pkg).

Service Chain:

Graphs in fabric and

Firewalls match SG fabric

attached VLANs/PGs.

Programmability:

Northbound API to script

full Tenant network and

unmanaged SG creation.

Firewalls still

managed separately

from APIC by the

security team.

EPG

Web

EPG

App

EPG

DB

Unmanaged Service Graphs Unmanaged Service Graphs

Physical appliance

attaches to the given

fabric ports and must

match VLANs.

Virtual appliance data

plane vNICs get

attached to proper

PGs via APIC.

BRKACI-2320 68


Service Automation Through Device Package

Open Device

Package

Policy

Engine APIC provides extendable policy model through Device Package

Configuration

Model

Device Interface: REST/CLI

APIC Script Interface

Call Back Scripts

Event Engine

APIC– Policy Manager

Configuration

Model (XML File)

Call Back Script

Provider Administrator can upload a Device Package

Device Package contains XML fine defining Device Configuration

Device scripts translates APIC API callouts to device specific callouts

APIC

BRKACI-2320 69


BD1

BD2BD1

BD2

• Routed Mode (Go-To)

• Transparent Mode (Go-Through)

FW Service Graph in the ACI Fabric

EPG App

EPGDB

ASAGraph B

10.0.0.0/24

Tenant B

External Internal

EPG Web

EPG App

Graph A

10.0.0.0/24 10.0.0.1 20.0.0.1 20.0.0.0/24

Tenant A

External Internal ASA

Bridge Domains need

flooding turned on, to

allow ASA to see and

bridge packets between

two EPGsBVI 10.0.0.10

Use port-channels on ESXi hosts instead of NIC teaming. It can break Go-Through mode.

BRKACI-2320 70


• Routed Mode (Go-To)

BD2BD

1

EPG Corp

EPG WebASA

Graph A

10.0.0.1 20.0.0.1

Tenant A

L3out - External L3out Internal

OSPF/BGPOSPF/BGP

SVI SVI

10.0.0.2 20.0.0.2 200.0.0.0/24

201.0.0.0/24

202.0.0.0/24

203.0.0.0/24

100.0.0.0/24

101.0.0.0/24

102.0.0.0/24

103.0.0.0/24

ASA(v) Dynamic Route Peering to ACI Leafs

BRKACI-2320 71


BD2BD

1

FirePOWER Insertion Topology in the ACI Fabric

• Transparent Mode

EPG A EPGB

NGIPS

Graph A

10.0.0.0/24

Tenant A

External Internal

BD

1

BD2

EPG A EPG BNGIPS

Graph B

10.0.0.0/24

Tenant B

External Internal

VRFs VRFs

OSPF/BGP

10.0.0.10 10.0.0.11100.0.0.0/24 200.0.0.0/24

201.0.0.0/24

202.0.0.0/24

203.0.0.0/24

101.0.0.0/24

102.0.0.0/24

103.0.0.0/24

BRKACI-2320 72


Why do we need Policy Based Redirection (PBR)?

• Options in ACI East to West traffic (EPG to EPG) is also desired Default route from EP is pointed to FW

Or Route Peering was preferred if the default route is to the Fabric

L3Out to EPG is done using L3Out + GoThrough (bridging on Firewall). Gothrough has scale

challenges

Route Peering was preferred

Route Peering needed 2 VRFs. VRF split is on the FW.

PBR eliminates the need for VRF split on the FW.

BRKACI-2320 73


PBR: Use case-1

EPG Client EPG Web

Only HTTP traffic is redirected

to FW, and then traffic is going

to Web endpoint

Other traffic permitted by

contract are going to Web

endpoint directly.

• Inspect specific traffic by FW.

BRKACI-2320 74


EPGs are in the same BD subnet

• Consumer and Provider EPGs are in the same BD subnet.

EPG Client EPG Web

BD1: 192.168.1.254/24

192.168.1.1/24192.168.1.2/24

BRKACI-2320 75


Supported topology

• If PBR for first node is enabled, PBR for 2nd node is not supported.

Example

L3

Unicast Routing: Yes

L3


PBR Node

L3

Unicast Routing: YesL2/L3

PBR Node

Normal Node

(Goto/Gothrough)

L3


L3


L2/L3

PBR Node

Normal Node

(Goto/Gothrough)

L3


EPGClient

BD1(192.168.1.254/24) BD2(192.168.2.254/24)

192.168.1.1/24 192.168.2.1/24

EPGWeb

external internal

Svc-BD1

(172.16.1.254/24)Svc-BD2

(172.16.2.254/24)

.100 .100

EPGClient

BD1(192.168.1.254/24) BD2 (no subnet)

192.168.1.1/24 192.168.2.1/24

EPGWeb

external internal

Svc-BD1

(172.16.1.254/24)Svc-BD2

(172.16.2.254/24)

.100 .100 .200

Svc-BD3

(172.16.3.254/24)

EPGClient

BD1 (no subnet)

192.168.1.1/24 192.168.2.1/24

EPGWeb

Svc-BD1

(172.16.1.254/24)Svc-BD2

(172.16.2.254/24)

external.100 .200

Svc-BD3

(172.16.3.254/24)

internal.100

L3 L3

L3 L3

L3 L3

PBR Node

PBR Node

PBR Node

.200

.200

Normal Node

(Goto/Gothrough)

Normal Node

(Goto/Gothrough)

BRKACI-2320 76

BD2 (192.168.2.254/24)


Supported topology

• VRF split design

VRF1

EPGClient

BD1(192.168.1.254/24) BD2(192.168.2.254/24)

192.168.1.1/24 192.168.2.1/24

EPGWebexternal internal

Svc-BD1

(172.16.1.254/24)Svc-BD2

(172.16.2.254/24)

.100 .100L3 L3

PBR NodeVRF2

VRF1

EPGClient

BD1(192.168.1.254/24) BD2(192.168.2.254/24)

192.168.1.1/24 192.168.2.1/24


Svc-BD1

(172.16.1.254/24)Svc-BD2

(172.16.2.254/24)

.100 .100L3 L3

PBR Node

VRF2

Route-leaking

Route-leaking

VRF1

EPGClient

BD1 (192.168.1.254/24) BD2(192.168.2.254/24)

192.168.1.1/24 192.168.2.1/24


Svc-BD1

(172.16.1.254/24)

Svc-BD2

(172.16.2.254/24)

.100 .100L3 L3

PBR Node

VRF2

Example

L3


L3


PBR Node

BRKACI-2320 77


What is Symmetric PBR?

Policy Based Redirection and Load balance to the L4toL7 devices simultaneously

Symmetric traffic distributionSame device receives both forward and reverse traffic

Resiliency using ECMP

Hash the traffic based on Source IP, Dest IP and Protocol Type

Scales to larger number of L4oL7 devices (32 devices per device cluster in Congo release)

Supported only on Sugar Bowl based TORs

Symmetric PBR is similar to N5k/6k/7k ITD Include feature.

BRKACI-2320 78


Symmetric PBR concept

SIP1

SN1

SN2

SN3

SIP1

SIP3

DIP1

DIP2

DIP3

Device ClusterFabricConsumer EPG Provider EPG

Fabric

Unmanaged

mode

Cluster Mode

GoTo

Add multiple

devices

BRKACI-2320 79


What is Copy Service ?

• Copies traffic flowing between two EPG’s

• Contract specifies what traffic is allowed & copied

• Can be sent to one or more destinations

• Support for uni and/or bi-directional traffic

• Traffic flowing through L4-7 devices can also be copied

• Supported only on Sugar Bowl ASIC’s

• Only physical copy devices supported

BRKACI-2320 80


Copy ServiceUse Case 1

EPG Client EPG Web

Traffic is copied to

IDS Original traffic is go to Web

endpoint directly.

EPGClient

EPGWeb

Contract

Copy

providerconsumer

IPS

• Inspect specific traffic.

BRKACI-2320 81


Copy ServiceUse Case 2

EPG Client EPG Web

Only HTTP traffic is copied

Original traffic is go to Web

endpoint directly.

EPGClient

EPGWeb

Contract

Copy

providerconsumer

Subject1 (permit HTTP)

Subject2 (permit ALL)

• Inspect specific traffic.

BRKACI-2320 82


Supported TopologyE

xam

ple

EPGClient

BD1 (192.168.1.254/24)

192.168.1.1/24 192.168.1.2/24

EPGWeb

Copy

Device

VRF1 VRF2

EPGClient

BD1 (192.168.1.254/24) BD2 (192.168.2.254/24)

192.168.1.1/24 192.168.2.1/24

EPGWeb

Copy

Device

VRF1

EPGClient

BD1 (192.168.1.254/24) BD2 (192.168.2.254/24)

192.168.1.1/24 192.168.2.1/24

EPGWeb

Copy

Device

VRF1

Route-leaking

• Copy Service can be deployed between EPGs in same BD, EPGs in different BD under

same VRF, EPGs in different BD in different VRF, EPGs in user tenant and tenant

common.

• Service Graph is mandatory

• Create Copy Device on APIC. (Physical device only)

• Supported only on Sugarbowl based HW. (Nexus 9300-EX)

• Copy applies for the traffic flow in both direction

BRKACI-2320 83

• Introduction

• Secure Fabric



• Service-Graphs


• Micro-Services

Agenda


Problem: Disjointed Identity & Security Policy Domains

Between Campus and Data Center

TrustSec domain

Voice Employee Supplier BYOD

Campus / Branch / Non-Fabric

TrustSec Policy Domain

Voice

VLANData

VLAN

Web App DBACI Fabric

Data Center

APIC Policy DomainAPIC

WAN

Disjointed

Identity

Policy Domains

TrustSec Policy Domain APIC Policy Domain

• Today customer has two disjointed identity and security policy domains in Campus and Data Center:

• TrustSec User Identity, SGT and SGACL in Campus

• APIC App Endpoint Identity, EPG and Contract in Data Center

• Customer Requirement:

• Need Common “Identity,” Tagging and “Security Policy” between TrustSec and ACI domains

BRKACI-2320 85


TrustSec

Domain

Solution: Normalize Identity and SGT/EPG

Phase 1

Identity and Policy Propagation between ISE and APIC

No SGT tags sent to ACI

Enforcement at N9300 border leaf

Leverage IP address as User identifier

Scale: TBD

Works with existing ACI infra: N9300 leafs and N9500 Spines

Target Timeframe: Q4 CY16

Phase 2

Policy Mapping between ISE and APIC AND Data plane Integration (ASR1K or ACI Spine)

ASR1K DCI translates SGT EPG-Class-ID

Enforcement at N9300 leaf

Scale: SGT/ EPG namespace

Works with existing N9300 leafs, requires upgrade of N9500 spines (line card/ fabric module available mid CY16)

Target Timeframe: Q1 CY17

TrustSec

Domain

ACI

Domain

SGT EPG

SGT EPG

ACI

Domain

iVXLANSGTASR1k

BRKACI-2320 86

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 87BRKACI-2320

Policy Mapping ISE to APIC Flow TrustSec SGT, Identity and Policy used to Program ACI EPG Policy

Enterprise

Backbone

ACI Policy Domain

ACI Border

Leaf (N9K)

ACI Spine (N9K)

Netw

ork

La

ye

rC

on

trolle

r La

ye

r


Netw

ork

La

ye

rC

on

tro

lle

r L

aye

r

ISE

BYOD

10.1.10.220SGT Mapping to ACI Policies

ISE Retrieves:

EPG Name: App EPG,

EPG Binding = 10.1.100.52

App Server10.1.100.52

App EPG

Endpoint = 10.1.100.52

External EPG Name = BYOD

EPG binding = 10.1.10.220

Plain

Ethernet

(no SGT)

BYOD

SRC:10.1.10.220

DST: 10.1.100.52

SGT: BYOD

xSRC:10.1.10.220

DST: 10.1.100.52

EPG BYOD

SRC:10.1.10.220

DST: 10.1.100.52

ISE Exchanges:

SGT Name: BYOD

SGT Binding = 10.1.10.220

SGT Policy

Enforcement

ACI Leaf

Enforcement

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 88BRKACI-2320

Policy Mapping APIC to ISE ACI EPG, App end-point and Policy used to Program TrustSec Policy

ACI Policy Domain

ACI Border

Leaf (N9K)

ACI Spine (N9K)

Netw

ork

La

ye

rC

on

trolle

r La

ye

r


Netw

ork

La

ye

rC

on

tro

lle

r L

aye

r

ISE

ISE Retrieves:

EPG Name: App EPG

EPG Binding = 10.1.100.52

App Server10.1.100.52

App EPG

Endpoint = 10.1.100.52

BYOD

10.1.10.220

Enterprise

Backbone

EPG Mapping to TrustSec Policies

Propagated with SXP

• SGT Name = BYOD

• EPG Binding = 10.1.100.52

BYOD

SRC:10.1.10.220

DST: 10.1.100.52

SGT: BYODACI Leaf

Enforcement

Plain

Ethernet

(no SGT)SGT Policy

Enforcement


ISE TrustSec SGT Policy Federated to APIC as External EPGs + Bindings

External EPGs Bindings

BRKACI-2320 89


Enterprise Branch

Phase 2 Data Plane Integration: ACI L3 DCI and TrustSec SXP exchange bindings

Data Center 1

iVXLAN

iBGP

IPVPN

PE-CECE-PE

MPLS

IPIP

WAN Management DomainiVXLAN

RD, Prefix , RT, L3 VNINext Hop – VTEP IPTunnel Encap – VXLANRouter MAC

PrefixNext Hop

EVPN Route

IP Route

Network

Admin

Application

Admin

PHYSICAL

SERVER

VLAN

VXLAN

VLAN

NVGRE

VLAN

VXLAN

VLAN

ESX Hyper

-VKVM

Hypervisor

Management

ACI FabricAPIC

VMware

Microsoft

Red Hat

XenServer

VMwareMicrosoft

Re

d

Ha

t

EVPN

HTTP/REST

iBGP / eBGP ICE Server

VRF-Lite or GlobalDMPVN, Ethernet

Features Scale (TBD)ASR1kACI

• DCI Scale:

• Required Scale: 4k VNIs, 4k VTEPs, 2k BDIs,

2k BGP Sessions, 4kVRFs, 250 Groups

• Offered Scale: 16 VNIs, 16k VTEPs, 16k Bridge

Domains, 4k VRFs, 4k BGP Sessions

• APIC and ISE controller policy plane

integration

• Golf/Multi-pod = iVXLAN-BGP-EVPN to ASR1k

• IVXLAN terminated into EVPN VRF-Lite or

global table in ASR1k

• Policy enforced in ACI

• No SGT carried in DMVPN or Ethernet

• SXP carries SGT and Bindings to ASR1k

• SGT translated to EPG class-id in IVXLAN

• EVPN – VRF Lite or Global Table with iVXLAN

spokes to each pod

• SGACL enforced as ACI contract in leaf

ACI Management Domain

40G links

SXP to exchange SGT

BRKACI-2320 90

• Introduction

• Secure Fabric



• Service-Graphs


• Micro-Services

Agenda


• ACI Integration Choices:

Application Driven: Suitable for dynamic creation of policies

Infrastructure Driven: Suitable when policies are pre-created by Infra team

• Policy Distribution:

Southbound Via APIC

Opflex*

• Stack of Choice:

Docker, Kubernetes, Mesos*

• Dimensions:

L4-L7 Services | Analytics | Visibility | etc.

ACI Integration Choices

BRKACI-2320 92


ACI Integration – Application Driven Workflow

Web

Plugins

Host-1 Host-n

DB Web DB

Container

Scheduler

DevOps (CI/CD)

Infra Admin

Image

Store

Plugins

Application Intent

Tenant-1:

External Web:80

DB:Port

Tenant-2:

External Web:80

DB:Port

2

Launching Apps

across Cluster4

DevOps Intent => ACI Policy3

Policy Instantiation5

Populate Infra

Policy Templates

1

Infra Admin

BRKACI-2320 93


ACI Integration – Infrastructure Driven

Web

Plugins

Host-1 Host-n

DB Web DB

DevOps (CI/CD)

Plugins

Application Intent

Tenant-1:

External Web:80

DB:Port

Tenant-2:

External Web:80

DB:Port

3

Launching Apps

across Cluster5

Policy Instantiation6

Populate Infra1

Infra AdminFetch EPG-Names

within a Container Domain

2Contiv

NetMaster4

Container

Scheduler

Image

Store

BRKACI-2320 94


Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.

• Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us.

95Presentation ID

CiscoLive.com/Online

http://ciscolive.com/Online

http://ciscolive.com/Online

http://ciscolive.com/us

http://ciscolive.com/us


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Related sessions

96Presentation ID

Please join us for the Service Provider Innovation Talk featuring:

Yvette Kanouff | Senior Vice President and General Manager, SP Business

Joe Cozzolino | Senior Vice President, Cisco Services

Thursday, July 14th, 2016

11:30 am - 12:30 pm, In the Oceanside A room

What to expect from this innovation talk

• Insights on market trends and forecasts

• Preview of key technologies and capabilities

• Innovative demonstrations of the latest and greatest products

• Better understanding of how Cisco can help you succeed

Register to attend the session live now or

watch the broadcast on cisco.com

Thank you


• GOLD Security labs – Advanced Security in ACI

• Links

Cisco Advanced Security in ACI (playlist)

https://www.youtube.com/playlist?list=PLvnemMVdgW1s77HuPk04VWwP47Y8EvlQl

Field Demonstration Options

BRKACI-2320 100

https://www.youtube.com/playlist?list=PLvnemMVdgW1s77HuPk04VWwP47Y8EvlQl


Security Ecosystem: Firewall Partner Device Package

Partner Device

Package

VM and

Physical

Mode Service Graph Model HA Feature

Cisco ASA FCS Yes Go-To Go-

Through

managed, unmanaged Yes FW, ACL,NAT

Palo Alto CA Yes Go-To Panorama is required

with managed or

unmanaged

in the works FW

Cisco

FirePOWER

FCS for

FirePOWER

5.4

Yes Go-Through managed, unmanaged No NGIPS, Advanced Malware

Protection

Check Point Q2CY16 Yes Go-To Go-

Through

Checkpoint console +

APIC (managed,

unmanaged)

Yes

(manual

OOB)

IPS, Antibot, sandboxing, AntiVirus,

App Control, DLP FW, ACL,NAT..

Fortinet v1.1 Released Yes Go-To Go-

Through

managed, unmanaged Yes FW

Fortinet v1.2 Q2CY16 Yes Go-To Go-

Through

managed, unmanaged Yes FW

BRKACI-2320 101


Security Ecosystem: ADC Partner Device Packages

Partner Device

Package

VM or

Physical

Mode Service Graph Model HA Feature

CitrixNetScaler GA Yes Go-To

(one-arm and two-

arm)

managed, unmanaged Yes

(manual /OOB)

ADC

F5 APIC Static Device

Package

(Direct BIG-IP

Integration)

FCS Yes Go-To managed, unmanaged Yes ADC

F5 APIC Dynamic

Device Package

(iApps based thru

connector)

FCS Yes Go-To managed, unmanaged Yes ADC, FW

A10 Thunder FCS Yes Go-To

(one-arm and two-

arm)

managed, unmanaged Yes ADC

Radware

Alteon

FCS Physical &

Virtual

(Q2/2016)

Go-To managed, unmanaged Yes ADC

Radware Defense Pro No Physical Go-Through managed, unmanaged No DDoS

Avi Networks FCS Virtual only Go-To managed, unmanaged

(Avi controller required)

Yes ADC

BRKACI-2320 102