Cisco ACE Training

© British Telecommunications plc

VDC DesignACE Training

November 2008

Andrew Holding

[email protected]


Agenda

•Scope•ACE Overview

– What is the ACE Module?– Software, Licences and Virtualisation– Deployment options– Resource Management– Redundancy– Key ACE terminology

•Layer 3-7 Server Load Balancing– TCP Review– Layer 3-4 Server Load Balancing– Layer 7 Server Load Balancing

•Configuration Examples & Important ACE features– Layer 3, Layer 4, Layer 7 load-balancing– Route Health Injection– Persistent Rebalance– ACE Connection handling– Stickiness – Source IP, Cookie– SNAT

•SSL•ACE Configuration Spreadsheet•ACE Documentation•Q & A


Agenda







Scope

•The scope of this training is to ensure that network designers understand the ACE topology and the configuration options used within the VDC design•This is a high-level training to explain basic features and ACE behaviour•It is assumed that attendees have basic load-balancing knowledge


Agenda







What is the ACE Module?

Application Control Engine

Layer 3-7 content-aware, virtualised, application load-balancer with SSL termination & initiation and security


What is the ACE Module?

CSS 11506

CSM

Appliances

Cat6K Modules

ACE Module

CSS 11503

CSS 11501

ACE Appliance


The Evolution of L4 to L7 Services

Infrastructure simplification with L4–7 Services integration Converged policy creation, management, and troubleshooting Reduced latency (single TCP termination for all functions)

Previous

IntegratedLayer 4

andLayer 7Rules

Now – with Application Control Engine


ACE Hardware Architecture

SwitchFabric

Interface

SwitchFabric

Interface

SupConnect

SupConnect

16G

100M DaughterCard 1

DaughterCard 1

DaughterCard 2

DaughterCard 2

8G

8G

SSLCryptoSSL

Crypto

10G

NP1NP1 NP2NP2

10G10G

ControlPlane

SAN OS

ControlPlane

SAN OS

2G

CDESwitch60Gbps

CDESwitch60Gbps


ACE Performance/Features

•Max of 4 ACE’s per chassis (64Gbps)

•4Gbps, 8Gbps, 16Gbps single link to Backplane

•4Million Concurrent connections

•~350K L4 connections per second

•Onboard SSL Offload (1K to 15K tps throughput)

•Virtualisation (250 Contexts)

•TCP Reuse

•DDoS protection

•etc


Agenda







ACE software versions

Version number for ACE: 3.0(0)A1(6.3b)

Based on SanOS release 3.0(0)BU identifier is “A”ACE software version 1.6(3b)

SanOS info has now (A2.x) been dropped for simplification;

“show ver” :-Software loader: Version 12.2[118] system: Version A2(1.1) [build 3.0(0)A2(1.1) adbuild_00:25:02-2008/06/05_/auto/adbu-rel3/rel_a2_1_1_throttle/REL_3_0_0_A2_1_1] system image file: [LCP] disk0:c6ace-t1k9-mz.A2_1_1.bin installed license: ACE-08G-LIC ACE-VIRT-050 ACE-SEC-LIC-K9 ACE-SSL-05K-K9

Note: ACE Module and ACE Appliance use different software images


ACE software versions (cont’d)

BNCMNSSW01>show mod Mod Ports Card Type Model Serial No.--- ----- -------------------------------------- ------------------ ----------- 1 1 Application Control Engine Module ACE10-6500-K9 SAD1021076N 2 6 Firewall Module WS-SVC-FWM-1 SAD100202V9 3 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL1005CBZP 4 1 SSL Module WS-SVC-SSL-1 SAD094307LT 5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL1004BPJU 6 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD1006061M 7 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD100301YX 8 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAL1005C12A 9 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAD100204FK

Mod MAC addresses Hw Fw Sw Status--- ---------------------------------- ------ ------------ ------------ ------- 1 0030.f275.b454 to 0030.f275.b45b 1.1 8.7(0.22)ACE A2(1.1) Ok 2 0013.c39f.63f8 to 0013.c39f.63ff 4.0 7.2(1) 3.2(4) Ok 3 0016.c810.3284 to 0016.c810.32b3 2.3 12.2(14r)S5 12.2(18)SXF1 Ok 4 0030.f274.f702 to 0030.f274.f709 4.0 7.2(1) 2.1(9) Ok 5 0013.c43a.8cb0 to 0013.c43a.8cb3 4.5 8.1(3) 12.2(18)SXF1 Ok 6 0013.c39f.cce0 to 0013.c39f.cce7 1.9 4.2(3a) Ok 7 0013.c39f.8530 to 0013.c39f.8537 1.9 4.2(3a) Ok 8 0016.c75a.a700 to 0016.c75a.a703 2.2 12.2(14r)S5 12.2(18)SXF1 Ok 9 0015.62e1.aee8 to 0015.62e1.aeeb 2.2 12.2(14r)S5 12.2(18)SXF1 Ok


ACE licensing

Base = 5 contexts (plus Admin), 1000 SSL tps, 4Gbps

Contexts = 50, 100 or 250Throughput = 8 or 16GbpsSSL = 5,000, 10,000, 15,000 tps

bncmnace02/Admin# show license status

Licensed Feature Count------------------------------ -----SSL transactions per second 5000Virtualized contexts 50Module bandwidth in Gbps 8

bncmnace02/Admin# show ver….Software loader: Version 12.2[118] system: Version A2(1.1) [build 3.0(0)A2(1.1) adbuild_00:25:02-2008/06/05_/a

uto/adbu-rel3/rel_a2_1_1_throttle/REL_3_0_0_A2_1_1] system image file: [LCP] disk0:c6ace-t1k9-mz.A2_1_1.bin installed license: ACE-08G-LIC ACE-VIRT-050 ACE-SEC-LIC-K9 ACE-SSL-05K-K9…


ACE Virtualisation

One Physical DeviceMultiple Virtual Systems

(Dedicated Control and Data Path)

• Traditional Device:– Single configuration file

– Single routing table

– Limited RBAC (Role Based Access Control)

– Limited resource allocation

25% 25% 20%15%15%100%

• Cisco Application ServicesVirtualisation:– Distinct configuration files– Separate routing tables– RBAC with contexts, roles, domains– Management and data resource control– Independent application rule sets– Global administration and monitoring


ACE Multiple Contexts

Physical Device

Context 1Admin

ContextContext

Definition,Resource

Allocation,FT Config

Managementstation

Context 2 Context 3

AAA

Admin Context + 250 Contexts (Licensed: five contexts in base code)


Agenda







ACE Deployment

Web Client 6500 with

ACE

Web Server

Physical View


ACE VLANs

Web Client

ACE

Web Server

Catalyst 6500

Client-side VLAN

Server-side VLAN

Logical View


Bridged (Layer 2) Mode

Server Default Gateway:Upstream Router

ACE Bridging

Subnet A

VLAN 10 VLAN 20


Routed (Layer 3) Mode

Server Default Gateway:ACE IP

ACE Routing

Subnet AVLAN 10

Subnet BVLAN 20


One-Armed Mode

Server Default Gateway:Upstream Router

Subnet BVLAN 20

Subnet CVLAN 30

Subnet AVLAN 10

ACE not in path – PBR or SNAT required for return traffic


Routed, Bridged or One-Armed Mode?

All of these “modes” can be mixed within, and between, contexts - the same context can have bridged interfaces, routed interfaces and one-armed interfaces

Advantages of bridged vs routed are;+ Routing protocols can be exchanged through the ACE+ Multicast packets can be passed through the ACE

Disadvantage of bridged vs routed;– Potential for bridge-loop if both ACEs go active-active (RPVST+

used to minimise impact. Note: MST not supported)– If SNAT required, then traffic must be “load-balanced”

One-armed (ACE is not inline for load-balanced traffic)+ Removes potential bottleneck- PBR or SNAT required


VDC ACE Topology

ACE has a static default route with a next-hop of the FW1 VRF, and server-subnet routes with a next-hop of the Cust VRF

VLAN%cust1-ace1-ss-vlan%

%cust1-fw1-vrf-name%

%cust1-ace1-vrf-name%

AC

E B

lock

VLAN%cust1-ace1-ns-vlan%

VLAN7

Fir

ewal

l Blo

ck

EIGRP%ace-blade1-hostname%-001/002

VLAN 501

VLAN 601

Subnet A


ACE Interface Configuration

interface vlan 231 description Client vlan ip address 172.16.31.5 255.255.255.0 no shutdown

– Routed interfaces:

interface vlan 231 bridge-group 3 no shutdowninterface vlan 232 bridge-group 3 no shutdown

interface bvi 3 description Server Access vlan ip address 172.16.31.5 255.255.255.0 no shutdown

– Bridged interfaces:


Which slot is the ACE in?

Cat6k>show modMod Ports Card Type Model Serial No.--- ----- -------------------------------------- ------------------ ----------- 1 1 Application Control Engine Module ACE10-6500-K9 SAD1021076N 2 6 Firewall Module WS-SVC-FWM-1 SAD100202V9 3 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX SAL1005CBZP 4 1 SSL Module WS-SVC-SSL-1 SAD094307LT 5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL1004BPJU 6 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD1006061M 7 4 SLB Application Processor Complex WS-X6066-SLB-APC SAD100301YX 8 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAL1005C12A 9 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAD100204FK

Mod MAC addresses Hw Fw Sw Status--- ---------------------------------- ------ ------------ ------------ ------- 1 0030.f275.b454 to 0030.f275.b45b 1.1 8.7(0.22)ACE A2(1.1) Ok 2 0013.c39f.63f8 to 0013.c39f.63ff 4.0 7.2(1) 3.2(4) Ok 3 0016.c810.3284 to 0016.c810.32b3 2.3 12.2(14r)S5 12.2(18)SXF1 Ok 4 0030.f274.f702 to 0030.f274.f709 4.0 7.2(1) 2.1(9) Ok 5 0013.c43a.8cb0 to 0013.c43a.8cb3 4.5 8.1(3) 12.2(18)SXF1 Ok 6 0013.c39f.cce0 to 0013.c39f.cce7 1.9 4.2(3a) Ok 7 0013.c39f.8530 to 0013.c39f.8537 1.9 4.2(3a) Ok 8 0016.c75a.a700 to 0016.c75a.a703 2.2 12.2(14r)S5 12.2(18)SXF1 Ok 9 0015.62e1.aee8 to 0015.62e1.aeeb 2.2 12.2(14r)S5 12.2(18)SXF1 Ok


Configuring ACE VLANs

– Create the necessary VLANs on the Cat6k.

– Group the VLANs into service line card VLAN groups.

– Assign the VLAN groups to individual ACE modules.

vlan 7,2001-2003, 3502,3504

svclc multiple-vlan-interfacessvclc module 1 vlan-group 1svclc vlan-group 1 7,2001,2002


Configuring ACE VLANs (cont’d)




vlan 7,2001-2003, 3502,3504



Configuring ACE VLANs (cont’d)




vlan 7,2001-2003, 3502,3504



Verify Cat6k Setup

Cat6k>show svclc vlan-groupDisplay vlan-groups created by both ACE module and FWSM commands

Group Created by vlans----- ---------- ----- 1 ACE 7, 2001-2002 2 FWSM 201-206,401-406,999-1000 3 ACE 2003

Cat6k>show svclc moduleModule Vlan-groups------ ----------- 01 1,3

Cat6k>show firewall moduleModule Vlan-groups------ ----------- 02 2,3

v2003 v2001v401

Group 3 Group 1Group 2


Accessing the ACE

Cat6k#session slot 1 processor 0

Connect to the ACE from IOS:

Processor “0” = Control Plane CPU for configuration

Processor “1” = NP1

Processor “2” = NP2


Creating ACE Contexts

bncmnace02/Admin# show vlanVlans configured on SUP for this module vlan7 vlan2001-2003

bncmnace02/Admin#configEnter configuration commands, one per line. End with CNTL/Z.bncmnace02/Admin(config)# context developmentbncmnace02/Admin(config-context)# allocate-interface vlan 7bncmnace02/Admin(config-context)# allocate-interface vlan 2001-2003bncmnace02/Admin(config-context)# exitbncmnace02/Admin(config)# exit

1. Create Context from within Admin context

2. Allocate Interfaces


Verifying ACE Setup

ACE-Module/Admin# show context development

Name: development , Id: 117Description:Resource-class: defaultVlans: Vlan7, Vlan2001-2003

ACE-Module/Admin# show runGenerating configuration....

context development allocate-interface vlan 7 allocate-interface vlan 2001-2003


Accessing ACE Contexts

bncmnace02/Admin# changeto developmentbncmnace02/development#

[Prompt shows ACE hostname and current context]

Access new context

From Admin Context

… or can Telnet/SSH direct to management interface of the relevant context (once it has been created)


Agenda







ACE Resource Management

Rates Memory

Per Context Control:

Resource levels for each context Support for oversubscription

– Bandwidth

– Data connections per sec.

– Management connections per sec.

– SSL bandwidth

– Syslogs per sec.

– Access lists

– Regular expressions

– Data connections

– Management connections

– SSL connections

– Xlates

– Sticky entries



Minimum Guarantee

Maximum Unlimited

Minimum Guarantee

Maximum Equal To Minimum



ACE-Module/Admin(config)# resource-class goldACE-Module/Admin(config-resource)# limit-resource all minimum 10% maximum unlimited

ACE-Module/Admin(config)# context development ACE-Module/Admin(config-context)# member gold

Context 1 MinimumContext 2 MinimumContext 3 MinimumContext 4 Minimum

Total ACE

resources

Oversubscribed Global Pool (unreserved resources)



ACE-Module/Admin# show resource allocation-----------------------------------------------------------Parameter Min Max Class-----------------------------------------------------------acl-memory 0.00% 100.00% default 20.00% 200.00% gold syslog buffer 0.00% 100.00% default 20.00% 200.00% gold ...

“default” resource class = 0% minimum, unlimited maximum

“gold” resource class = 10% minimum, unlimited maximum

Looking at the above figures, the gold class is applied to 2 contexts, meaning there is a 200% oversubscription

By default a context is a member of the “default” resource group


ACE Resource Management – gotcha’s

• Only allocate the minimum resources required/estimated initially (its hard to recoup resources later), and ensure you have a “reserve”

• Unlike other resources, sticky resources are not allocated by using the “all” keyword. Sticky resources must be allocated individually if required

resource-class gold limit-resource all minimum 20.00 maximum equal-to-min

limit-resource sticky minimum 20.00 maximum equal-to-min

• Bandwidth value is shown in “Bytes” (not Bits)bncmnace02/Admin# show resource usage Allocation Resource Current Peak Min Max Denied--------------------------------------------------------------------------

Context: development <snip> throughput 316 6125 0 500000000 0 <snip>

500,000,000Bps = 4Gbps


ACE Resources and Licence Upgrades

• ACE licence can be upgraded from 4-8-16Gbps, and SSL 1K, 5K and 15K SSL tps

• These ACE resources can be limited however a percentage figure is used, not an absolute amount

• This means the amount of resources allocated will vary depending upon the current licence

– 20% of the 4Gbps licence is 800Mbps, whereas 20% of the 8Gbps = 1.6Gbps– 10% of 1000 SSL tps = 100tps, whereas 10% of 5000 SSL tps = 500 tps

• When upgrading an ACE licence, the percentage figure in the resource-class does not change, therefore you must change the percentage allocated if you want the same amount of resources to be allocated to members of that resource-class after the upgrade


Agenda







ACE Redundancy

• Two ACEs form a Redundancy pair• Single FT VLAN required between ACEs (not one per context)• Redundant ACEs can be in the same, or different, Catalyst 6500 Chassis

• Each pair of contexts (on two distinct ACE modules) form a redundancy group, one being active and the other standby

• Both ACE modules can be active at the same time, processing traffic for different contexts, and backing-up each other (stateful redundancy)

Example:2 ACE modules4 FT groups4 Virtual Contexts (A,B,C,D)

ACE-1

ACE-2

FT VLAN

AActive

A’Standby

FTgroup 1

BActive

B’Standby

FTgroup 2

CActive

C’Standby

FTgroup 3

DActive

D’Standby

FTgroup 4


ACE Redundancy

• Fault-Tolerant (FT) VLAN (/30) carries FT packets, heart beats, config-sync packets, state replication packets

• Configuration synchronisation (bulk and incremental) & state replication is enabled by default

• SSL files (keys and cert’s) are not replicated

• Much like HSRP, each Context is assigned a priority, and the highest priority will become master (if pre-emption enabled)

• Normally recommend pre-emption is only used for operations (failing back to a recovered ACE)

• Possible to oversubscribe resources on both ACEs (active/active), however, a failure of one of the ACEs (or path to the ACE) will reduce capacity by half


Agenda







Key ACE terms

• North (Client) side & South (Server) side VLANs • Real Server – load-balanced servers• Serverfarm – a group of Real servers• Probe – keepalive to Real Servers• Predictor – load-balancing algorithm (e.g. round-

robin, least-connections etc)• VIP – Virtual IP Address. Typically NATed to the

address of the real-servers. Has no dependence on connected subnets.

• Route-Health Injection (RHI). ACE Module* can advertise the reachability of the VIP to the MSFC

* RHI not supported on ACE Appliance


ACE Key Terms

Server/South side VLAN

Client/North side VLAN

Real Server

Server Farm

RHI if VIP is Active


ACE Interface Configuration

• Think of the ACE as a Firewall– By default, traffic is not allowed “through” or “to” the ACE

• Access-list type “management” is required for traffic “to” the ACE

• IP access-list is required for traffic “through” the ACE

• N.B. Access-list type “ethertype” required in order to allow STP BPDUs (when ACE is in Bridged mode)


ACE Interface Configurationaccess-list nonip ethertype permit bpdu

access-list permit-all line 10 extended permit ip any any

interface vlan 2001

description Client_VLAN

bridge-group 1

access-group input nonip

access-group input permit-all

no shutdown

interface vlan 2002

description Server_VLAN

bridge-group 1

access-group input nonip

access-group input permit-all

no shutdown

interface bvi 1

ip address 10.1.1.4 255.255.255.0

no shutdown


ACE routes

• Routes are not shared between contexts• Each load-balancing context requires route(s) to servers AND a

route back to the client, before forwarding traffic• Admin context will typically only need management routes

• Within VDC each Context requires;– the default route will have a next-hop of the North-side VRF

HSRP address– Route to server subnets with next-hop of South-side HSRP

address– Management route(s)

ip route 0.0.0.0 0.0.0.0 10.80.199.109 default routeip route 10.80.202.0 255.255.255.192 10.80.199.94 route to rserversip route 10.80.196.0 255.255.254.0 10.80.193.3 management routeip route 147.149.163.128 255.255.255.128 10.80.193.3 management route


ACE Real Server Health Monitoring

- “Out-of-band” monitoring (Probes/Keepalives)- Probes can be used to

- Detect the loss of a real server- Monitor a gateway or other remote device for failover

purposes

- Optional port and ip-address probe configuration

- Multiple different native probe types including TCL

support

- Typically recommend a frequent simple probe (e.g. ping

every 5 seconds) combined with a less-frequent more

complicated probe (e.g. HTTP GET every 30 seconds). If

either probe fails, the server will be declared down


Rservers, ServerFarms, Predictors and Probes

probe icmp ping interval 5 passdetect interval 120 receive 5

probe tcp tcpprobe port 80 interval 30 open 5

probe http httpprobe port 81 interval 30 passdetect interval 300 request method get url /index.shtm expect status 200 299 open 5

rserver host server1 ip address 10.1.4.101 probe ping inservicerserver host server2 ip address 10.1.4.102 probe ping inservice

serverfarm host farm1 predictor leastconns probe tcpprobe rserver server1 inservice rserver server2 inservice

serverfarm host farm2 probe httpprobe rserver server1 81 inservice rserver server2 81 inservice


Agenda







TCP Connection

ServerClient

SYNSYN_ACK

ACK

DataACK

Data

More Data

ACK

FIN

ACK

ACK

FIN

Initialize

Use

Close


Agenda







ACE Load-Balancing Configuration

1. Create L3/L4 class map (define match criteria)2. Create load-balancing policy map (define actions

to perform)3. Create a multi-match policy map to tie the L3/L4

class-maps and policy maps together4. Activate the classification-action rules on either

an interface or “globally”

class-map C1 match <criteria>

class-map C1 match <criteria>

policy-map type loadbalance P1 <action>policy-map type loadbalance P1 <action>

interface vlanX service-policy input MMP1interface vlanX service-policy input MMP1

policy-map multi-match MMP1 match C1

policy P1 match C2

policy P2

policy-map multi-match MMP1 match C1

policy P1 match C2

policy P2


ACE Load-Balancing Configuration

L3/L4 Class-map defaults to “match-all”, which means only one VIP address is allowedbncmnace02/dev(config)# class-map fredbncmnace02/dev(config-cmap)# match virtual-address 1.1.1.1 tcp eq 80bncmnace02/dev(config-cmap)# match virtual-address 1.1.1.1 tcp eq 443Error: Only one match virtual-address is allowed in a match-all class-map and it cannot mix with any other match typebncmnace02/dev(config-cmap)#

“match-any” L3/L4 Class-map allows multiple VIP’s

class-map match-any fred 2 match virtual-address 1.1.1.1 tcp eq www 3 match virtual-address 1.1.1.1 tcp eq https


Layer 3 & Layer 4 Load-balancing

• L3 & L4 information is present in the first packet of the flow:

Source IP address

Destination IP address

IP Protocol

Protocol ports

• Load-balancing can be made on first packet of a flow


Layer 3/4 Flow Setup

SYN

SYN

Identifies VIP (matches class-map)Selects Server FarmMakes Load Balancing Decision


Layer 3/4 Flow Setup

SYN

SYN_ACK

ACK

Data

SYN

Identifies VIP (matches class-map)Selects Server FarmMakes Load Balancing Decision


Agenda







Layer 7* Flow Setup

• L7* load-balancing:

URL parsingCookie parsingGeneric HTTP header parsingSSL IDetc

* Layer 5 and above (SSL is Layer 5)

• Requires TCP termination and buffering of multiple packets before a LB decision can be made (this is why L7 load-balancing can never be as fast as L4 load-balancing)


Sniffer Trace of HTTP Connection

“Interesting” information only arrives in the 4th packet

GET /css/cavendish/template.css


Layer 7 Flow Setup (1/3)

SYN

SYN_ACKChooses seq# and replies w/ SYN_ACK



SYN

SYN_ACKChooses seq# and replies w/ SYN_ACK

ACKs data received from clientACK

ACK

Data (e.g. HTTP GET /)Starts buffering client packets



SYN

Buffers all packets …… until it has enough data for policy matchingElects serverfarm, makes balancing decisionSends previously buffered SYN to real server



SYN

Buffers all packets …… until it has enough data for policy matchingElects serverfarm, makes balancing decisionSends previously buffered SYN to real server

ACK

SYN_ACK

Does not forward SYN_ACK



Data

Empties buffer and sends data to server

ACK

Does not forward ACKStarts splicing the flows



Data (e.g. HTTP GET /)

Empties buffer and sends data to server

ACK

Does not forward ACKStarts splicing the flows

Data

ACK


Agenda







ACE test topology.100

.101 .102

VLAN 2004 10.1.2.0/24

11Port8110Port81

3/21

3/22 3/23

Sets Cookie serverid=server1


VLAN 2006 10.1.4.0/24

54:be

0d:17 ef:5e

VLAN 2001 10.1.1.0/24

VLAN 2002 10.1.1.0/24

Context “landing” b4:55

.4

.1

.5

.6

.1

bt-fwsm-ace

bt-customer


Agenda







ACE Layer 3 Policy

•Destination IP address of incoming packet must match the VIP address(es) in the class-map•Any protocol•Any port


ACE Layer 3 Policyrserver host server1

ip address 10.1.4.101inservice

rserver host server2ip address 10.1.4.102inservice

serverfarm host farm1 rserver server1 inservice rserver server2 inservice

class-map match-all classmap1 2 match virtual-address 10.1.1.100 any

policy-map type loadbalance first-match policy1 class class-default serverfarm farm1

policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1

interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdowninterface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all no shutdown

interface bvi 1 ip address 10.1.1.4 255.255.255.0 no shutdown

ip route 0.0.0.0 0.0.0.0 10.1.1.5ip route 10.1.4.0 255.255.255.0 10.1.1.6


ACE Layer 4 Policy

•Destination IP address of incoming packet must match the VIP address(es)•Protocol(s) must match•Port(s) must match


ACE Layer 4 Policyserverfarm host farm1 predictor leastconns probe tcpprobe rserver server1 81 inservice rserver server2 81 inservice

class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www

policy-map type loadbalance first-match policy1 class class-default serverfarm farm1

policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active



ip route 0.0.0.0 0.0.0.0 10.1.1.5ip route 10.1.4.0 255.255.255.0 10.1.1.6


ACE Layer 7 Policy

•Destination IP address of incoming packet must match the VIP address(es)•Protocol(s) must match•Port(s) must match•Layer 5-7 information (e.g. HTTP URL, Cookie, Header, SSL session ID etc) must match

Note: Regular expression matching is case-sensitive by default


ACE Layer 7 Policy

Typically used;• when traffic differentiation is required (e.g. *.jpg sent to farm of Cache Engines, everything else sent to the Web servers)• when traffic manipulation is required (e.g. Cookie insert, HTTP Header insert)

Performance is less than L3/L4 due to;• Delayed Binding• (Layer 7 ME required (depends on persistent rebalance))


ACE Layer 7 Policy

Layer 7 Class-maps & Policy-maps can be used to;

•Match on HTTP URL•Match on HTTP headers (cookie, language, host, browser, etc)•Match on string within HTTP payload (not header)•Insert/Delete/Modify HTTP headers (e.g. Insert ClientIP, rewrite URL etc)•Match RADIUS, RDP, RTSP and SIP fields•Generic TCP/UDP data parsing•Match on Source-IP address•Set IP QoS (DSCP) values•TCP Connection re-use

Layer 7 class-maps can use a match-all, match-any, or use nested class-maps (match A or B or [C & D])


ACE Layer 7 Policyserverfarm host farm1 serverfarm host caches rserver server1 transparent inservice rserver cache1 rserver server2 inservice inservice rserver cache2

inservice

class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq www

class-map type http loadbalance match-any checkforstatic 2 match http url .*\.jpg 3 match http url .*\.pdf

policy-map type loadbalance first-match policy1 class checkforstatic serverfarm caches class class-default serverfarm farm1

policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1



ip route 0.0.0.0 0.0.0.0 10.1.1.5ip route 10.1.4.0 255.255.255.0 10.1.1.6


Agenda







ACE Route Health Injection

•ACE can advertise the reachability of a VIP to the MSFC. If the VIP goes down, the route is withdrawn.•Appears as a /32 static route, with the next-hop of the ACE•Allows the MSFC to redistribute the route and advertise using routing protocol•VRF-aware•Default AD = 77

BNCMNSSW01#show ip route vrf bt-fwsm-aceRouting Table: bt-fwsm-ace<Snip>

10.0.0.0/8 is variably subnetted, 4 subnets, 2 masksC 10.1.2.0/24 is directly connected, Vlan2004C 10.1.1.0/24 is directly connected, Vlan2001D 10.1.4.0/24 [90/3072] via 10.1.1.6, 5d23h, Vlan2001S 10.1.1.100/32 [77/0] via 10.1.1.4, Vlan2001B* 0.0.0.0/0 [20/0] via 10.1.2.0 (bt-sc1-fusion), 7w0d


VDC ACE RHI

ACE RHI injects active VIPs into Firewall Block VRF

VRF redistributes static routes into EIGRP and advertises northwards


Agenda







Persistence Rebalance

• HTTP 1.0 requires a separate TCP connection for each HTTP request

• HTTP 1.1 supports persistent TCP connections, allowing pipelining of multiple HTTP requests within the same TCP connection

• Processing Layer 7 (within ACE HTTP ME) information is more

resource intensive than simply checking Layer 4 information

• By default, once the ACE has made a Layer 7 (check URL, Language etc) decision on the first packet of a flow, (which farm/server), all subsequent traffic will be sent to that server (“fast-switched”)

• “Persistence rebalance” disables this feature “Persistence” refers to a persistent TCP connection (multiple

pipelined HTTP requests) “Rebalance” refers to whether traffic should be re-balanced to

another serverfarm


Persistent Rebalance (cont’d)

• Only required if need to check (or manipulate) every HTTP packet within the same (persistent) TCP connection e.g.;– URL *.jpg & *.gif send to cache engines– HTTP Header “Language=French” send to French farm– HTTP Header Insert – insert information into EVERY HTTP

packet (rather than only the first one)

• Persistence rebalance is disabled by default on ACE (enabled by default on CSM)

• HTTP parameter-map required to modify behaviour


Agenda







ACE Connection Handling

• ACE can handle a maximum of 4 million concurrent connections

• Will continually monitor all connections to check whether the connection has closed, and resources can be freed and made available for new connections

• TCP is normally simple – watch for FIN or RST• Impossible to tell for UDP, or “broken” TCP

connections


ACE Connection Handling

ACE idle timers• TCP default = 1 hour• UDP default = 2 minutes• ICMP default = 2 seconds

DNS, RADIUS etc LB may need to reduce the timeout so the connection entry does not stay up unnecessarily

With default timers 33K DNS requests per second will utilise 100% of connections (within 2 minutes)

Use connection “parameter map” to change the settingValue = 0 to 4294967294 seconds (136 years )Set timeout to zero to disable the timeout (connection will stay up

for ever)


Agenda







ACE Stickiness

• Required when you need multiple sessions (concurrent or subsequent) from the same user to be sent to the same backend server.

• Many applications work by the Client initiating multiple connections e.g. HTTP sessions

• Without sticky, if ACE load-balances on round-robin, least-connections etc, then connections from the same client are likely to be sent to different servers

• If no sticky entry exists (e.g. first time a client connects), then the Predictor configured on the serverfarm is used to select which server to send the traffic to. At this point, a sticky table entry is created, and can then be used for subsequent connections (until the entry times out)


ACE Stickiness

ACE can stick on the following information;

It is important to understand the application and the client profile before deciding which method to use

N.B. Sticky resources are not allocated to a context by default (not included in the “resource all” designation), and need to be specifically assigned

* Requires ACE A2.x

RADIUS attributes*

RTSP Header*SIP Header*SSL Session ID*

Source/Dest IP addressLayer 4 Payload*HTTP Content*HTTP CookieHTTP Header


Source IP Stickiness

• Advantages– Simple to configure and troubleshoot

• Disadvantages– Proxy Servers in the path can present a single source IP

address (SNAT) for many clients. Result is all users are sent to the same rserver

– Mega Proxies can change the SNAT IP address mid-session



.101 .102

VLAN 2004 10.1.2.0/24

11Port8110Port81

3/21

3/22 3/23



VLAN 2006 10.1.4.0/24

54:be

0d:17 ef:5e

VLAN 2001 10.1.1.0/24

VLAN 2002 10.1.1.0/24


.4

.1

.5

.6

.1

bt-fwsm-ace

bt-customer


ACE Source-IP Stickiness

serverfarm host farm1 rserver server1 inservice rserver server2 inservice

sticky ip-netmask 255.255.255.0 address both group1 timeout 60 replicate sticky serverfarm farm1


policy-map type loadbalance first-match policy1 class class-default sticky-serverfarm group1

policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown



Cookie Stickiness

• Cookie can be – set by the rserver (which is learned by ACE)– set by the ACE (Cookie-insert)

• Cookie can be server-specific (sticky-serverfarm), or per-serverfarm (HTTP class-map)

• Advantages– Combats Proxy issues relating to source-IP stickiness

• Disadvantages– Only supported with HTTP– Client browser must support cookies



.101 .102

VLAN 2004 10.1.2.0/24

11Port8110Port81

3/21

3/22 3/23



VLAN 2006 10.1.4.0/24

54:be

0d:17 ef:5e

VLAN 2001 10.1.1.0/24

VLAN 2002 10.1.1.0/24


.4

.1

.5

.6

.1

bt-fwsm-ace

bt-customer


ACE Cookie Match

serverfarm host farm1

rserver server1

inservice

rserver server2

inservice

sticky http-cookie serverid cook_group

serverfarm farm1

class-map match-all classmap1

2 match virtual-address 10.1.1.100 tcp eq www

policy-map type loadbalance first-match policy1

class class-default

sticky-serverfarm cook_group

policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input mmp_ns1 no shutdown


Cookie name set by server


Agenda







ACE SNAT

• Source-NAT can be required for Client to Server, Server to Client, Server to Server

• NAT can be performed using either a pool of addresses, or statically with a one-to-one mapping (use where predictable IP is required)

• Within the policy-map you must configure which NAT pool number and which egress interface is to be used

Caveats;The ACE will *not* NAT bridged traffic. It must hit a load-balancing policy in order for SNAT to be implemented

SNAT to the VIP address requires ACE 2.x software


ACE SNAT

Dest IP = InternetSource IP = Web

Dest IP = InternetSource IP = ACE NAT

1

2

ACE requires LB policy in order to

“catch” traffic to NAT

e.g. RFC1918-addressed server requires connectivity to the Internet


ACE SNATrserver host gwnorth ip address 10.1.1.1 inservice

serverfarm host gateway_north_farm transparent rserver gwnorth inservice

class-map match-all SNAT-CLASS 2 match virtual-address 0.0.0.0 0.0.0.0 any

policy-map type loadbalance first-match SNAT-POL class class-default serverfarm gateway_north_farm

policy-map multi-match SLB-SNAT class SNAT-CLASS loadbalance vip inservice loadbalance policy SNAT-POL nat dynamic 1 vlan 2001

interface vlan 2001 description Client_VLAN bridge-group 1 access-group input nonip access-group input permit-all nat-pool 1 10.1.1.250 10.1.1.251 netmask 255.255.255.0 pat no shutdown

interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all service-policy input SLB-NAT no shutdown



ACE Server to Server SNAT

Some applications require server to server load-balancing. For example, load-balanced Web server to Application server traffic

Some topologies (e.g. VDC) require extra configuration in order to ensure server-to-server load-balancing occurs correctly


Server to Server without SNAT (1/2)

1. Web Server initiates traffic to Application VIP

2. ACE load-balances traffic to Application server B

By default source-IP is maintained

Application Application

Dest IP = VIPSource IP = Web

Dest IP = App BSource IP = Web1 2

A B


Server to Server without SNAT (2/2)

3. App Server replies to Web IP

4. MSFC routes to directly-connected subnet

5. Web Server sends TCP RST since the source IP (and SEQ info) does not match any open sessions


Dest IP = WebSource IP = App B

Dest IP = WebSource IP = App B

43

Server sends TCP RST5

A B


Server to Server with SNAT (1/2)

1. Web Server initiates traffic to Application VIP

2. ACE load-balances traffic to Application server B

•ACE configured to change source IP to a SNAT IP


Dest IP = VIPSource IP = Web

Dest IP = App BSource IP = ACE SNAT1 2

A B


Server to Server with SNAT (2/2)

3. App Server replies to ACE SNAT IP

•MSFC routes to ACE

4. ACE changes he Source and Destination IP back to the VIP and Web, and traffic routed correctly


Dest IP = WebSource IP = VIP

Dest IP = ACE SNAT

Source IP = App B4

3

A B


ACE Server to Server LBserverfarm host farm1 predictor leastconns rserver server1 inservice rserver server2 inservice


policy-map type loadbalance first-match policy1 class class-default serverfarm farm1policy-map multi-match mmp_ss1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active nat dynamic 1 vlan 2002

interface vlan 2002 description Server_VLAN bridge-group 1 access-group input nonip access-group input permit-all nat-pool 1 10.1.1.254 10.1.1.254 netmask 255.255.255.255 pat service-policy input mmp_ss1 no shutdown



ACE SSL

• What is SSL• Why terminate SSL on ACE• SSL Termination• Certificate Chains


What is “SSL”

• Secure Sockets Layer• Layer 5 protocol – above TCP and below Applications,

such as HTTP, FTP etc


ACE SSL components

• SSL Server Certificate• SSL key pair (private key and public key)• Optional – Certificate Chain


Without SSL Accelerators

• Server terminates SSL session• Certificates and keys are held on the server• Load-balancer can only act at Layers 3-5, since the layers above

are encrypted (cannot see URL or Cookie)

Client ServerSLB

HTTPS HTTPS


Benefits of SSL Accelerator

• Manageability - One cert vs Many (cost, operations effort)

• Troubleshooting – can “sniff” HTTP layer• Stickiness – can see HTTP Cookies• Performance/Scalability

Client ServerSLB

HTTPS HTTP


SSL Certificates

Public Key

Private Key

CertificateSigning Request

Public KEYCommon nameDomain name

LocationE-mail

CertificateSigning Request

Public KEYCommon nameDomain name

LocationE-mail

ValidationProcess

Server Public Key

Server Public Key


Company Docs

KEY Pair

Server Private Key

Certificate Authority

SSL Server

CertificateCertificate


SSL Fundamentals: Key Exchange Packet Flow Overview

Server Public Key

RandomNumber

Generator

RSA Encrypt

“Shared” Secret Key

“Shared” SecretEncrypt & Decrypt

Client Browser

Server Public Key

RSA Encrypt

“Shared” Secret Key

“Shared” SecretEncrypt & Decrypt

Server

Private Key

Data

Data Data

Public Key

Client “Hello”

Server “Hello”

SAasdfkjw1340+jakjb//alkjt


“Data Exchange”


“Key Exchange”

Data

Certificate


ACE SSL Termination

ACE SSL configuration is MUCH simpler (single termination point) than CSM/SSLM

The ACE requires the following in order to terminate SSL connections

ACE/context(config) # show crypto filesFilename File File Expor Key/ Size Type table Cert----------------------------------------------------mycert.pem 1275 PEM No CERTmykey.pem 283 PEM Yes KEY

•SSL Server Key-pair (Private and Public Key)•SSL Server Certificate•Optionally – SSL Certificate Authority Certificate Chain


SSL Termination

ACE

EncryptedUnencrypted

parameter-map type ssl sslparam

cipher RSA_WITH_3DES_EDE_CBC_SHA

cipher RSA_WITH_AES_128_CBC_SHA

cipher RSA_WITH_AES_256_CBC_SHA

!

ssl-proxy service sslproxy

key mykey.pem

cert mycert.pem

ssl advanced-options sslparam

!

serverfarm host farm1

rserver server1 81

inservice

rserver server2 81

inservice

class-map match-all classmap1 2 match virtual-address 10.1.1.100 tcp eq https !policy-map type loadbalance first-match policy1 class class-default serverfarm farm1!policy-map multi-match mmp_ns1 class classmap1 loadbalance vip inservice loadbalance policy policy1 loadbalance vip icmp-reply active loadbalance vip advertise active ssl-proxy server sslproxy


SSL Certificate Chains

• Optional• Typically required when the Certificate Authority that has signed

the Server Certificate is not trusted by the Client• ACE will send the complete certificate chain, and the client will

check each certificate in turn to see if it trusts the signer (CA)

crypto chaingroup InternalCAcerts cert rootCA.pem cert ouCA.pem cert deptCA.pem

ssl-proxy service secure_access key mykey.pem cert mycert.pem chaingroup InternalCAcerts


ACE Configuration spreadsheetNew Context

Layer 3 LB

Layer 4 LB

Layer 7 LB

L4 SSL L7 SSL SNAT

Resource-Class x

Context Name x

FT Group x

BVI x x x x x x x

Routing x

Parameter-map x x x

Crypto chaingroup x x

Ssl proxy service x x

Probe x x x x x x

Rserver x x x x x x

Server farm x x x x x x

Class-map Match-all virtual-address (L3/4)

x x x x x x

http loadbalance (L7) x x

Policy-map Type loadbalance x x x x x x

multimatch x x x x x x

Access-list x

WorksheetRequirement

For stickiness, apply a sticky-serverfarm to the LB policy-map, and apply the serverfarm to the sticky-group


ACE Documentation

• Cisco ACE Documentation http://www.cisco.com/en/US/partner/products/ps6906/tsd_products_support_model_home.html

• ACE Design Guidelines coming soon..

• How to use the ACE Packet Capture featurehttp://livelink.intra.bt.com/livelink/livelink.exe?func=ll&objId=70435818&objAction=browse&sort=name&viewType=1


Questions?

Documents

Cisco ACE Training