GEC16: OpenFlow Switches in GENI

Sponsored by the National Science Foundation

GEC16: OpenFlow Switches in GENI

Marshall Brinn, GPO March 21, 2013

Sponsored by the National Science Foundation 2January, 2013

Outline

• Nick Bastin, Big Switch– Introduction to Hardware Switch Architectures

• Marshall Brinn, GPO– Network Slicing and Programming with VLAN’s and

OpenFlow


NICK BASTIN<*** NICK’s TITLE ***>

Sponsored by the National Science Foundation

GENI: Network Slicing and Programming

with VLANs and OpenFlow

Marshall Brinn, GPOMarch 21, 2013


Introduction

• GENI has focused on specifying requirements on Aggregates for resource allocation through the Aggregate Manager (AM) API

• But there are GENI requirements about network slicing and programmability that aren’t specified in the AM API. – Specifically, how to support the common (though not universal)

use case of network management with VLAN’s and OpenFlow • These slides propose a set of standards for GENI

aggregates with respect to network slicing and programmability using VLANs and using OpenFlow– And describe some simple examples and possible engineering

approaches.

By establishing these standards, we can then assess existing and developing aggregates to make the experimenter experience more uniform and reliable over time.


Slicing and Programming the Network in GENI

• GENI network slicing will be done by VLAN tags– Why? The simplest, standard way to partition L2 traffic

• GENI network programming may be done by OpenFlow– Note: It isn’t a requirement that GENI aggregates use

OpenFlow for network programming.– But if they DO use OpenFlow, we would like there to be

common conventions for that use, particularly wrt. slicing the network by VLAN tags

– We are particularly considering the case of GENI racks, which we expect will use OpenFlow to provide network programmability

We aren’t saying one couldn’t also slice or program a network in other ways. But these slides focus on the case of using OF to program the VLAN-sliced network.


Preliminaries

• GENI networking operates on two distinct planes:– The Control/Management Plane (L3+) for:

• Traffic between tools and aggregates (AM API)• Intra-aggregate control traffic• Extra-aggregate control traffic (to GMOC, CH)• OpenFlow Control traffic• SSH to log into resources

– The Data Plane (L2) for experimenter traffic• Each slice has one or more VLAN’s uniquely assigned to it.• Slice traffic is VLAN tagged and (therefore) segregated across

slices• [Note: Some deployments will require sharing VLAN’s across slices]

An Aggregate should provide two different network interfaces to support and segregate these two different kinds of network traffic.


GENI OpenFlow Networking: The cast of characters

• Switch: The point of network ingress/egress for an aggregate [Ignore, for now, any aggregate-internal switches]

• Controller: Experimenter-provided OF Controller• Proxy-Controller: Managing interface between

Switch and Controller [Think: FlowVisor or similar]

• Host: Network-addressable ‘edge node’ compute resource in a topology

Obviously, a given topology may have many instances of these, configured in arbitrary ways. But these are the Lego-pieces from which we build a sliced, stitched, programmable network topology.


The Simple Case

ExperimenterController (VLAN=v)

Proxy-Controller

Switch Host

1) A packet comes into the switch.

2) IF the packet doesn’t match any current switch flow rules, it passes the packet to the Proxy Controller.

3) IF the packet is associated with an experimenter-provided Controller (based on VLAN of packet and slice), the packet is dispatched to the experimenter Controller.

4) The Controller may drop the packet, or pass back a modified packet, or propose flow rules to install in Switch.

5) The Proxy Controller may allow the packets/rules to flow to the switch, or may filter or modify them to protect the segregation of slice traffic.

6) The packet is (possibly) passed along to host.


But things aren’t always so simple…

• Different classes of OF switches• VLAN translation• Special topologies require special tagging and

control


Three Classes of OF Switches

Pure OF Switch Port Hybrid Switch VLAN Hybrid Switch

OF Granularity Each port is OF enabled

Some ports are OF enabled, some aren’t

Some VLAN’s are OF enabled, some aren’t

DPID’s Single DPID for entire switch

Single DPID for all OF-enabled ports

One per VLAN

Controllers One (proxy-) controller for entire switch

One (proxy-) controller for all OF-enabled ports

One (proxy-) controller per DPIDBut could use same (proxy-) controller for multiple DPID’s

Traffic to Controller

VLAN-tagged VLAN-tagged Not VLAN-tagged

Proxy-Controller Discriminant

Dispatch by VLAN-tag

Dispatch by VLAN-tag

Dispatch by DPID

Think of the Port Hybrid as two switches: An OF switch with fewer ports, and a non-OF switch for the rest of the ports.

To handle the general set of switches, Slices and Experimenter controllers must be tagged by a unique VLAN/DPID tuple.


Switch: Description and Requirements

• There may be one or more outward-facing (linked to resources and networks outside the aggregate) ports on the switch– As well as one or more inward-facing ports (linked to aggregate

resources)• OF-enabled Switches must provide an OpenFlow datapath

(DPID) or multiple OF DPID’s– Supporting OF V1.0

Not every Switch must be OF-enabled (on all or any ports). But consider those Switches that are OF-enabled.


Switch: Description and Requirements [2]

• The Switch should support VLAN translation– To translate external VLAN tags to aggregate-internal VLAN tags

as needed.• Why?

– Traffic that never reaches ION or another translation service (e.g. traffic between two campuses of the same regional, or traffic between two aggregates on the same campus) have no default VLAN translation mechanism

– Making stitching a manual and less-likely prospect.

[Note: We recognize that some campuses may connect to GENI in other ways that will require special engineering (e.g. tunneling).]

This is a key enabler of GENI scalability and new racks must provide this capability


Controller: Description and Requirements

• The Controller may create any flow entry or packet – But only flow entries and packets for VLAN’s owned by

the slice associated with the controller will be forwarded to the switch by the proxy-controller

– That is, the controller can only program traffic for the DPID(s) or VLAN(s) of the associated slice

• Traffic reaching the controller will be tagged with a sliver-unique ‘discriminant’: either VLAN or DPID (or both)– Depending on the slice topology and switch configuration


Proxy-Controller: Description and Requirements

• The Proxy-Controller performs several functions:– Multiplexes multiple experimenter controllers, based on

VLAN– Distributes OF messages (including packets) from

switches to experimenter controllers based on discriminant [VLAN, DPID]

– Monitors and filters data from experimenter controllers to OF switch• Making sure packet VLAN is properly set for slice traffic• Adding VLAN match criteria on any flow entries provided by

experimenter controller

Note: I intentionally avoid specifying FlowVisor here. While it is a perfectly acceptable implementations of the Proxy-Controller, an aggregate can implement these requirements as it chooses.


Proxy-Controller: Description and Requirements [2]

• For slices for which no controller is supplied, Proxy-Controller operates as standard L2 learning switch• Learning port MAC mapping for nodes on that VLAN by

flooding/remembering when an unknown MAC destination is encountered– Writing this mapping into OF switch

• An experimenter should not create a topology with a loop without providing a controller– Though the Proxy-controller could use spanning tree algorithms to

detect and avoid bad consequences.

Note: The Proxy-Controller is not necessarily an Aggregate Manager and doesn’t need to speak the AM API. It is the job of an aggregate (be it FOAM or the ‘compute resource’ aggregate) to inform the Proxy-Controller about new flow space requirements.


Proxy-Controller: Example Operations

Controller (VLAN=v)

Proxy-Controller

Switch

Flow Entries provided by Controller have VLAN entries added to match clauses

{Match: DEST=a, Action: out=p}

{Match: DEST=a, VLAN=v Action: out=p}

Controller (VLAN=v)

Proxy-Controller

Switch

Flow Entries tagged with wrong VLAN dropped

{Match: DEST=a, VLAN=w Action: out=p}

Controller (VLAN=v)

Proxy-Controller

Switch

Packets tagged with wrong VLAN dropped

{SRC=s, VLAN=w}


Proxy-Controller: Example Operations

Controller (VLAN=v)

Proxy-Controller

Switch

Unmatched packets dispatched to Controller by VLAN

{VLAN=v, SRC=s, DST=d, …|}

{VLAN=v, SRC=s, DST=d, …|}

Proxy-Controller

Switch

No Controller: Act as L2 learning switch

Receive unknown packet, flood and learn PORT MAC rules

Controller (DPID=d)

Proxy-Controller

VLAN Hybrid Switch

Unmatched packets dispatched to Controller by DPID

{DPID=d, SRC=s, DST=d, …|}

{DPID=d, SRC=s, DST=d, …|}


VLAN Hybrid Switches and Controllers

• In the case of VLAN Hybrid Switches, there are many individual DPID’s provided and each can be associated with a controller.

• It is still desirable to interpose a proxy-controller between the controller and the switch:– To protect against controllers that don’t reliably drop or

fix improper VLAN tagging on packets or flows– To protect against unreliable switch firmware


Ports/VLANs/DPIDs are the Unique Tuple

• In the general case, OF rules discriminate traffic on the basis of a unique [PORT, VLAN-tag, DPID] tuple – There are potentially multiple ingress/egress ports on a switch

(especially beyond edge nodes, at backbones or regionals)– There are potentially multiple paths for L2 traffic between two edge

nodes– There are potentially multiple VLAN’s per slice spanning multiple

aggregates• Consider the case of three switches connected in a

triangular topology:S1

S2 S3

Traffic from a node on S1 to a node on S3 cannot be uniquely specified by a VLAN, nor by an output port, but by the pairing of the two


Some Engineering Details:An interesting example

GA Tech

SOX (OF)

U. FLA Clemson

Juniper(non-OF)

A controller managing SOX switch MUST write VLAN-tagged packets:• Juniper switch is invisible to GENI

(not in stitching manifest). • SOX indicates that it has traffic

going out same port but different VLAN’s.

VLAN=6VLAN=7

Port=1, VLAN=6 Port=1, VLAN=7

VLAN=100


Some Engineering Details: Stitching

• From the AGG’s perspective, the act of “creating a stitch” is precisely the act of establishing VLAN translation between external VLAN tags/ports and internal VLAN tags/ports

Agg 1

Switch 1

Agg 2

Switch 2

Topology with VLAN=v1 Topology

with VLAN=v2

Extra-aggregate traffic on VLAN=v0

Switch 0 Switch Rule “Map V0=>V2 incoming, V2=>V0 outgoing” is the stitch

Switch Rule “Map V0=>V1 incoming, V1=>V0 outgoing” is the stitch


Stitching to non-GENI Campus resources

• This same approach to stitching allows aggregates to stitch non-GENI campus resources into a given slice.– Administrators arrange for VLAN-tagged traffic to appear on a

particular port of aggregate switch• Avoiding conflicts on a shared VLAN is a human activity.

– The aggregate maps this traffic into the slice topology

CampusResource Agg 2

Switch 2

Topology with VLAN=v2

Extra-aggregate traffic on VLAN=v0Switch Rule “Map V0=>V3 incoming, V3=>V0 outgoing” is the stitch


Summary

• The different kinds of OpenFlow switches (pure, VLAN-hybrid, PORT-hybrid) have different semantics and require different handling

• In the general case, OpenFlow controllers need to manage a unique tuple of [PORT, DPID, VLAN] to manage (route, distinguish) traffic

• The Proxy-Controller must, in addition to filtering improper rules and packets, add VLAN, DPID or PORT match criteria to controller-provided rules.

• There are configurations for which a GENI aggregate must perform VLAN translation (or fail to stitch)

The main ‘take away’ points from this brief which we’d like your help refining.


Conclusion

• These slides try to lay out some principles for providing network programmability and slicing using OpenFlow and VLAN tags

• I hope that over time we can flesh these out to be more correct and complete

• Then I expect we can use these to assess current and developing aggregates in terms of the OpenFlow network programmability capability they may provide

Documents

GEC16: OpenFlow Switches in GENI