Cisco - OSPF Routing Protocol - MIK Routing Protocol(1).pdf · OSPF Routing Protocol Contents Introduction Network Architecture Campus Design Architecture Building Block Design Server

OSPF Routing Protocol

ContentsIntroductionNetwork Architecture

Campus Design ArchitectureBuilding Block DesignServer Farm DesignCore Block DesignWAN Design Architecture

Protocol Design

Campus Design ConsiderationsLayer 2 versus Layer 3 Core DesignWAN Design ConsiderationsDesign Recommendation Summary

Planning and Implementation

IP AddressingSummarizationScalabilityMigrationConvergence TuningDesign Case Study

Operating the Solution

Operation VerificationTroubleshooting

Related Information

IntroductionOpen Shortest Path First (OSPF) is an interior gateway protocol (IGP) link state protocol. Contrary to the distancevector protocol in which the actual Internet Protocol (IP) network is advertised periodically, in a link state protocolthere is no IP route exchange. Every participant router creates a Link State Advertisement (LSA) describing itslocal interface (IP address, network mask, reachable neighbor, link type, and so on) and places it in its database.LSAs are distributed through reliable flooding during database synchronization, and the collection of all LSAsconstitute a link-state database.

Cisco - OSPF Routing Protocol

http://www.cisco.com/partner/sdm/ci/routing/ospf/CI_OSPF.html (1 of 48) [10/11/2001 5:34:43 PM]

http://www.cisco.com/pcgi-bin/imagemap/navbar

http://www.cisco.com/partner/sdm/ci/routing/ospf/CI_OSPF.pdf

All routers within an area have the exact same link state database and run in parallel with the shortest path orDijkstra algorithm. Each router constructs a tree of shortest path with itself as a root. The shortest path tree givesthe route to all destinations within the autonomous system.

Compared to distance vector protocols that have a flat architecture, OSPF uses a hierarchical architecture. Byhaving a hierarchical design, routing control packets in the domain are decreased and limited to a given area. Inaddition, summarization between different hierarchical levels significantly increases the stability of the networkand decreases the size of the routing table.

OSPF allows a network to be segmented into multiple areas. An area is a collection of routers and networks. Allareas are attached, physically or logically, to a common area called the backbone area (area 0). Routing betweenareas is achieved through area 0, and summarization occurs at Area Border Routers (ABRs) that are attached to thebackbone area 0 and another non-backbone area.

The three components in OSPF include:

Neighbor discovery●

Database synchronization●

Shortest Path First (SPF) calculation●

Router Classifications

There are four types of routing nodes in OSPF. Each routing node provides a specific function.

Internal router—A router that has all its interfaces in a given area●

ABR (Area Border Router)—A router that has active interfaces in at least two areas, one being the backbonearea 0

●

ASBR (Autonomous System Border Router)—A router that injects external routes into the OSPF domain byredistributing any routing protocol or external route to the OSPF domain is known as an ASBR

●

Backbone router—A router that has an interface to the backbone area 0, this can be an ABR or backboneinternal router

●

Area Types

There are four types of areas in OSPF. Each area provides a specific function.

Area 0—The backbone area having the specific function of connecting all areas together and passinginformation between areas.

●

Transit area—Any area including area 0, having external routing capability. In other words, type 5 LSA willbe flooded into such an area. A transit area can also exist to provide a virtual link between an area notphysically connected to backbone area 0.

●

Stub area—An area that does not have external routing capability, hence type 5 LSAs are not flooded intothis area. ASBR cannot be placed inside this area and a virtual link cannot be configured through this area.

●



Not So Stubby Area (NSSA)—NSSA has the same capability as a stub area in that type 5 LSA are notflooded into this area and a virtual link cannot be configured through this area. However, an ASBR could beplaced inside such an area and external routes could be imported into the NSSA area and flooded furtherinto OSPF domain.

●

Control Packets

OSPF runs on top of IP and is assigned protocol 89. OSPF control packets have a 24-byte common header. Toguarantee neighbor discovery and maintenance and database synchronization, the following packet types aredefined within the Type field of an OSPF control packet.

PacketType Description

1 Hello

2 Database Description

3 Link State Request

4 Link State Update

5 Link State Acknowledgment

Packet Type 1

Hello packets are used to establish and guarantee neighbor discovery and maintenance.

Packet Type 2

Database Description packets are used in the initial database synchronization. In order to check what instance oftheir database needs to be exchanged, routers exchange a summary of their database (LSA header) and mark anymissing LSA or a newer instance in order to request it through the Link State Request packet.

Packet Type 3

During a database description exchange, the routers request their missing LSA. If they need a more recent instantof an LSA, they add these LSAs in the Link State Request list. Once the database description exchange iscomplete, the routers send Link State Request packets in order to request these LSAs.

Packet Type 4

A router replies to the Link State Request packet by sending a Link State Update. Link State Update is also usedwhen the routers are in Full state and there is a need to generate a new LSA due to any changes.

Packet type 5

The flooding operation should be reliable in order to guarantee that no information was lost while synchronizingthe database. Link State Acknowledgment is sent in reply to a Link State Update packet.



LSA Types and Definitions

Link State Advertisement is the information generated by every router describing its local interface in an area.Depending on the LSA type, a router might generate additional information such as a reachable network outside anarea or OSPF domain.

The following table lists the 11 LSA types.

LSA Type Description

1 Router LSA

2 Network LSA

3 Summary LSA

4 Summary LSA

5 External LSA

6 MOSPF LSA

7 NSSA LSA

8 External Attribute LSA

9 Opaque LSA

10 Opaque LSA

11 Opaque LSA

LSA Type 1

LSA Type 1 is a router LSA that is generated by every router into a given area. Routers attached to multiple areasgenerate this LSA into each attached area. A Router LSA describes the router's interface for a given area. Itcontains information such as IP address, network mask, remote neighbor, link type, link cost, and so on. This LSAis flooded within each area.

LSA Type 2

LSA Type 2 is a network LSA. In order to better understand the use of this LSA, you should first understand howOSPF considers multi-access networks.

A network is said to be multi-access if it can have more than two routers attached to it. Depending on broadcastcapability, this is further divided into two types of networks.

Broadcast network—Has the broadcast capability for example Ethernet●

Non Broadcast MultiAccess (NBMA)—Does not have the broadcast capability for example AsynchronousTransfer Mode Permanent Virtual Circuit (ATM PVC)

●

SPF needs to consider a network as a collection of nodes and point-to-point links.

To satisfy the SPF requirement, consider the media itself as a node (Pseudonode) that is represented by one of therouters on this media called Designated Router (DR). Thus, the adjacency between routers attached to themulti-access is the adjacency between every attached router and the Pseudonode.



All attached routers advertise a link to the Pseudonode, represented by DR in their Router LSA. An advertisementis needed from Pseudonode to all attached routers: This is performed by LSA Type 2.

LSA type 2 is generated by DR on behalf of the network and announces all routers attached to the multi-accessnetwork (also referred to as transit network). This LSA is flooded within an area.

LSA Type 3

Summary type 3 announces the IP destination outside a given area in order to ensure that an ABR does thefollowing:

Summarize intra-area route to the backbone●

Summarize intra-area and inter-area route (learned through the backbone) into non-backbone area●

Note that routing between areas has a distance vector behavior. This means that the route learned from an area, andinstalled in the routing table, is summarized for other attached areas. It is not advertised back to the same area.

A backbone router processes only the summary received from the backbone. The only time an ABR processes asummary received through a non-backbone area is:

If the ABR loses its connection to the backbone (no neighbor) but still is an ABR and has an active interfacein area 0

●

If there is a virtual link in the TransitArea and the TransitCapability of the area is set to true (see later)●

LSA Type 4

When a type 5 LSA is flooded within a domain, the location of the ASBR (advertising router) is only knownwithin the area in which the type 5 LSA is flooded. It is the responsibility of the ABR attached to this area tosummarize the reachability of the ASBR to other areas. The ASBR in normal operations is the exit point towardexternal destinations.

Note that type 5 and type 4 LSAs are not flooded into Stub or NSSA areas and there is no need to announce ASBRreachability within Stub or NSSA areas.

LSA Type 5

An External LSA is generated by an ASBR when some external destination is redistributed into OSPF. A routerbecomes ASBR as soon as it redistributes external routes learned by way of any protocol, static, or connected intoOSPF. Type 5 LSAs are flooded domain-wide into all areas except Stub areas and NSSA areas.

LSA Type 6

The group membership LSA (MOSPF) is not used by Cisco. This LSA is used in order to create a shortest pathtree for every source or multicast destination.

LSA Type 7

LSA type 7 is generated by an ASBR in an NSSA area. This allows external routes to be imported into an NSSAarea as Type 7.

Type 7 is flooded only within the NSSA area. In order to flood the external information further, an ABR couldtranslate type 7 LSAs into type 5 LSAs and further flood into the OSPF domain.



LSA Type 8

This is an obsolete LSA. It was previously intended to carry external attributes between border routers through theOSPF domain.

LSA Types 9, 10, and 11

Opaque LSA defines a range of LSAs that provide more flexibility to the OSPF LSA packet format by not havingto define a new LSA type for every new application. There are three flooding scopes depending on the type:

Type 9 is flooded locally (not beyond the local network)●

Type 10 is flooded area wide●

Type 11 is flooded domain wide●

Currently, type 10 is used for MPLS Traffic Engineering in order to announce more attributes for network links.

Neighbor Discovery

When OSPF is enabled on an interface, the router starts sending Hello packets periodically in order to discover theremote neighbor(s). The discovery could be dynamic, or a configuration might be needed in the case of amulti-access network that lacks broadcast capability.

When a router receives a Hello from its neighbor, it includes the Router ID of this neighbor in its next Hello. Thisensures that there is two-way connectivity between routers.

A Hello packet is also used to elect DR/BDR on a multi-access network (broadcast and NBMA) and make surethat the neighbors agree on the area type based on the option fields in the Hello packet.

Hello packets behave differently depending on the network type as described below.

Point-to-point Network

Neighbor discovery is dynamic●

Hello is sent to the multicast address AllSPFRouter (224.0.0.5)●

No DR/BDR election●

Broadcast Network

Neighbor discovery is dynamic●


DR/BDR election takes place●

NBMA Network

A configuration is required in order to discover the neighbor●



Hello is sent unicast to each remote neighbor's IP address●

DR/BDR election takes place●

Note that since DR/BDR takes place, you would need at least a connection from DR/BDR to all the neighbors inorder to guarantee the proper operation in this mode.

DR/BDR sends Hello to all routers●

A router is eligible to become DR/BDR (priority different than 0) and it sends Hello packets only toDR/BDR and those routers that are eligible to become DR/BDR

●

A router not eligible to become DR/BDR (priority 0) sends Hello packets only to DR/BDR●

Point-to-Multipoint Network

Point-to-multipoint is considered a collection of point-to-point networks. Depending on the configuration,neighbor discovery could be dynamic.

ip ospf network point-to-pointNeighbor discovery is dynamic●


No DR/BDR election●

In this case the Hello is sent to the multicast address AllSPFRouter and replicates the packet over each connection.It is important to note that the connection should be permanent (PVC) and not dynamic (SVC) as this would fail.

ip ospf network point-to-multipoint non-broadcastA configuration is required in order to discover the neighbor●

Hello is unicast to each remote neighbor IP address●

No DR/ BDR election●

The RFC definition of point-to-multipoint requires configuration and that neighbor discovery is not dynamic.Cisco implements both methods.

Database Synchronization

It is very important that the database of all routers within an area are synchronized in order so they have the sameview of the network. Synchronization is either the initial router synchronization (when two routers establishadjacency) or if there is any change to the network topology and the routers need to resynchronized.Synchronization is performed by way of OSPF packet type 2, 3, 4, and 5.

Once two routers achieve the 2-way state, they start synchronizing their database by establishing a master-slaverelationship. This is necessary so that one of the routers leads the database synchronization exchange.

After the 2-way state, the routers go to the Exstart state where they have to find out who is the master. This is done



by Router ID. Initially both routers declare themselves as master but the router with the higher Router ID is electedas master.

After the election, they go to the Exchange state at which point they start exchanging Database Descriptionpackets. The master sends a Database Description packet by incrementing the LS sequence for every differentpacket, and the slave just echoes back the LS sequence number sent by the master. Only one Database Descriptionpacket is sent and echoed at a time. This guarantees a reliable exchange between the two routers.

During the exchange phase the routers make notes of their missing LSA or if the LSA that the neighbor has has anewer instant. Once the exchange is over, they are going to make a request (packet type 3) from their neighbor andthey go to the Loading state. Once the loading is over, or if the routers do not have any LSA to request (link staterequest list empty), the routers go to Full adjacency.

Note that in a multi-access network (broadcast, NBMA) where DR/BDR election takes place, only DR/BDRsynchronize their databases with all the routers. In other words, two non DR/BDR routers remain in 2-way/DOTHER state.

Neighbor State Machine

The following diagram shows the neighbor state machine.

The following describes the states of the OSPF neighbors.



Down—The initial state of a neighbor conversation. It indicates that there has been no recent information receivedfrom the neighbor. On NBMA networks, Hello packets may still be sent to "Down" neighbors, although at areduced frequency.

Attempt—Is only valid for neighbors attached to Nonbroadcast Multiaccess (NBMA) networks. It indicates that norecent information has been received from the neighbor, but that a more concerted effort should be made to contactthe neighbor. This is done by sending the neighbor Hello packets at intervals of set by the Hello interval.

Init—A Hello packet has recently been seen from the neighbor. However, bi-directional communication has notyet been established with the neighbor (i.e., the router itself did not appear in the neighbor's Hello packet). Allneighbors in this state (or higher) are listed in the Hello packets sent from the associated interface.

2-Way—Communication between the two routers is bi-directional. This has been assured by the operation of theHello Protocol. This is the most advanced state short of beginning adjacency establishment. The (Backup)Designated Router is selected from the set of neighbors in state 2-Way state or greater.

ExStart—The first step in creating an adjacency between the two neighboring routers. The goal of this step is todecide which router is the master, and to decide upon the initial DD sequence number. Neighbor conversations inthis state or greater are called adjacencies.

Exchange—The router is describing its entire link state database by sending Database Description packets to theneighbor. Each Database Description Packet has a DD sequence number, and is explicitly acknowledged. Only oneDatabase Description Packet is allowed outstanding at any one time. In this state, Link State Request Packets mayalso be sent asking for the neighbor's more recent LSAs. All adjacencies in Exchange state or greater are used bythe flooding procedure. In fact, these adjacencies are fully capable of transmitting and receiving all types of OSPFrouting protocol packets.

Loading—Link State Request packets are sent to the neighbor asking for the more recent LSAs that have beendiscovered (but not yet received) in the Exchange state.

Full—The neighboring routers are fully adjacent. These adjacencies now appear in router LSAs and networkLSAs.

Shortest Path First Calculation

After database synchronization, all routers in an area will have the exact same link state database. The database isjust a collection of different LSAs that the router has received and should build up the routing table based on thisinformation.

There are two types of routes:

Network route—A destination IP address●

Router route—The path (outgoing interface and next hop) to ABR or ASBR●

This will be used later in order to consider the inter-area route advertised by the ABR and the external routeadvertised by ASBR. In order to see the router route, use the show ip ospf border-routers command.

SPF needs to see the network as a collection of nodes and point-to-point links. A multi-access network isrepresented by a Pseudonode (DR). Every router announces a connection to the transit network (DR) and the DRannounces a connection to all attached routers.

A router first finds out the path (outgoing interface and next hop) to all the nodes in its area by running the Dijkstraalgorithm. The IP address is just additional information that is part of the node (included in the router LSA). Once



the path to all the nodes is found (outgoing interface and next hop), the path to all the IP addresses advertised bythe node is calculated.

The following describes the steps a router goes through in order to calculate the route to all destinations.

The router considers LSA type 1 and type 2 in order to build a shortest path tree. This means a router findsthe outgoing interface and the next hop to reach all of the nodes. It is important to note that we are nottalking about IP address, and a node is just represented by its router ID that identifies the router in the area.

NOTE: In order to accomplish step 1, which is to find the shortest path from a node to all the nodes in anarea, Dijkstra algorithm is executed.

1.

Install the IP address advertised by the node in the routing table. At this stage, all intra-area routes have beenfound for a given area.

2.

Consider LSA type 3 and 4 in order to find all destinations to other areas and the path to an ASBR.3.

This step is only performed by ABRs in an area in which there is a virtual link. In other words, theTransitCapability of the area is set to True.

This step is necessary to find if there is any shorter path than found previously in step 3. An ABR considersthe summary of the non-backbone transit area.

Note that this is the only time an ABR considers a summary from a non-backbone area as an ABR considersonly summary from backbone areas (except if it has lost all its connections to the backbone but still has anactive interface in area 0). At this stage all the inter-area routes have been found.

4.

The router considers the LSA type 5 in order to install the path to all external destinations. If an area isNSSA, LSA type 7 is considered instead.

5.

For an ABR attached to a NSSA area type 7, LSAs are processed after type 5.6.

Dijkstra Algorithm

Given a collection of nodes connected by way of point-to-point links, this algorithm finds the shortest path from agiven node (root) to all the nodes.

The router keeps track of three lists:

Unknown list—All the nodes to which the destination has not been found (distance is infinity). All routersstart in this list.

●

Candidate list—The list of nodes that have been found in the step-by-step process from the root and arecandidate to the shortest path tree.

●

Shortest path list—The list of paths that the calculating router has found to be the shortest path to all thenodes. In each step, one router is found and added to this list.

●

A calculating node S (source) does the first iteration by placing itself in the shortest path tree and adding itsneighbor to the candidate list.

In each of the following iterations, the router performs one of the following functions:

Move the shortest candidate (metric wide) from the candidate list to the shortest path list. Call this newlyadded node the active node. Initially, the source is itself the active node.

●

Look at the neighbors of active nodes to perform the following:●



If the shortest candidates are not already in the candidate list, move them into the candidate list. Thecost from the root is the cost to the active node plus the cost from the active node to the newly addedneighbors. The outgoing interface and next hop is inherited from the active node.

❍

If the shortest candidates are already in the candidate list and the current cost from the root is morethan the cost through the active node, update the candidate list with this new information.

❍

Otherwise if the node already exists and its cost is equal to the cost through the active node, add thenew next hop and outgoing interface (inherited from the active node) to the existing node.

❍

Check to see if the candidate list is empty. If the candidate list is not empty, start the iteration again.●

The algorithm requires N iteration (including the first one) where N is the number of nodes (router + transitnetworks represented by DR).

Virtual Link

To guarantee connectivity between areas, all areas should be connected to the backbone. There is no need forphysical connectivity: An area can be connected to the backbone logically through the use of a virtual link.

A virtual link connects two ABRs having in common a non-backbone area called a Transit area. A virtual link isconsidered as an unnumbered point-to-point link. However, there is an IP address associated with the virtualinterface. The virtual interface is the interface through which a shortest intra-area path to the remote ABR isdetermined.

Apart from attaching logically an area to the backbone, a virtual link can be used in order to have some controlover a given path. This is illustrated in the following diagram.

Since intra-area paths are always preferred over inter-area paths, if the link between RB and RC is placed in area 0,



area 1 will not have an optimal path since RB should go through RA to reach RC. If the link is placed in area 1, thetraffic in area 0 will not have the optimal path. By placing the link in area 1, and configuring a virtual link betweenRB and RC, both areas will use the RB and RC link since the virtual link is part of area 0.

As mentioned previously in the SPF section, after considering the summary LSA in order to find all destinations toother areas and the path to an ASBR, if there is a virtual link in the area, the ABR should set the TransitCapabilityto True in order to process the summary from the transit area (non-backbone). This is illustrated in the followingdiagram.

There is a virtual link between RC and ABR1 in area 1. ABR1 announces a summary LSA for network X with acost of 100. ABR2 announces the same summary but with a cost of 10. All routers in area 1 choose the shortestcost path and go through ABR2. However, since there is a virtual link between RC and ABR1, and this is part ofarea 0, RC will choose the intra-area path (area 0) rather than inter-area path advertised by ABR2. This could leadto a routing loop as RC will go through RB to reach ABR1 (virtual link) and RB will go through RC to reachABR2

By setting the TransitCapability to True in the SPF, an ABR considers the summary from the non-backbone area(area 1). Therefore, RC looks at the summary advertised by ABR2 and sees that there is a shorter path throughABR2.

Summarization

Summarization consists of combining a set of IP addresses and advertising it as a block instead of advertisingevery specific component within the address range. This reduces the amount of information to be propagated andreduces the amount of information other routers should store. Most importantly, it increases the stability of thenetwork as any change in a more specific component of the address range does not need to be propagated.

Internal Route Summarization

In order to guarantee inter-area routing, an ABR advertises through LSA type 3 destinations that are reachable ineach attached area into other areas.

An ABR can be configured to advertise a range of IP addresses and, therefore, summarize a given subnet ofnetwork for a given area instead of individually announcing all of the subnets within the range.



By configuring area x range < network> < mask>, an ABR summarizes the IP destination in area x using thespecified network range and advertises this range into other areas.

NOTE: An ABR needs at least a given IP address that is reachable in the summary range in order to advertise therange.

If area x is backbone (x = 0), only the intra-area route (native) in the backbone is summarized and not the routelearned through other areas. For example, an area range for area 1 is configured and announced to area 0. There isalso an area range for the backbone covering the network range advertised by area 1. The area range of thebackbone is only for its native route (intra-area ) and the summary of area 1 is leaked into other areas.

It can be desirable to hide a set of destinations in an area from being announced into other areas (see RouteFiltering) . In order to achieve this, the area x range network mask not-advertise command needs to beconfigured.

Internal Route Summarization Cost

If an ABR is configured to summarize a range of IP address, the cost of the summary range is:

The lowest cost of any IP address in the range if compatible rfc 1583 is enabled which is the default●

The highest cost of any IP address in the range if compatible rfc 1583 is disabled●

External Route Summarization

In OSPF, external routes are advertised by an ASBR. It is very important to understand that only the originator of atype 5 (ASBR) can summarize its external routes. In order to summarize the external information thesummary-address <network> <mask> command needs to be configured.

The not-advertise key word can also be used in order to suppress the summary.

In the case of an NSSA area, the external destination routes are imported into an NSSA area as type 7 and areflooded further by an ABR performing the type 7 to 5 translations.

An ABR performing type 7 to type 5 translations could summarize the external information generated by aninternal NSSA ASBR. By performing the type 7 to type 5 translations, the ABR becomes ASBR and changes theadvertising router field in the LSA header. Since the ABR is now the ASBR, it can summarize the externalinformation.

External Route Summarization Cost

If an ASBR summarizes a set of external IP addresses, the cost of the route is always the minimum cost of anygiven IP address. Note that RFC does not specify any recommendation regarding this.

Route Filtering

Filtering in OSPF is not as obvious and possible as is the case of the distance vector protocol. The reason is thatOSPF does not advertise any routes. Hence, filtering cannot be performed. To prevent a route from being installedin the routing table, an inbound distribute list can be configured to deny a given route. However, this does notprevent other routers to learn this route as the information is flooded through LSA.

As mentioned previously, between areas is similar to distance vector behavior. Cisco IOS has implementedinter-area route filtering (see CSCdi43518).

It is now possible to use an inbound or outbound filter to filter a route to be injected to an area or to be advertised



out of the area.

Another method of filtering is to configure a summary range for the area and use the not-advertise key word in thesummary range in order to suppress the advertisement of the route into other areas. This option provides lesscontrol since it can only be used for a given range to be suppressed (outbound) so there is no control to specify aspecific route within an IP range.

Parameters Affecting the Convergence

Convergence, by definition, is the time required for all routers in a domain to process any change introduced andgo back to the previous stable station.

There is always a tradeoff between convergence and stability in the network. A fast convergence requires a quickreaction to the change and this affects the stability of the network.

There are many parameters in OSPF affecting the convergence. These following table describes these parameters.

OSPF Parameter Convergence Affected

RouterDeadInterval The time during which the router maintainsthe adjacency even if there is no Helloreceived from the neighbor. After eachHello reception, the timer is reset to itsinitial value.

The RouterDeadInterval allows a router todetect a dead neighbor and notify otherrouters in the domain by generating a newLSA. The lower the value, the faster theconvergence. By default, the Dead Intervalis 40s for point-to-point, and Broadcastnetwork. For NBMA andpoint-to-multipoint, it is 120s. It can beconfigured to a lower value.

Depending on the link type and topology,the neighbor down situation can be noticedthrough Layer 2 and is much faster than theRouteDeadInterval.

For example, a back-to-back GE link willbe noticed through Layer 2 and will bemuch faster than the detection byRouterDeadInterval.

To change the default value, use thefollowing commands under a giveninterface:

ip ospf hello-interval <sec>ip ospf dead-interval <sec>

Note that all routers attached to a giveninterface require the same hello and dead



interval. Otherwise the adjacency will notbe formed.

LSA_Delay_Interval The initial interval time to wait beforesending an LSA (type 1 and 2 only). Thisparameter is Cisco-specific and its value is500 msec.

Currently it is not possible to change thisvalue but a back-off LSA generation will beimplemented in the near future and theinitial delay interval will be configurable.The reason behind this constant is that if alink flaps very quickly it won't be reportedbefore this interval time.

MinLSInterval This is an architectural constant of value 5seconds defined in RFC, and is theminimum time a router should wait beforegenerating the same LSA. This is also therate-limiting LSA mechanism for LSA type1 and 2. Other LSAs are based on the routeinstallation in the routing table and arerate-limited by SPF.

Cisco IOS, however, allows this parameterto be changed with a hidden command.In the near future, a back-off algorithm willbe implemented for LSA generation andwill make the value configurable.

MinLSArrival This is an architectural constant of value 1second defined in RFC, and is theminimum time required between thereception of the same LSA. If the sameLSA is received less than this time, it isignored and no Ack is sent to the neighbor.

Cisco IOS allows this parameter to bechanged with a hidden command.

SPF_SCHD_DELAY Once an LSA has been received, the SPF isscheduled to run. However, the SPFactually runs after SPF_SCHD_DELAYtime which is 5 seconds by default.

This parameter can be changed using theTimers spf <delay > <hold> command.



SPF_HOLD_INTERVAL If two SPFs have to run consecutively, theywill wait as specified by theSPF_HOLD_INTERVAL time which is 10seconds by default. This prevents runningtoo many SPFs due to a quick change.

This parameter can be changed using theTimers spf <delay > <hold> command.

Network Architecture

Campus Design Architecture

A scalable network is always designed in a multilayer or hierarchical manner. This allows for easy future growthand simplifies troubleshooting. It also increases the performance and isolates the problem caused to some parts ofthe network.

A multilayer campus network consists of three blocks:

Building block—Consists of Layer 2 switches in the wiring closet to connect users. The wiring closetmerges into a redundant distribution Layer 3 switch.

●

Server farm block—Consists of enterprise servers located usually in a separate block. The servers connect toLayer 2 switches, which in turn connect to a redundant Layer 3 switch.

●

Core block—The block that connects different building blocks and the server farm block. Depending on thenetwork size, this can be a direct connection between a distribution layer switch in a fully or partiallymeshed topology, or through Layer 2 switches. For a large campus design, Layer 3 switches in the core areused to connect the different distribution layers.

●

The following section describes each block and the different design models.

Building Block Design

The following diagram shows the architecture of a multilayer building block design. Users are connected to accessLayer 2 switches in the wiring closet that are dual-homed to redundant Layer 3 switches in the distribution layer.

Layer 3 switches reduce the scope of the broadcast domain and segment the campus into smaller and moremanageable sections.



The following two scenarios are available.

Every Layer 2 Switch in Wiring Closet is in Different VLAN

This scenario results in no virtual local area network (VLAN) trunking and no Layer 2 spanning tree loop. HotStandby Routing Protocol (HSRP) is configured between Layer 3 switches. Each Layer 3 switch is the primarygateway for one VLAN and the backup for another VLAN.

Redundancy—There are two links from every Layer 2 switch to the Layer 3 switch. If one of the links fails, theother link is used and HSRP maintains the connectivity of users.

Load Balancing—There are two ways to achieve load balancing:

Use a different subnet (VLAN) for the uplink toward the Layer 3 distribution switch. In this case, every hoston a subnet uses the corresponding subnet link.

●

Use Multigroup Hot Standby Routing Protocol (MHSRP) for the same subnet but use a different IP address.Every Layer 3 switch is active for a given IP address. Load balancing can then be achieved by using the twoactive HSRP addresses as two different gateways for hosts.

●

For example, Layer 3 switch A is active for group 1 with IP address 192.168.1.3 and the backup for group 2 withIP address 192.168.1.5. Layer 3 switch B is the backup for group 1 with IP address 192.168.1.4 and active forgroup 2 with IP address 192.168.1.6.



Same VLAN Across Layer 2 Switch

If the same VLAN is configured on two different Layer 2 switches, a trunk is needed between Layer 3 switches inthe distribution layer to guarantee the connectivity. This is illustrated in the following diagram. Layer 3 switches Aand B are at the distribution layer and Layer 2 switches C and D are at the access layer.

Switch A runs OSPF and, therefore, advertises the passive interface in its VLAN toward the core. If the linkbetween switch A and D fails, switch A still advertises this subnet since it has still another port up on this VLAN(link A to C). So when the traffic reaches switch A for the users connected to switch D, if there is no link betweenA and B, the return path is broken and there is black-hole traffic for users connected to switch D.

OSPF Design Perspective for Building Blocks

Since the access layer consists of a Layer 2 switch, OSPF is only relevant to Layer 3 switches in the distributionlayer. A Layer 3 switch in the distribution layer need not establish adjacency through the Layer 2 switch in theaccess layer since the Layer 3 switch needs only to announce the connectivity of its attached interfaces. For thisreason, the passive interface router OSPF command is used for the Layer 3 switch interface toward the wiringcloset. This reduces the routing protocol exchange and reduces the CPU overhead.

A routed link is used between Layer 3 switches in the distribution layer to avoid intra-building traffic passingthrough the core.

Server Farm Design

The server farm design architecture is similar to the building block architecture. However, because of its criticaloperation, it should be implemented with high capacity links and maximum redundancy to ensure connectivity allthe time.

The following diagram shows a server farm design with servers dual-homed to Layer 2 switches that are in turndual-homed to Layer 3 switches in the distribution layer.

NOTES: There is a trunk between the distribution layer switches. This is necessary for backing up the path of aLayer 2 switch should an uplink fail.

One of the Layer 3 switches is designated as the primary HSRP gateway and also is the root of the spanning tree(both ports are in forwarding state).

To increase the spanning tree recovery, UplinkFast is enabled in each wiring closet switch.



OSPF Design Perspective for Server Farm Block

As in the case of the building block design, only a Layer 3 switch is relevant to OSPF. There is no need for a Layer3 switch to establish adjacency through a Layer 2 switch link. Passive interfaces are configured on the Layer 3switch link in the distribution layer toward the Layer 2 switch. This reduces the routing protocol update anddecreases CPU overhead.

Core Block Design

The different building blocks and the server farm block communicate to each other through the core block. Ittypically consists of Layer 3 switches but a Layer 2 design could exist as well. This section focuses on twodifferent core designs: Layer 2 versus Layer 3, and the OSPF design practices for each. This section is describestwo scenarios:

Core design without a Layer 3 switch, essentially Layer 2●

Core design with a Layer 3 switch●

The different designs are discussed with respect to the size of the campus network for which they are best suited.

Core Block Without Layer 3 Switch

Without the presence of a Layer 3 switch in the core, connecting the different distribution Layer 3 switches can bedone through a direct Layer 3 connection or through the connection by way of a Layer 2 backbone switch.



Fully Meshed Campus Backbone

For a small enterprise backbone, it may be desirable to directly connect the Layer 3 distribution switch of differentbuilding blocks. Note that this is not a scalable solution and should be used for small enterprise backbones as thenumber of the link increases as switches are added.

OSPF Design Perspective

All links between Layer 3 switches are placed in the same area 0 and each Layer 3 switch is adjacent to allneighbors. The interface toward the Layer 2 switch is passive.

If a Layer 3 switch is connected to a high number of VLANs, it is desirable to summarize remote subnets beforeadvertising them into the backbone. To summarize the building block subnets, a routed link is configured betweenthe Layer 3 switch in each building block and server farm and is placed in a different area. In addition, the passiveinterface is part of this same area. Therefore, the Layer 3 switch becomes an ABR and is able to summarize thedifferent subnets in a building block into a single IP range and advertise to the backbone.

Partially Meshed Campus Backbone

This is similar to a fully meshed backbone and should be used for small to medium size campus networks. Sincethe server farm plays a centralized role, all building blocks are connected to the server farm block.




All links between Layer 3 switches are placed in area 0 and the interface toward Layer 2 switch is passive in orderto reduce the routing protocol update.

To summarize the building block subnets, a routed link is configured between the Layer 3 switches in each blockand placed in different areas. Therefore, the Layer 3 switch becomes an ABR and is able to summarize the subnetof a building block into a single IP range and advertise the summarized routes toward the backbone.

Layer 2 Core Backbone

To reduce the number of interface connections between the Layer 3 switches in the distribution layer, a Layer 2core can be used. The Layer 2 core connects all Layer 3 switches in a single VLAN, making them part of the samesubnet.

To prevent spanning tree loop and its delayed convergence time, the link to the backbone is defined as a routedinterface (no VLAN trunk) and there is no loop in order to put spanning tree in a blocking state. This design can beused for a small or medium campus network.


All Layer 3 switches in the distribution layer are in the same VLAN and share the same IP address. All Layer 3switches in the distribution layer are in a single area 0 and have a passive interface on interfaces toward the wiringcloset switch in order to reduce the routing table update.



In terms of OSPF operation, a DR/BDR is elected for the common subnet. To reduce the number of subnetsadvertised toward the core, a routed link is configured between the Layer 3 switch in each block and placed in adifferent area. Therefore, the Layer 3 switch becomes an ABR and is able to summarize the subnet of a buildingblock into a single IP range and advertise it toward the backbone.

Core Block with Layer 3 Switch

To build large and scalable campus networks, the core block should be based on Layer 3 switches, connecting allLayer 3 switches in the distribution layer. This reduces the peering of Layer 3 switches in the distribution layeraltogether and therefore scales for a large campus network.

The exact topology of the Layer 3 switch in the core depends on the size of the campus network, but a minimum oftwo Layer 3 switches with dual links between them is recommended to guarantee maximum redundancy and fastconvergence.

Redundancy—Every distribution switch should have two links to the core router. The core routers shouldhave two links between each other. If one of the links goes down, the other is immediately used since thereare two equal path costs.

●

Load Balancing—Every Layer 3 switch should have two equal paths to other Layer 3 switches and loadbalancing. Depending on the configuration, per destination, or per packet load, balancing is used. However,it is recommended to leave the default destination as load balancing in order to avoid out-of-sequence packetreception.

●




Every building block or server farm block should be placed in a different area. That is, the link between thedistribution Layer 3 switch in each block and the core Layer 3 backbone is placed in different areas.

The link between the Layer 3 switch core is in area 0, making the core Layer 3 switch an ABR. Summarization isdone on the ABR core Layer 3 switch in order to decrease the number of subnets advertised to each area.

In addition, a routed link is configured between Layer 3 switches in the distribution layer of each block. Thisallows the traffic in a given block to go directly between Layer 3 switches in the distribution layer without havingto go through the core.

Every Layer 3 switch in the distribution layer has two equal cost paths to the other Layer 3 switches in thedistribution layer. If one of the links becomes unavailable, the other link is used immediately and the convergencetime is minimum.

Alternative Layer 3 Core Topology

Depending on the size of the campus network and the number of building blocks and server farms to interconnect,it is desirable to use a core Layer 3 switch consisting of more than two switches. This decreases the number ofpeering per ABR. This can be used for a very large campus network.

The following diagram shows a campus design consisting of four switches in the core. Building block Layer 3switch peering has been divided among the four core Layer 3 switches in order to decrease the number of



adjacencies per ABR.

Every building block and the server farm is placed in different areas represented by a different color in thediagram.

A, B, C, and D are core Layer 3 switches having their link in area 0 represented in black.

Note that every Layer 3 switch in the distribution layer has two equal paths to all other Layer 3 switches in thedistribution layer. All building blocks connected to the same core Layer 3 switch is two hops away from eachother, otherwise there is three hops.

For example, the building block 1, 2, 3 and 4, 5, 6 are three hop away since they are not connected to the sameLayer 3 core switch.

However, every building block is two hops away from the server farm. This is desirable since the majority of thetraffic is between each building block and server farm.

It should be noted that although each Layer 3 switch has two equal path costs to the server farm, ABR switches A,



B, C, D will only pick one link since intra-area routes are always preferred. Should the preferred intra-area routelink go down, OSPF would need to converge in order to have another path.

If a fast convergence is required, it is desirable to have two equal paths from each ABR to each distribution layerswitch in the server farm. In this case, the two Layer 3 switches in the distribution layer of the server farm blockshould be connected to all four ABRs in the core. The following diagram shows this topology

WAN Design Architecture

There are four main WAN design architectures. The choice of the topology depends on the number of sites andoptimal routing for intersite communication.

Simple Point-to-Point

When the number of sites or campuses to interconnect are small (3 to 4), a point-to-point architecture can be used.All routers connected through a point-to-point network should be part of area 0, summarizing each area or sets ofareas in each site and the OSPF network type would be point-to-point.

Partial Mesh

Use this solution if the number of sites is substantial and when there is no need for optimal routing between



different sites. The exact topology of the partial mesh depends on the importance of the traffic pattern betweensites.

If the sites that interconnect each other are campus networks, they would be part of area 0. If they are remote sites,they can be placed in a non-backbone area that connects to area 0.

The OSPF network type used could be point-to-point or point-to-multipoint but note that the latter introduces /32route.

Full Mesh

Use a fully meshed solution if the number of sites is relatively small and there is a need for optimal routingbetween sites.

Although all of the routers are fully meshed, the OSPF network broadcast or NBMA type should not be used. Thiswill cause many problems. For example, if the link between two non-DR/BDR routers goes down, the trafficbetween the two routers will be black holed since the DR still tries to communicate with both routers. Also, if alink between the DR and another router goes down, this router will be isolated and lose all connection. Therefore,it is required to use a point-to-point or point-to-multipoint network type.

Hub and Spoke

Hub and spoke is used when there is a high number of remote sites that connect to the main campus orheadquarters, and optimal routing is not required or at least does not justify the higher cost of additionalconnections between sites.

Since the spoke sites goes through the hub site to reach any destination, a default route is sufficient in order tomaintain the connectivity in the remote sites. For this reason, distance vector protocols are more suited for thistopology as the route advertised to remote sites can be controlled (filtered). In OSPF, if a link between the hub anda spoke goes down, there is a new LSA generated and flooded to all the spoke routers. This action does not affectthe routing at the spoke site since spoke sites rely on the default route to reach other destinations.

Protocol Design

Campus Design Considerations

This section builds on the Network Architecture section and provides all design considerations required for OSPFaddressing.

When building a large and scalable network, the first step is to plan a network IP structure that can be summarizedin a hierarchical level. Summarization decreases memory utilization on the routers holding the routing table. Moreimportantly, it increases the stability of the network and decreases the routing control packet update to bepropagated. This saves CPU cycles as the loss of a given route within a range is not propagated to other parts ofthe domain.

The IP subnet in each building block should be a contiguous IP block address in order to be summarized at anABR level. As a general OSPF rule, IP address structures should be contiguous within the area in order to besummarized.

Distribution Layer

All interfaces of Layer 3 switches in the distribution layer toward the wiring closet switch are passive. A routed



interface is configured between each Layer 3 switch in the building block and server farm block.

There are three primary reasons why this is important:

Traffic within a building block should not go through the core.●

In the absence of a Layer 3 switch in the core, the connection between Layer 3 switches in the distributionlayer constitutes the core. This puts this link in a non-backbone area and summarizes the subnet of abuilding block for other areas.

●

In a building block topology in which the same VLAN is spread over both switches, if a Layer 3 switch issummarizing and one of the links toward the Layer 2 switch goes down, the Layer 3 switch still advertisesthis subnet to the core since it has another port up in this VLAN. However, when traffic is destined to theusers connected to the Layer 2 switch that has the broken link, if there is no link to the other Layer 3 switch,a black hole occurs for the users connected to the Layer 2 switch.

●

Core Layer

This section discusses two core design scenarios: Layer 3 switches in the core and Layer 2 switches in the Core.

Design with Layer 3 Switches in the Core

This design is used for large and scalable campus networks. All Layer 3 switches in the distribution layer are dualhomed to the core Layer 3 switch. The exact topology of the core depends on the size of the campus, but aminimum of two Layer 3 switches is required for redundancy and proper operation.

NOTE: To avoid the core Layer 3 switch (ABR) from having a high number of adjacencies, divide the Layer 3switch in the distribution layer between different Layer 3 switches in the core.

Area Partitioning—A passive link is the link between Layer 3 switches in the distribution layer and theLayer 3 switch in the core. For each building block, the passive link is in a non-backbone area. The linkbetween Layer 3 switches in the core is in area 0.

●

Summarization—The subnets advertised in each building block are advertised by the Layer 3 switch in thecore (ABR) for other areas. This increases the stability and decreases the number of routing entries in thecore.

●

Convergence—To increase the convergence, every Layer 3 switch should have two equal cost paths to alldestinations in the network. This reduces the convergence time to a minimum. This is achieved by having adual link from every Layer 3 switch in the distribution layer to the Layer 3 switches in the core. Also, it isdesirable to have the same equal cost path on the Layer 3 core switch.

●

Design without Layer 3 Switch in the Core

Without the presence of a Layer 3 switch in the core, the Layer 3 switch in the distribution layer has to peerdirectly which makes this solution less scalable and is normally used for small to medium campus networks. TheLayer 3 switch in the distribution layer can be peered in one of the following ways:

Fully Meshed●

Partially Meshed●

Connected by way of a Layer 2 switch in the core and all Layer 3 switches being part of the same subnet●



Use a fully meshed topology for a small network design because the number of the links grows as a square of thenumber of Layer 3 switches in the distribution layer.

Use a partially meshed topology and Layer 2 core for a medium sized campus network. In a Layer 2 core design,all Layer 3 switches in the distribution layer are part of the same subnet and DR/BDR election takes place. Inaddition, flooding is more optimized in a partially meshed design.

Area Partitioning—For each building block and sever farm block, the passive interface and the link betweenLayer 3 switches in the distribution layer are placed in a different non-backbone area. The connectionbetween Layer 3 switches in the distribution layer is part of area 0. This makes the Layer 3 switch in thedistribution layer ABR.

●

Summarization—The subnets advertised in each building block are summarized by the Layer 3 switch in thedistribution layer (ABR) for other areas. This increases the stability and decreases the number of routingentries in the core.

●

Convergence—The Network Architecture section represented the network architecture for fully meshed,partially meshed, and core Layer 2. In all three network architectures there is only one link from each Layer3 switch in the distribution layer which makes the convergence less efficient than if there are two equal pathroutes from each distribution switch.

Two links from every Layer 3 switch in the distribution layer can be used, but because they are directlyinterconnected this significantly increases the number of links required. Note that in the case of Layer 3switches in the core, only two links are required to connect every Layer 3 switch in the distribution layer tothe core switch.

In the case of a core Layer 2 switch, there is a common Layer 2 core used with dual links to eachdistribution Layer 2 switch, and two common subnets. This increases the convergence time. The followingdiagram shows an example of a Layer 2 core with dual links from each distribution layer. All interfaces arerouted and there are no spanning tree loops. There are also two different subnets in different VLANs.

●



Layer 2 versus Layer 3 Core Design

The choice between a Layer 2 versus Layer 3 design depends on the actual size of the campus network and futuregrowth. As a design rule, it is best to start with a Layer 3 switch in the core since it is more scalable and easier toextend to a larger network as the campus network grows.

Also, in the core design with a Layer 2 switch, in order to avoid spanning tree loop and still have two redundantlinks for fast convergence, the link from each Layer 3 switch in the distribution layer is connected to the sameLayer 2 switch in the core.

Therefore, should one of the Layer 2 switches go down, the Layer 3 switch loses both of its links and shouldconverge through the other Layer 3 switch in the building block. It may take a few seconds. However, in a coredesign with Layer 3 switches, each Layer 3 switch in the distribution layer is connected to two different Layer 3switches in the core.

WAN Design Considerations

This section focuses primarily on the hub and spoke topology, as this is the most common WAN design.Guidelines for point-to-point, partially meshed, and fully meshed topologies are also discussed.

Simple Point-to-Point

This topology is used for interconnecting very few sites. The WAN link is point-to-point and, depending on thepresence of ABR already in each site, the link between the sites is part of area 0.

Partially Meshed



This topology can be used if there is a need for optimal routing between certain sites. Do not use NonbroadcastMultiaccess (NMBA) mode on the WAN link because the failure of a link between DR and a neighbor or twoneighbors will result in a black hole for some sites.

Point-to-point or point-to-multipoint OSPF network types can be used for the WAN link.

Fully Meshed

This topology is not scalable as the number of sites grows rapidly with the number of site to interconnect andshould be used for a small WAN topology that requires optimal routing between sites. Do not use OSPF networktype broadcast or NBMA as this will cause problems.

The high number of link redundancies between routers results in a high number of flooding and acknowledgments.For example, if a link flaps, there would be 0(n^2) flooding generated, where n is the number of routersinterconnected. If one router goes down, there would be 0(n^3) flooding generated. To decrease the flooding, usethe interface blocking feature.

The idea is to reduce the flooding over a redundant link in order to decrease the LSA update and acknowledgment.This saves bandwidth and CPU cycles. Routers still maintain neighbor relationships by way of the blockedinterface. However, when neighbors exchange their Database Description packets they are just sending emptyDatabase Description packets. The database of all the routers is still consistent, as the same information reaches agiven router by way of another path.

To configure the interface blocking feature, use the ip ospf database-filter all out interface command.

Only one side of the link needs to have this configured. For troubleshooting purposes, it is better to have both sidesconfigured with this command. If only one side is configured, then during the DD exchange the side not having thecommand still sends a summary of its database to the remote side.

Hub and Spoke

This topology is usually used when connecting a high number of remote sites to the main site, and when the use ofother topologies is not possible or is too expensive due to the number of links to be used. Since the spoke isconnected to the hub, a default route is sufficient in order to guarantee the connectivity of the remote site.

The problem with using OSPF over hub and spoke is that any link flapping between the hub and spokeunnecessarily floods to other spokes. This consumes bandwidth and CPU cycles without affecting routing since thespoke site still relies on the default route to maintain connectivity.

To reduce flooding due to a link flap, the remote spoke should be placed in a different area such as a totally Stubor totally NSSA. This will not generate any other summary other than the default type 3. By having a differentarea, a flap in one area will not affect other spokes in other areas.

As a rule, the size of the area should be decreased if the links are unstable. An area of 20 to 50 routers is a goodtradeoff depending on the overall number of spokes.

Alternatively, between the hub and spoke, other routing protocols such as the Enhanced Interior Gateway RoutingProtocol (EIGRP) stub feature can also be used and redistributed into OSPF.

Also, if the WAN link is not ATM, use Open Demand Routing (ODR) for a high number of hub and spoke. Formore detailed information, read the Designing Large-Scale Stub Networks with ODR white paper.

There are usually three types of connections from a spoke to the hub:

Simple link from spoke with backup ISDN●



http://www.cisco.com/warp/public/105/39.html

Dual link from spoke to two different hubs●

Dual link from spoke to two different hubs with backup ISDN●

Simple Link from Spoke with Backup ISDN

The spoke has ISDN as a backup interface for the primary interface and runs OSPF. As long as the primary link isup, the ISDN interface is standby and OSPF does not bring up the link. As soon as the primary link goes down, thebackup interface kicks in and OSPF establishes adjacency over the dial up link. When the primary link comes backup, the ISDN link automatically goes back to standby mode and the line is dropped.

Dual Link from Spoke to Two Different Hubs

The spoke has two equal path costs to any destination and performs load balancing. If one of the links goes down,the other is available.

A link is needed between the two hubs if summarization is performed by the hub routers. The reason being thatboth hub routers advertise a summary range for the area to which they are connected. If the link between one huband a remote spoke is down, one of the hub routers still advertises the summary range but the traffic to the spoke isdropped if there is no link between the hub.

The link should be part of the configured area in order to receive the more specific route. This will not work if thelink is placed in area 0 since the hub will receive a summary from the other hub and will ignore it. In addition togenerating a summary route, a route to Null 0 is installed on the hub router.

NOTE: If the hubs are connected to different areas, there should be an interface in each area in order to learn themore specific route. An InterSwitch Link (ISL) subinterface or a WAN subinterface can be used to have a link ineach area.

Dual Link from Spoke to Two Different Hubs with Backup ISDN

The ISDN line should be activated when both interfaces are down. However, a backup interface cannot be used fortwo different interfaces: A floating static can be used in this scenario.

A static route (default) is configured with an admin distance more than OSPF (110) and pointing to the ISDNinterface. OSPF is not configured as interesting traffic in order to keep the line inactive under normal operation.When both links go down, the static route is installed and traffic forwarded to the ISDN interface brings up thelink. OSPF then establishes adjacency and routes over the ISDN dial up. When one of the primary links comesback up, the default or other route from the primary interface is received. However, since the cost of the dial upinterface is higher, the primary interface is preferred and the ISDN line is dropped.

To Which Area the Backup ISDN Belongs

For an explanation of the OSPF dial-up scenarios, read the OSPF Dial-Backup Scenarios document.

NOTE: In a dual link with ISDN backup scenario, if the ISDN link is placed in area 0 and once the ISDN line isup, the remote spoke becomes ABR and therefore always goes through area 0 which is the ISDN line. This meansthat even if the primary interface is up, the ISDN link continues to be used since all summaries advertised by thehub (ABR) are ignored by the remote spoke (ABR) through a non-backbone area (the primary link is in anon-backbone area).

Dial Up Support on Hub Site



http://wwwin-cons/teams/team-isp/ospf/OSPFDial-Backup.html

If the hub site supports a high number of dials from different areas, read the Scalable ISDN Backup Strategy forLarge OSPF Networks document.

Design Recommendation Summary

Implement an IP addressing plan in order to summarize the different building block subnets into the coreand have a contiguous IP block for the area.

●

Use a routed link between the two Layer 3 switches in the distribution layer. This prevents the traffic withinthe same building block from being directed to the core. In a VLAN scenario spread over both distributionlayer switches, return traffic will be black holed if the link from one of the access switches to the Layer 3switch in the distribution layer goes down, and summarization is used. This occurs because there is anotherinterface up in this VLAN and the summary route will still be advertised even if there are no paths betweenthe Layer 3 switches in the distribution layer.

●

Use Layer 3 switches in the core, as this is the most scalable solution and allows an easy future growth.●

To decrease the convergence to a minimum, have two links from each Layer 3 switch in the distributionlayer to the Layer 3 switch in the core. The equal path cost installs both routes. If one link becomesunavailable, the convergence is immediate.

●

For a very large campus network, split the connection from the Layer 3 switch in the distribution layeramong different Layer 3 switches in the core. This decreases the number of adjacencies per Layer 3 switchcore (ABR).

●

For all Gigabit or Fast Ethernet links that have two connections to each switch, use the ip ospf networkpoint-to-point command. This reduces the link state database size as there would be no type 2 LSA. In caseof a link failure, when the two connected links come back up, the two switches do not have to wait (thedefault Wait time = RouterDeadInterval) 40 seconds before establishing adjacency. The reason for the 40seconds is for DR/BDR election.

●

By default, the auto-cost reference bandwidth is 100M. This makes the cost of all links above 100 M equalto 1. You can configure the cost manually or change the auto-cost reference bandwidth to 1G or 10 G. Thecommand to use under the OSPF process is auto-cost reference-bandwidth.

●

Planning and Implementation

IP Addressing

The IP addressing structure is one of the most important parts for a large and scalable network as summarizationdepends on an efficiently designed addressing structure.

The IP addressing in the campus can have the following simple structure 10.building.vlan.0/24. In this model, it iseasy to identify the IP address of each building and the VLAN within the building.

It is also very easy to extend the number of VLANs within a given building block. Following this structure, it iseasy to summarize each building block address with 10.building.0.0/16.





The first block can be used for the core link 10.1.vlan.0/16 and the building number starts from 2.

It is desirable to have a consistent router ID for all switches running OSPF. A loopback address with the followingstructure could be configured 10.0.bulding.switch-number /32 on each switch.

Summarization

Summarization takes place at the ABR level. Each Layer 3 switch in the core summarizes the IP address of eachbuilding block. Since the IP structure is 10.bulding.vlan.0/24, summarization is very easy to achieve and everyABR attached to a building block summarizes as 10.building.0.0/16.

NOTE: Without an IP addressing structure it is not always possible to summarize in an efficient way. Therefore,summarization relies totally on an efficiently designed IP address plan.

Scalability

A design should scale and grow without redesigning the whole network. The following points should beconsidered when designing a scalable network.

The IP address structure should be extensible and there should be reserved IP addresses for additional userswithout compromising the summary structure plan. For example, the structure 10.building.vlan.0/24 willhave space for any additional VLANs and summarization will remain 10.buildng.0.0/16.

●

For a large and scalable network, it is desirable to have a core architecture with Layer 3 switches. Thenetwork can grow by adding more core Layer 3 switches and having the distribution layer switches dividedbetween different core Layer 3 switches.

●

Convergence is an important factor. Always design two equal path costs from the distribution layer switchesto the core Layer 3 switches by having the distribution layer switches dual homed to two different switches.This avoids the Layer 3 core switches from having too many adjacencies and divides the distribution layerconnection among different pairs of Layer 3 switches in the core. In general, 50 to 60 adjacencies (25 to 30building blocks) can be a good starting point before adding more Layer 3 switches in the core.

●

For a hub and spoke topology, use totally Stub or totally NSSA areas for the spokes. This reduces anychange in type 3 and type 5 LSAs from flooding to the remote sites.

●

Divide your spoke as much as possible into different areas. This reduces type 1 flooding due to a link flap toother spokes.

●

Alternatively, rather than running OSPF, EIGRP can be used between hub and spoke. The EIGRP stubfeature or a higher number of spoke ODRs can be used.

●

If the remote site has dial up and there are a high number of remote sites to back up, read the read theScalable ISDN Backup Strategy for Large OSPF Networks document.

●

Migration

This section presents two migration scenarios:

Migration from fully meshed to Layer 3 switch in the core●




Migration from core Layer 3 switch into core Layer 3 switch●

Migration from Fully Meshed to Layer 3 Switch in the Core

The following diagram illustrates a fully meshed topology. The Layer 3 switches in the distribution layer are fullymeshed, and the link between the Layer 3 switches in each building block is in a different area thus making theLayer 3 switch an ABR. Summarization is done at each ABR level.

Migration StepsThe link between each Layer 3 switch in the distribution layer and the Layer 3 switch in the core is addedand the link belongs to the same area as the building block area. At this stage, the Layer 3 switches (see L1and L2 in the following diagram) learn all subnets within each building block but are not generating anysummary since they are not yet ABR (no link in area 0).

1.

An area range is configured for each area on the Layer 3 switch in the core. This will be used in step 3 whenthe Layer 3 switch in the core becomes ABR.

2.

Two links are configured between the Layer 3 switches in the core and are in area 0. This makes the Layer 3switches ABRs and they will start generating the summary range into other areas.

The following diagram illustrates the above three steps, but for clarity only one area is represented.

3.



At this stage, the Layer 3 switches in the distribution layer ignore the summary received by Layer 3 switchesin the core since they are ABR. An ABR only considers summary from the backbone as long as it has aneighbor in area 0.

Since HSRP is running between the two Layer 3 switches in the distribution layer, A1, B1, and C2 are theprimary HSRPs for all of the VLANs. In this case, we can safely remove the full mesh between A2, B2, andC1 as shown in the following diagram.

4.

At this stage, A2, B2, and C1 are internal routers and are considering the summary from Layer 3 switches inthe core. After checking that the summaries are present and installed in the routing table, reverse the HSRPpriority making A2, B2, and C1 the active HSRPs for all of the VLANs. In this case, we are routing throughthe Layer 3 switch in the core.

5.



Remove the link between A1, B1, and C2 and reestablish the HSRP priority as before (load balancingbetween different VLANs).

At this stage, all Layer 3 switches in the core are interconnected through Layer 3 switches in the core. Thefollowing diagram shows the final topology.

6.

Migration from Core Layer 3 Switch into Core Layer 3 Switch

The following diagram illustrates the core Layer 3 switch into core Layer 3 switch topology. All Layer 3 switchesin the distribution layer are connected through Layer 3 switches in the core, and are part of the same subnet(VLAN). Also note that there is no spanning tree loop.



Migration StepsThe link between each Layer 3 switch in the distribution layer and the Layer 3 switch in the core is addedand the link belongs to the same area as the building block area. At this stage the Layer 3 switches (see L1and L2 in the following diagram) learns all the subnets within each building block but are not generating anysummary since they are not ABR yet (no link in area 0).

1.

An area range is configured for each area on the Layer 3 switch in the core. This will be used at the next stepwhen the Layer 3 switches in the core become ABRs.

2.

Two links are configured between Layer 3 switches in the core and are in area 0. This makes the Layer 3switches ABR and they will start generating the summary range into other areas.

The following diagram shows the above three steps but for clarity only one area is represented.

3.



The Layer 3 switch in the distribution layer ignore those summaries received by the Layer 3 switches in thecore since they are ABR. An ABR only considers summaries from the backbone as long as it has a neighborin area 0.

Since HSRP is running between the two Layer 3 switches in the distribution layer, A2, B2, C2, and D2become the primary HSRPs for all of the VLANs. In this case, all connections between A1, B1, C1, and D1can safely be removed.

Make sure that the DR or BDR are part of the remaining switches (A2, B2, C2, and D2) so noresynchronization will occur and the interruption is kept to a minimum.

4.



A1, B1, C1, and D1 are internal routers and are considering the summary from the Layer 3 switch. Afterchecking that the summary is present and installed in the routing table, reverse the HSRP priority makingA1, B1, C1, and D1 the active HSRP for all VLANs. In this case, we are routing through Layer 3 switch inthe core.

5.

Remove the link between A2, B2, C2, and D2 and reestablish the HSRP priority as before (load balancingbetween different VLANs).

At this stage all Layer 3 switches in the core are interconnected. The following diagram shows the finaltopology.

6.



Convergence Tuning

The Parameters Affecting the Convergence section described the many parameters that affect convergence.Configuring all of the parameters is not desirable as this affects the stability of the network. Fast convergenceshould be achieved through equal cost path. If one path becomes unavailable, routing through the other path isalmost immediate.

The following tuning practices assist in achieving faster convergence.

Use back-to-back links between switches whenever possible. In this case, the neighbor-down situation isdetected through Layer 3 which is much faster than RouterDeadInterval.

●

Configure all Gigabit Ethernets (GEs) and Fast Ethernets with only two switches as a point-to-point OSPFtype. This increases the convergence for establishing adjacency. By default, when two routers become activeon a multi-access network, they have to wait for the Wait timer = RouterDeadInterval in order to establishadjacency.

●

When there is DR/BDR and the RouterDeadInterval is relied upon for neighbor-down detection, the helloand dead timer can be tuned to 1 and 3 seconds, respectively. This type of scenario can consist of a coredesign with Layer 3 switches, and all Layer 3 switches in the distribution layer are on the same subnet.

●

The following commands should be used under the interface configuration mode:

ip ospf hello-interval 1

ip ospf dead-interval 3

The LSA generation and SPF calculation can be tuned. However, because of the presence of a dual path it is not



recommended to change or tune these parameters. In the near future a back-off algorithm will be implemented forboth LSA generation and SPF calculation.

Design Case Study

This section demonstrates an example of a large campus design. The objective is to build an enterprise networkconsisting of 50 building blocks and a server farm. There are 200 remote sites connected through a hub and spoketopology to the main campus through Frame Relay.

IP Structure Plan

The campus IP addressing plan is 10.building.vlan.0/24.

The first block, 10.1.vlan.0/24, is used for the core link.●

The 50 building blocks use 10.2.vlan.0/24 to 10.51.vlan.0/24. The next blocks, 10.52.vlan.0/24 to10.99.vlan.0/24, are reserved for future extension of building blocks.

●

The server farm block uses 10.100.vlan.0/24.●

To easily identify each router by its router ID, loopback interfaces are configured using the following structure:

10.0.building.switch-number/32

For example, the switch in the core has the router ID of 10.0.1.x /32 where x is the number of switches. Building 2has 10.0.2.x/32 where x is the switch number at the distribution layer, and so forth.

Since all summaries within each building block is 10. Building.0.0/16 and every building block is in an area. Eacharea ID is configured to match the summary. This quickly identifies each route to which the area belongs. Forexample, the area ID for building 2 is 10.2.0.0, and so on.

For WAN connectivity, 200 spokes are divided between two pair of hubs, with each supporting 100 spokes. The100 spokes are divided into 5 areas of 20 routers and the area is totally Stub. The first 100 spokes and pair of hubsuse IP block 10.101.0.0 and the second 100 spokes and pair of hubs use IP block 10.102.0.0.

If the remote sites are small sites then a /24 can be allocated for each spoke.

The following table summarizes the campus and WAN IP address plan.

Location IP Address Router ID Area ID Area Rangeon ABR

Corebackbone

10.vlan.0/24 10.0.1.switch-number/32 0 —

Building nwith n =2->51

10.n.vlan.0/24 10.0n.switch-numberN/32 10.N.0.0 10.n.0.0/16

ServerFarmBlock

10.100.vlan.0/24 10.0.100.switch-numbern/32 10.100.0.0 10.100.0.0/16

Spokes 1to 20

10.101.1->20.0/24 10.0.101.1->20/32 10.101.0.0 10.101.0.0/19



Spokes 20to 40

10.101.33->43.0/24 10.0.101.33->43/32 10.101.32.0 10.101.32.0/19

Spokes 40to 60

10.101.65->75.0/24 10.0.101.65->75/32 10.101.64.0 10.101.64.0/19

Spokes 60to 80

10.101.97->117.0/24 10.0.101.97->117/32 10.101.96.0 10.101.96.0/19

Spokes 80to 100

10.101.129->139.0/24 10.0.101.129->139/32 10.101.128.0 10.101.128.0/19

Spokes 1to 20

10.102.1->20.0/24 10.0.102.1->20/32 10.102.0.0 10.102.0.0/19

Spokes 20to 40

10.102.33->43.0/24 10.0.102.33->43/32 10.102.32.0 10.102.32.0/19

Spokes 40to 60

10.102.65->75.0/24 10.0.102.65->75/32 10.102.64.0 10.102.64.0/19

Spokes 60to 80

10.102.97->117.0/24 10.0.102.97->117/32 10.102.96.0 10.102.96.0/19

Spokes 80to 100

10.102.129->139.0/24 10.0.102.129->139/32 10.102.128.0 10.102.128.0/19

Network Architecture

The following diagram shows the network architecture.



There are 50 buildings to interconnect, therefore, a multilayer campus is used. The design consists of two Layer 3switches in the access layer (wiring closet) dual homed to two Layer 3 switches in the distribution layer, which inturn are dual homed to Layer 3 switches in the core. The 50 block is divided into two sets of 25 blocks, eachconnecting to a different pair of Layer 3 switches. This reduces the number of adjacencies per Layer 3 switch inthe core to 2*25 = 50 since there are two Layer 3 switches in each building block.

For a WAN design, the 200 spokes are divided in two sets of 100 spokes each. These are connected to a differentpair of hubs. Each 100-spoke set is divided into 5 areas of 20 routers.

All areas are totally NSSA in order to reduce all type 3 and type 5 flooding into the spokes. By reducing thenumber of routers per area, type 1 flooding is minimized and a flap in one area does not affect another area.

Each pair of hubs is connected to Layer 3 switches in the core through Layer 2 switches. This makes all routerspart of the same subnet (VLAN).

Note that there are two different subnets for connectivity between each pair of hubs and Layer 3 switches in thecore.

OSPF Area Partition

On all links toward the access switch, a passive interface is configured on the Layer 3 switches in the distributionlayer. There is a link between Layer 3 switches in each distribution layer in order to optimize intra-building traffic.



All interfaces of Layer 3 switches in the distribution layer (including passive) are placed into one area.

The backbone OSPF consists of Layer 3 switches in the core, and the two pair of hub switches connecting thespoke sites. Each pair of hubs has 5 areas of 20 spokes.

A link is needed between each hub pair because if the permanent virtual circuit (PVC) from one of the hubs to theremote spoke goes down, that hub should still learn the route from the other hub and could redirect the traffic.Since both hubs are summarizing the remote area and also installing this summary to Null0, the link between thetwo hubs should not be placed in the backbone so that the more specific route can be learned and also be able toredirect the traffic.

Since there are five areas supported by ABR, and in order to avoid five links, ISL is used with five subinterfaces ora Frame Relay subinterface link with each subinterface placed in the corresponding area.

Summarization

The IP structure plan makes the summarization very easy to achieve. Every ABR summarizes all its attached areas.This significantly increases the stability of the network and reduces the routing table.

The summary ranges that need to be configured are represented in the above table and are actually the same asArea ID.

Fast Convergence

Every Layer 3 switch in the distribution layer has two equal path costs to all destinations in the network. Thismakes the convergence almost immediate.

Note that the server farm is fully meshed to all core Layer 3 switches. This is necessary in order to have two equalpaths from each core Layer 3 switch to the server farm distribution switch because intra-area paths are preferredover inter-area paths.

Also note that although it seems there are too many paths from the server farm switch to Layer 3 switches, there isactually only two equal cost paths since intra-area paths are preferred over inter-area paths.

All links in the core Layer 3 switches are point-to-point network type, therefore, no DR/BDR election takes place.The switches are connected back-to-back, therefore, the neighbor-down detection is found through Layer 3 whichis very fast and hello timers do not need to be tuned.

Hello timers are tuned for the connection between the hub pairs and Layer 3 switches since they are part of thesame subnet.

The HelloInterval is set to 1 second and the RouterDeadInterval is set to 3 seconds.

Operating the Solution

Operation Verification

This section provides a summary of verification and troubleshooting of the OSPF operation. To verify the properoperation of OSPF and routing, the following commands can be used.

Command Description



show ip ospf Provides general information such as:

OSPF router ID●

Number of links in different areas●

Type of router (ABR or ASBR)●

SPF and LSA generation timers●

Number of SPFs executed within an area●

Area range for a given area●

show ip ospfinterface < intf >

Identifies information such as:

If OSPF is enabled on a given interface●

The state of the interface●

Hello and Dead interval●

OSPF network type●

show ip ospfneighbor

Indicates the state of the router adjacency withother neighbors. Anything other than Fullindicates a problem. Note that in a multi-accessnetwork, only DR and BDR become fullyadjacent with all routers and two non-DR/BDRsremain in the 2way/DOTHER state.

show ip ospfdatabase

Displays a summary of the OSPF database andcan be used to quickly identify if there are anymissing LSAs in a given area.

Show ip ospfdatabase <type><link-state-Id> |<adv-router> |<self-originate>

Provides detail information regarding thedatabase for a given LSA type and should beused for troubleshooting purposes.

Show ip ospfdatabasedatabase-summary

Displays the number of LSAs in each areabased on the type as well as a total of all LSAs.

Show ip ospfborder-routers

Displays the router type route, which is a routeto ABR and ASBR and is used for type 3 andtype 5 troubleshooting issues.

Show ip route<x.x.x.x>

Displays the routing table and next hop, and therouting protocol from which the route waslearned. It can be used to determine if a specificroute is or is not in the routing table.

Show ip routesummary

Provides the total number of routes learned byway of each protocol and the memory used foreach protocol.



Troubleshooting

OSPF Adjacency Issues

One of the most common issues in OSPF is the failure of two neighboring routers to become adjacent. There canbe many causes. The following items can be verified when troubleshooting adjacency issues:

Make sure the network type is the same on all routers attached to a media.●

If the hello timers have been changed, make sure all routers have the same value for hello intervals and deadintervals on a given media.

●

Make sure you have the same maximum transmission unit (MTU) on all routers attached to a media.●

Run debug ip ospf adjacency to find out more about the cause of the problem. This command should beexecuted on both sides of the link. Also, this debug command can be safely executed on a router withoutproducing any problems.

●

If the routers change to the Exchange state and go no further, there may be an MTU mismatch.●

The Hello packets are small packets and pose no problem. The Database Description packets are larger and maycause the interface to drop the packets before OSPF can see them. This is often a problem when interfacing toother vendors. Some vendors also count the MTU differently than does Cisco. They may include the Frame Relayheader bytes in the MTU calculation, and so even if it looks like the MTUs match, they in fact may not.

To determine the MTU of an adjacent router use an extended ping with the DF bit set. Use a sweep range of sizesstarting at a number that is close to the MTU with a delta of 1 byte per packet. When the pings fail, you can countthe number of successful packets to determine the neighboring router's true MTU.

For more information, read the Why Are OSPF Neighbors Stuck in Exstart/Exchange State? tech note.

Route Not in the Routing Table

First determine what type of route this should be. Then proceed as follows:

Intra—Determine to which router this route should belong (connected interface). Then check the router IDof this router in your database to see if this router is reachable and look at the link advertised by its router IDdatabase.

●

Inter—Check if the summary route exists in your database and which ABR is advertising it. Also check ifthe routing bit is set for this LSA. If the routing bit is set it doesn't necessarily mean that the summary madeit in the routing table. If the routing bit is not set, this indicates a problem. Check to see if the advertisingrouter (ABR) is reachable. If the advertising router (ABR) is not reachable, check the Why Are OSPFNeighbors Stuck in Exstart/Exchange State? tech note for further troubleshooting information.

●

External—Refer to the Why Are OSPF Neighbors Stuck in Exstart/Exchange State? tech note.●

Network Stability (How Often the SPF Runs)

The show ip ospf stat hidden command shows the last 10 times that the SPF ran the SPF algorithm. The timebetween SPFs tells you if SPF is run too often. The reason (LSA type) is specified.







The following example explains the different output.

router#show ip ospf statArea 2: SPF algorithm executed 6 timesarea 0: SPF algorithm executed 33 times

The following table shows the SPF calculation time.

DeltaT Intra D-Intra Summ D-Summ Ext D-Ext Total Reason

2d00h 4 0 4 4 0 0 16 R, SN

2d00h 4 0 4 4 4 0 16 N, SN

2d00h 4 0 4 4 0 0 16 R, N,SN

1d07h 4 0 4 4 4 0 16 N, SN

1d07h4 4 0 4 8 0 0 16 R, N,SN

1d07h 4 0 4 4 4 0 16 N, SN

1d07h 4 0 4 8 0 0 16 R, N,SN

1d07h 4 0 4 4 4 0 16 N, SN

1d07h 4 4 0 0 0 0 12 R, N

1d07h 4 0 4 8 0 0 16 R, SN

Delta T—Time since SPF ran. It tracks the last 10 times.

Intra—Number of intra-area LSAs that changed time for intra-area route (SPF) calculation.

D-Intra—Time spent computing these in 1/1000s, and for removing old intra route.

Summ—Number of summary LSAs that changed time for inter-area SPF calculation.

D-Summ—Time for removing old summary route.

Ext—Number of externals that changed time for external route.

D-Ext—Time for removing old external route

Total—Total SPF calculation time

Reason:

R—Router link changeN—Network link changeSN—Summary network link changeSA—ASBR summary changeX—External change



Related Information● Cisco AVVID Network Infrastructure Page

● OSPF: Frequently Asked Questions

● OSPF Design Guide

● Configuring OSPF

● OSPF Commands

All contents are Copyright © 1992--2001 Cisco Systems Inc. All rights reserved. Important Notices and Privacy Statement.



http://www.cisco.com/partner/sdm/ci



http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122cgcr/fipr_c/ipcprt2/1cfospf.htm

http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122cgcr/fiprrp_r/1rfospf.htm

http://www.cisco.com/pcgi-bin/imagemap/guestbar

http://www.cisco.com/public/copyright.html

http://www.cisco.com/public/privacy.html

Documents

Cisco - OSPF Routing Protocol - MIK Routing Protocol(1).pdf · OSPF Routing Protocol Contents Introduction Network Architecture Campus Design Architecture Building Block Design Server