Best Practices Enterprise IP LAN-WAN Design

© 2001 TAOS - The Sys Admin Company. All Rights Reserved.

BEST PRACTICES DOCUMENT

ENTERPRISE IP LAN/WAN DESIGNby Ya Wenversion 1.1

NETWORKING


Enterprise IP LAN/WAN Design version 1.1

CONTENTS

SECTION I: INTRODUCTION

SECTION II: BEST PRACTICES of ENTERPRISE IPLAN/WAN DESIGNPart I: Identifying the Client’s Business and TechnicalGoals

Step 1: Identifying Business GoalsStep 2: Identifying Technical Goals

Part II: Assessing and Characterizing the Client’sExisting Infrastructure

Step 1: Developing a Set of Existing LAN/WANNetwork Diagrams

Step 2: Identifying the Physical and LogicalLocations of Important Internetwork Equipment, forexample, the Routers, Switches, and Firewalls.Obtaining the Management Passwords to Access TheseDevices

Step 3: Documenting the Names andAddresses of Existing Network Servers andInterconnection Devices

Step 4: Documenting the Types and Lengths ofthe Physical Cabling and All the Layer Two CircuitsPart III: Logical LAN/WAN Network Design

Step 1: Selecting a Logical Network TopologyStep 2: Designing a Scalable, Redundant, and

Super Fast Multilayer Switching LAN InfrastructureStep 3: Designing a Scalable, Reliable, and

Fault-Tolerant WAN InfrastructureStep 4: Selecting Interior Routing Protocols

Part IV: Documenting Your Design and Making a FinalProposal

SECTION III CONCLUSION

REFERENCES

3

4

455

10

10

14

14

141515

18

2122

24

25

26

2



Imagine this scenario before you read further: you are hired todesign a network for a client who recently acquired a newoffice building. The client wants you to propose an internet-work solution to connect its current building with the new one.The network has to be state-of-the-art. The client has veryspecific technical requirements for the scalability, availability,and reliability as well as network performance and security. Asa systems and networking consultant, you are asked to do twotasks. One is to survey and evaluate an existing network byrecommending what to keep and what to change. Second, youneed to design a new network accommodating the existinginfrastructure.

“It sounds like a perfect project, but there is too much to do,so where should I start?,” you might ask. Well, this paperhelps you answer this question by presenting a systematicdesign methodology you can follow to meet the client’s busi-ness and technical goals. It works whether your client is in adifferent department within the company, or a new client com-pany to which you've just been assigned. After completing thispaper, you will be well equipped to design enterprise LAN/WANnetworks from a more systematic perspective instead of justkeeping busy configuring routers and switches. In particular,this paper addresses the major network design phases andhelps you create a more sophisticated method when buildingan enterprise network.

Unlike a how-to document, this paper primarily focuses on theenterprise LAN/WAN network design topics by answering whatand why. Therefore, no specific implementation solutions aregiven. It is recommended that you read a complementaryhow-to paper with detailed router configurations.

This paper focuses on enterprise IP LAN/WAN design only.That means if your client is an ISP or any sort of serviceprovider, some of the design philosophies presented here maydiffer. Also, this paper talks only about the IP network design,so you need to consult additional materials for the LAN/WANdesign of IPX, AppleTalk, or any other networks running non-IPprotocols.

3

SECTION I: INTRODUCTION

When you are asked to upgrade or design a new network foryour client, you should always address the potential problems,how they will be solved, and what the goals are for this net-work design project.

You need to work with the client closely and find out both theirbusiness and technical goals. For example, what kind of newapplications does the client want to add into the networkbecause of new business requirements? How much network



In this section, we present a systematic network designprocess for meeting clients’ business and technical goals. It isorganized into four parts that correspond to the major phasesof designing an enterprise LAN/WAN network:

Part I: Identifying the client’s business and technical goals

In this part, we define and discuss the business and technicalgoals a client wants to achieve after the network is complete.

Part II: Assessing and characterizing the client’s existing infra-structure

Here we determine the existing client environment, what kindof applications are running, and what needs to upgraded andwhat can be kept intact.

Part III: Logical LAN/WAN network design

This part describes the three-tier hierarchical logical networkdesign model. This is a key part of this paper.

Part IV: Documenting your design and making a final proposal

The final step of network design is to write a proposal andcommunicate your ideas with the client. This is a step any net-work designer must take.

4

SECTION II: BEST PRACTICES OF ENTERPRISE IPLAN/WAN DESIGN

Part I: Identifying the Client’s Business and Technical Goals



availability is required to support its core business operation?Accurately defining these goals is essential before starting thedesign work, because they are critical to the final "success"measurements of your job. It is beneficial for the later designphases if you initially understand the client's criteria for suc-cess and what goals must be met for the client to be satisfied.

Step 1: Identifying Business GoalsIn most cases, the network design is done in tandem withdefining the corporate structures. For example, one of themost popular virtual local area network (VLAN) designs isbased upon different departments. Therefore, it is advisable tounderstand how your client's company is structured in depart-ments, lines of business, partners, and remote offices. Thishelps you locate and identify all major user communities andalso helps you characterize network traffic flow.

In the early network design stage, network engineers com-monly spend a lot of time developing the technical specifica-tions, but overlook the importance of understanding theclient’s business goals. Ultimately, the goal is to service theclient's business and service it well.

For an enterprise LAN/WAN design, some of the typical busi-ness goals are to:

• Increase the company revenue and profit• Increase the employee productivity and improve

corporate communication• Reduce the telecommunication and network costs• Improve the security of sensitive and proprietary

corporate data• Provide better customer support service• Make data readily and securely available to all

employees regardless of location• Build partnerships with other companies

Step 2: Identifying Technical GoalsOnce you understand the client’s business goals well, you willface the challenge of working with the client and identifying allof their technical goals. Sometimes, this task is more difficultif the client has limited technical knowledge. In that case, as anetwork consultant, you have to take the initiative to define allthe technical goals based upon the client’s business require-ments and then get the client to approve the specifications.

Apparently, the technical goals and requirements must bespecified as clearly and precisely as possible in the early

5



LAN/WAN design stage. Depending upon the goals you and theclients set, totally different layer two technologies will beselected and internetworking products from certain vendorswill be preferred over others. For example, if the clientrequires 99.9 percent LAN/WAN availability in order to supportits Web hosting business operations, then network redundancyis the biggest issue of your design. Technologies like HSRP(Hot Standby Roter Protocol), BGP (Border Gateway Protocol),and spanning tree will be picked up to achieve this goal. Sodon’t just rush to the design; instead, spend a good deal oftime with your clients, understanding what they want toaccomplish technically and communicating with them to deter-mine their goals and how to achieve them. This work will payoff when you start the logical and physical design.

The most typical technical goals in an enterprise LAN/WANdesign include scalability, availability, performance, and securi-ty.

Scalability. Scalability is the ability of the LAN/WAN networkto continue to function well as it is changed in size or volumeto meet new enterprise traffic or application requirements.Typically, the rescaling is to a larger size or volume. In today’sdynamic and rapid changing business environment, enterprisesare rapidly adding users, various applications, opening newbranch offices, and connecting with external partners to meetnew business challenges. Therefore, scalability is always thebiggest concern, and the primary technical goal for an enter-prise network design.

Scalability never ends. Its design should be reflected and inte-grated into every single design and implementation decisionmade throughout the entire process. When surveying theexisting network, check out the overall LAN/WAN infrastructureand note the places that lack scalability. For example, arethere enough switch ports available in the wiring closet? Howdoes the bandwidth utilization of the frame relay link look? Inthe meantime, the client should help you understand howmuch the network will expand in the next year and in the nextone to two years. Be sure to ask the client the following scala-bility questions:

• How many new users are expected to be added into the corporate internetwork in the next 12 months? Thenext 24 months?

• How many more servers (or hosts) will be added intothe internetwork in the next 12 months? The next 24months?

6



• How many new network applications will be introducedinto the internetwork? How much more LAN and WANbandwidth will be needed to run these new applications?

• How many more sites will be added into the network inthe next 12 months? The next 24 months?

• How many more external partners will be joined to theextranet in the next 12 months? The next 24 months?

Availability. The second most important technical goal everyenterprise LAN/WAN network requires is high availability. Thisrefers to the amount of up time the network is available toend users. It can be expressed as a percent of uptime peryear, month, week, day, or hour, compared to the total time inthat period. For example, a network kept up and running for165 hours in one week (7 x 24 = 168 hours), means that theavailability in that week is 98.21 percent.

Unlike scalability, availability can be defined numerically, mak-ing it much easier to evaluate if a network design is successfulor not. Depending upon the essence of your client's business,the requirement for availability varies greatly. For instance, ifthe client relies heavily on the network for mission-criticalbusiness operations, then the need for availability will be muchgreater than a company that uses the network occasionally tosend and receive e-mails. It is very important to communicatewith your client and develop a reasonable and realistic avail-ability number, as it will relate directly to the allocation of theproject budget. Specify the number to be at least two digitsfollowing the decimal point. Consider the following example:99.70 percent availability means the network could be downfor 30 minutes per week; however, 99.95 percent availabilityindicates that the network will only be down five minutes perweek.

Ask your client the following questions to determine how muchavailability they will need:

• What is the nature of your business and how muchdoes it rely on the network?

• Which business operations rely on the network? Whatwill be the business and political consequences if thenetwork goes down?

• How much money will the company lose per hour ofnetwork downtime?

• Do you have enough IT budget to purchase all theredundant WAN circuits and internetwork equipment?

7



Performance. Network performance is a very broad and com-plicated topic. The requirement for it could be a very vaguedefinition stated by the client. They could say something like,“as long as the users have no complaints” or it could be a sci-entific one like, “I want to see a LAN throughput of at least1000 PPS (packets per second).” In the former case, you haveto work out more specific goals for the client to achieve theperformance they really need. In general, network perform-ance can be specified in very scientific and precise terms.Parameters such as capacity utilization, throughput, and delayare just some of the terms that indicate the health of anenterprise LAN/WAN. There are many management tools areavailable to collect and analyze these data.

Capacity utilization measures how much bandwidth is usedduring a specific time period. In a shared Ethernet LAN, due tothe character of Carrier Sense Multiple Access and CollisionDetection (CSMA/CD), a typical rule is that the utilizationshould not exceed 37 percent, because the collision ratebecomes excessive beyond this point. However, in a token-passing LAN, such as token ring or FDDI (Fiber DistributedData Interface), utilization could go as high as 70 percentbefore LAN segmentation or more capacity is needed. For WANcircuits, such as T1 and frame relay, optimum network utiliza-tion is about the same as for token-passing LANs. Do not con-sider adding more capacity if the current utilization is under 70percent. Management tools like HP OpenView and CiscoWorksare very useful for constantly monitoring the utilization of thenetwork bandwidth.

Throughput is the quantity of error-free data that is transmit-ted per unit of time. Don’t confuse it with the capacity.Capacity is a constant and determined by the physical layertechnologies, while throughput is a variable and determined bysuch factors as packet access method (CSMA/CD or token),network load, and error rate. The most popular way to meas-ure throughput is in PPS. The following table lists the theoreti-cal maximum PPS that an internetwork device could forwardbased on the frame size:

8



Table 1. Maximum Packets Per Second.

When using different layer two technologies, the maximumframe size also varies. Table 2 shows the maximum frame sizecorresponding to different technologies:

Table 2. Maximum Frame Size.

Delay, also referred to as latency, is another parameter thatmeasures network performance, especially for user interactiveapplications such as videoconferencing or voice-over IP. Inthese cases, users could very easily sense the network delayand thereby complain about the performance. Delay is causedby many reasons and some delays simply cannot be avoided.To set a realistic goal for how much delay is tolerable in yourclient’s environment, you have to research and thoroughlyunderstand which applications are running in the network andwhat kind of delay requirement exists for each application.Obviously, one way to solve the delay problem is to keepadding more LAN/WAN bandwidth into the network. However,if budget is a concern, technologies like priority queuing, com-pressing, and traffic shaping greatly helps improve networkperformance.

Security. Among all the technical goals, security is becomingmore important as enterprises keep connecting their privateinternetworks to the Internet as well as their partners andremote workers. Therefore, security concerns should be care-fully integrated into every step of the network design andplanning. The first and most important step to achieve good

9

You could be hired by the client to upgrade or enhance anexisting network instead of designing a new one from scratch.If so, you should initially develop a strategy to examine andcharacterize the existing infrastructure. With a completeunderstanding of its topology, uses, and behavior, you will beable to tell if the business and technical goals are realistic andwhere the problems exist.

To examine and characterize an existing network, you need todo the following:

• Develop a set of existing LAN/WAN network diagrams• Identify the physical and logical locations of important

internetwork equipment, for example, routers, switches,and firewalls. Obtain the management passwords toaccess these devices

• Document the names and addresses of existing networkservers and interconnection devices

• Document the types and lengths of the physical cablingand all the layer two circuits

Step 1: Developing a Set of Existing LAN/WAN NetworkDiagramsHopefully, your client can provide you network diagrams forthe existing LAN/WAN infrastructure. Occasionally, you will findno documentation. Regardless, the first thing you should do isto research and develop your own version of the LAN/WANnetwork diagrams. They should illustrate the most up-to-datenetwork topology and all addressing details. It could be justone diagram covering all the details in a small network, or itcould comprise many diagrams in a complicated network infra-structure. The goal is to understand what is going on in thecurrent network, and what could be done to upgrade the net-work to meet new business and technical requirements.



security for your client is to come up with a thorough corpo-rate security policy. You and your client should jointly makethese policies. Ask as many questions as possible to under-stand the risks associated with not implementing a secure net-work. Determine how sensitive is the data and where will it bestored. What are the consequences if someone breaks into thenetwork and acquires these data?

10

Part II: Assessing and Characterizing theClient's Existing Infrastructure



You should make both logical and physical network diagramsto describe the existing network infrastructure. The logical dia-gram should illustrate the logical architecture of the network,which can be hierarchical or flat, layered or not. The physicaldiagram should depict the physical location and connection ofthe network devices. You should also include as many detaileddiagrams as possible for specific parts of the internetwork,such as a detailed network diagram illustrating the site-to-siteVPN structure with a branch office. These diagrams will comehandy when you are troubleshooting part of the network.

A good network diagram should include:

• Detailed geographical information, such as countries,states, cities, and campuses

• WAN connections between different geographic locations

• Different buildings and floors • WAN and LAN connections between the campus, build-

ings, and floors• An indication of the data-link-layer technology for WANs

and LANs as well as the bandwidth; for example, framerelay, ISDN, 10Mbps Ethernet, or token ring

• Name of the WAN service provider; if peering, theautonomous system (AS) number should also be included

• Location and address of the important interconnectiondevices, such as routers, switches, and firewalls

• Location and address of major enterprise servers, suchas the mail server, file server, and DNS servers

• Location and address of any VPN severs that connectcorporate sites via WANs

• Location and address of any remote access servers andauthentication servers

• Location and address of major network managementstations

Figure 1 shows a sample network diagram for a client’s head-quarters office with three floors in a same building. It is obvi-ous that this is a physical network diagram based on the phys-ical locations of the internetworking devices.

11

Figure 1. Physical Headquarters Network Diagram.

Observing the preceding diagram, it is easy to identify theimportant devices and the corresponding IP addresses. Thisdiagram should help when making sound upgrade decisions.

Figure 2 shows a sample network that depicts logically adetailed site-to-site VPN network.



12



13

Figure 2. Logical Site-to-Site VPN Network.



Step 2: Identifying the Physical and Logical Locations ofImportant Internetworking Equipment, for example, theRouters, Switches, and Firewalls. Obtaining theManagement Passwords to Access These DevicesAn existing network is mainly controlled by the location andconfiguration of some of the important interconnectiondevices. For example, the routing behavior in an enterpriseLAN/WAN is always determined by the location and configura-tion of the layer three devices. Therefore, you need to identifywhich devices influence the network behavior and where arethey physically located. Typically, upgrading or enhancing anexisting network results in having to login to these devices andmake configuration changes. Make sure you have administra-tive access to these devices. If not, attempt to obtain pass-words from your client or implement password recovery proce-dures recommended by the vendors.

Step 3: Documenting the Names and Addresses ofExisting Network Servers and Interconnection DevicesAfter identifying the important devices from step 2, documenttheir names and the current network layer addresses and, insome cases, even the MAC addresses. You need this informa-tion when discussing the new design with the client.Meanwhile, this information helps you standardize the namingand addressing design. An easy-to-remember and well-docu-mented naming and addressing design greatly improves themanagement efficiency and makes future network trou-bleshooting easier. Choose your naming and addressing con-ventions for all machines and network devices and keep it con-sistent throughout the internetwork.

Step 4: Documenting the Types and Lengths of thePhysical Cabling and All the Layer Two CircuitsBefore upgrading or enhancing an existing network, determinethe type of existing physical connections. This helps you deter-mine the performance bottleneck and what kind of upgradethe client will need. Obtain the physical cabling informationfrom the client’s facility team, if your client happens not tohave a facility team by tracing down the cables. Acquire thelayer two circuits from the service providers and the relatedtechnical specifications. For example, find out how many WANconnections the client has, the type of layer two technologies(T1 or frame relay), the circuit IDs and line encoding methodused, and the technical support numbers for these WAN serv-ices. After collecting this information, document it by filling inthe following table:

14

Now you need design solutions to meet the scalability, avail-ability, performance, and security goals.

Step 1: Selecting a Logical Network TopologyA topology pictorially describes the configuration or arrange-ment of a conceptual network, including all the network nodesand connecting lines. During this phase, identify the networksegments and interconnection points between segments, sizeand scope of networks, and the types of required internet-working devices; for example, hub, switch, or router. You donot have to decide which devices just yet.

One of the best practices when designing an enterpriseLAN/WAN network is to develop a layered and hierarchicaldesign model, which is recommended by most network designexperts. Compared to other design models — such as flat ormesh — the hierarchical model allows an internetwork to bedesigned in three discrete layers, each focusing on specificfunctions. Modularity in network design allows designers tocreate elements that can be replicated as the network growsand changes. As each element changes, the cost and complex-ity of making the upgrade is constrained to a small subset ofthe overall network. Therefore, the hierarchical design modelprovides us the best network scalability.

A typical hierarchical network topology includes the followingthree layers:

1 A core layer consisting of high-end routers and switchesthat are optimized for availability and performance

2 A distribution layer of routers and switches that imple-ment network policies

3 An access layer that connects users via hubs, switches,and other interconnection devices



Table 3. Sample Table for WAN Links.

15

Part III: Logical LAN/WAN Network Design



Figure 3-1 shows a typical enterprise network topology usingthe hierarchical design:

Figure 3. Three-Layered Hierarchical Network Design Model.

1) The core layer design. Some refer to the core layer as “thebackbone” of the network. This is because it is always a high-speed switching backbone and should be designed to switchpackets as fast as possible. This layer should not perform anypacket manipulation, such as access list filtering that slowsdown packet switching. Also, neither servers nor end-usermachines should be attached to the core layer. When configur-ing routers in this layer, you should maximize the packetswitching throughput and optimize the core for low latencyand good manageability.

2) The distribution layer design. This layer is the demarcationpoint between the access and core layers and helps define anddifferentiate the core layer. Its purpose is to provide boundarydefinition; it's also where packet manipulation can take place.

When designing an enterprise LAN/WAN network, you caninclude the following functions into the distribution layer:

• Address or area aggregation• Departmental or workgroup access• Broadcast/multicast domain definition

16



• VLAN routing• Any media transitions that need to occur• Security policies

There are two ways to achieve routing — static and dynamic.

Static routing is the simplest form of routing. Discoveringroutes and propagating them throughout a network are left tothe network administrators. Statically programmed routes canmake the network more secure. There can be only a singlepath into and out of a network connected with a staticallydefined route unless multiple static routes are defined.Another benefit is that it is much more resource-efficient.Static routing uses far less bandwidth across the transmissionfacilities, doesn't waste any router CPU cycles trying to calcu-late routes, and requires far less memory. In some networks,you might even be able to use smaller, less-expensive routersby using static routes. Despite these benefits, you must beaware of some inherent limitations.

The biggest drawback is manageability. In the event of a net-work failure, or other source of topology change, the networkadministrator would have to manually accommodate thechanges everywhere in the internetwork. When the networkgrows to a certain level, it almost is impossible to manuallykeep track and update all the routing tables.

Dynamic routing is always preferred over static routing in amid-sized and large networking environment. The two classesof dynamic routing protocols are distance-vector protocolssuch as Routing Information Protocol (RIP) and InteriorGateway Routing Protocol (IGRP), and link-state protocols suchas the Open Shortest Path First (OSPF) protocol andIntermediate System to Intermediate System (IS-IS).

In the non-campus environment, the distribution layer can bea redistribution point between routing domains or the demar-cation between static and dynamic routing protocols. It canalso be the point at which remote sites access the corporatenetwork. The distribution layer provides policy-based connec-tivity.

3) The access layer design. The access layer is the pointwhere local end users are connect directly into the network.This layer may also use access lists or filters to further opti-mize the needs of a particular set of users. In an enterprise

17



campus environment, access layers can be designed to havethe following functions:

• Shared bandwidth• Switched bandwidth• MAC layer filtering• Microsegmentation

In the non-campus environment, the access layer can giveremote sites access to the corporate network via wide-areatechnology, such as frame relay, ISDN, or leased lines.

Step 2: Designing a Scalable, Redundant and Super FastMultilayer Switching LAN InfrastructureIn a campus environment, the network design should meet aclient’s goal for scalability, availability, and performance byfeaturing small broadcast domains, redundant distribution lay-ers, mirrored servers, and multiple ways for an end-user work-station to reach a gateway router. Usually these are consid-ered LAN design issues.

Formerly, network designers had only a limited number ofhardware options when purchasing a technology for corporatenetworks. Hubs were for wiring closets and routers were forthe data center. The increasing power of desktop processorsand the requirements of client-server and multimedia applica-tions, however, have driven the need for greater bandwidth intraditional shared-media environments. These requirementshave prompted network designers to replace hubs in thewiring closets with layer two switches, as shown in Figure 4.

Figure 4. Replacing Hubs with Switches in the Wiring Closet.

18



Replacing traditional hubs with switches in LAN environmentsshould achieve the following:

• Increased bandwidth available to users• Increased manageability of VLANs by organizing net-

work users into logical workgroups that are independ-ent of the physical wiring closet hubs, thereby reducingthe cost of moves, adds, and changes while increasingnetwork flexibility

• The ability to deploy emerging multimedia applicationsacross different switching platforms and technologies,making them available to a variety of users

• A smooth evolution path to high-performance switchingsolutions, such as Fast Ethernet and ATM

Segmenting shared-media LANs divides users into two or moreseparate LAN segments, reducing the number of users con-tending for bandwidth. LAN switching technology, which buildsupon this trend, employs microsegmentation. This further seg-ments the LAN to fewer users and ultimately to a single userwith a dedicated LAN segment.

Segments are interconnected by internetworking devices thatenable communication between LANs while blocking othertypes of traffic. Switches monitor traffic and compile addresstables, so they can forward packets directly to specific ports inthe LAN. Switches also usually provide non-blocking service,which allows multiple conversations (traffic between two ports)to occur simultaneously.

With the price of layer two switches decreasing rapidly, LANswitching technology has become the preferred solution formost network designers. Many companies are migrating fromthe traditional shared LAN technologies to switched technolo-gies to improve their LAN performance and reliability. Sincethere are hundreds of layer two switches in the market, whendeciding which to use, consider the kind of port density youare seeking. For example, do you need an intelligent switch orjust a dummy switch? Do you need VLAN support? What aboutthe spanning tree protocol?

In an increasingly dynamic computing environment, VLANdesign is becoming more significant for the enterprise net-work. With VLAN, the physical location of a user no longermatters, as users can be moved anywhere within the enter-prise without changing the static IP address as long as theVLAN number remains the same. This gives the highest scala-bility for an enterprise LAN.

19



To gain high availability in a LAN environment, a commonpractice is to implement the IEEE802.1d spanning tree proto-col (STP) between all the layer two switches. The STP guaran-tees that there is only one active layer-two path between thetwo network stations. A redundant path will be automaticallyselected and activated when the active path experiences prob-lems. Figure 5 is a typical design of an enterprise LAN. SwitchA acts as the root bridge for VLAN 2,4,6, while switch B actsas the root bridge for 3,5,7 when both switches are up andrunning. If switch A fails, switch B can become the root bridgefor VLAN 2,4,6; switch A can become the root bridge for VLAN3,5,7 too if switch B fails. The result is that both links from anaccess-layer switch carry and load balance the traffic and willfail over to a new root bridge if one switch fails. This designoffers the best availability as well as scalability to an enter-prise LAN environment.

20

Figure 5. High Availability LAN Design.

Besides STP, technologies like HSRP, server redundancy, andDHCP (Dynamic Host Configuration Protocol) are also veryimportant to achieve a high availability LAN.



Step 3: Designing a Scalable, Reliable, and Fault-Tolerant WAN InfrastructureDesigning and building a scalable and highly available WANnetwork is much harder than LAN design. Unlike campus LAN,you have full administrative control of all the links and theequipment, as WAN links are leased from various serviceproviders. Therefore, your connectivity relies heavily on serv-ice from external sources. To guarantee a scalable and reliableenterprise WAN network, ensure that the service providersoffer your client a very competitive service level agreement(SLA). The SLA should clearly guarantee the maximum latencya packet would experience when traveling throughout theprovider’s backbone and the best availability the provider canpromise. You should obtain a trustworthy SLA for all WANlinks. If the provider does not have one, switch to a newprovider who can offer it. In addition to adding competitiveSLAs for WAN links, make sure the WAN links are more scala-ble and reliable.

Adding Backup WAN Links. One obvious way to obtain WANreliability is to add a backup WAN link to the primary link andto configure the routers. This guarantees the availability of allthe WAN service even if the primary link goes down. Forexample, a common practice in today’s enterprise network toensure reliable Internet connectivity is to add an ISDN backupcircuit to the primary T1 or frame relay circuit. However, thissolution is not possible if the company does not have thebudget to lease backup circuits. Another possible weak point ofthis solution is because carriers lease capacity to each otherand use third-party companies to provide capacity to multiplecarriers, it is harder to ensure that the circuits are trulydiverse. They might share the same telcom facilities some-where in between, which defeats the backup mechanism.

Multi-Homing the WAN Links Using BGP4. A new way todesign a reliable WAN network is multi-homing, or providingan enterprise network with more than one entry point into theInternet. This provides fault tolerance for applications requiringInternet access and high availability for enterprise publicaccessible servers, such as public e-mail web servers.

BGP is an extremely complicated exterior routing protocoldefined in RFC 1771, as it allows an enterprise to create loop-free, inter–domain routing between different autonomous sys-tems (AS). It provides redundant Internet connections and isalso used by some large enterprises to connect isolated cam-puses. If the client has complex scalability and availability

21



goals for the WAN routing, BGP provides the best and mostscalable solution. Before implementing BGP in the WAN design,be sure that you or your client have the necessary knowledgeand skills to correctly configure and maintain it, as this power-ful application could cause a network disaster if not configuredproperly.

Figure 6 shows an enterprise called Foobar.com with twoInternet connections peering with two different serviceproviders using BGP4. Both links are configured to do Internetaccess load balancing as well as connection fail over.

22

Figure 6. Redundant WAN Connection Using BGP4.

Step 4: Selecting Interior Routing ProtocolsOne of the most important decisions you have to make indesigning an enterprise IP LAN/WAN network is to select thebest routing protocols. Even though they all have the samegeneral goal, which is to share network reachability informa-tion among all the routers, different routing protocols havedistinctly different scalability and performance characteristics.Some were designed for small internetworks while others work



best in a larger-scale network. Therefore, no single routingprotocol is a universal solution for an enterprise. However, as anetwork designer, you should be able to make the right choice.

The dynamic interior routing protocols used widely today areRIP, IGRP, EIGRP (Enhanced IGRP), and OSPF. Each protocolhas its own advantages and disadvantages. Whether to usethem or not depends upon the specific routing requirementsand the particular scenario. For example, if you want UNIXservers to participate in the routing domain, then RIP is theonly option since UNIX servers talk only RIP. If you haverouters from different vendors in the same routing domain,then EIGRP will not work because it is a proprietary Ciscosolution. Another important consideration when selecting theinterior routing protocol is the IP addressing used for the inter-network. RIP and IGRP only work in a fixed length subnetmasking (FLSM) environment, while OSPF and EIGRP advertiseIP routes along with the subnet masks; hence both OSPF andEIGRP could handle variable length subnet masks (VLSM).Therefore, if using VLSM IP addressing and discontinuous sub-nets within the client’s network, either EIGRP or OSPF must beused as the interior routing protocol.

When selecting a routing protocol, always consider of theclient’s technical goals and ask the following questions:

• Are there any limits placed on the metrics?• Can it support variable subnet length?• How quickly can the routing protocol converge when

upgrades or changes occur?• How often are routing updates or link-state

advertisements transmitted?• How much data is transmitted in a routing update?• How much bandwidth is used to send the routing

updates?• How widely are routing updates distributed?• How much CPU utilization is required to process routing

updates or link-state advertisements?• Are static and default routes supported?• Is route summarization supported?

By answering the preceding questions, you should be able todetermine the best routing protocol available to you under cer-tain circumstances. On the other hand, selecting the routingprotocol is simply a personal preference. Network designerswith more experience in OSPF design and configuration willprobably feel more comfortable using OSPF than EIGRP.

23



Now you have a design solution for your client. But you arenot done yet. You need to deliver your final design with all thedetails to the client. Don’t ignore the importance of the pro-posal. A well-written and articulated proposal is usually thedecisive factor for the client to approve or disapprove thewhole project. It is not uncommon that some network design-ers have excellent design skills but lack the ability to writethem. Documentation is a must-have skill for any serious net-work designer.

Sometimes, a client will provide you an RFP. If so, the finalnetwork design should follow the format prescribed in it. Ifnot, you should develop a document that describes the busi-ness and technical requirements, the existing network, thelogical and physical LAN/WAN design, and the budget andassociated expenses. It should also include an executive sum-mary and a primary project goal with all the details about thenetwork topology, naming and addressing schemes, and secu-rity policies. In general, the final design proposal should becomprehensive enough to cover the following topics:

• Executive summary• Project goal• Project scope• Design requirements (both business and technical)• Current state of the network• Logical network design• Physical network design• Implementation plan• Project budget

24

Part IV: Documenting Your Design and Making a FinalProposal



SECTION III: CONCLUSION

This document has outlined a systematic methodology forenterprise IP LAN/WAN design. It covers the four major designphases and provids step-by-step procedures in each of thenetwork planning and designing phases. Although there are noimplementation solutions, it establishes which questions to askduring the design process and should be considered a networkdesign guide.

25



REFERENCES

Kennedy Clark, Kevin Hamilton. 1999. CCIE ProfessionalDevelopment: Cisco Lan Switching. Cisco Press.

Douglas Comer. 2000. Internetworking with TCP/IP Vol. I:Principles, Protocols, and Architecture. Prentice Hall.

Jeff Doyle. 1998. CCIE Professional Development: RoutingTcp/Ip. Cisco Press.

Bassam Halabi. 1997. Internet Routing Architectures. CiscoPress.

Priscilla Oppenheimer. 1999. Top-Down Network Design. CiscoPress.

Catherine Paquet. 1999. Building Cisco Remote AccessNetworks. Cisco Press.

Ivan Pepelnjak. 2000. EIGRP Network Design Solutions. CiscoPress.

W. Richard Stevens. 1994. TCP/IP Illustrated Volume 1.Addison Wesley Longman, Inc.

Thomas M. Thomas. 1998. OSPF Network Design Solutions.Cisco Press.

Karen Webb, Thomas M. Kelly, Karen Bagwell. 2000. BuildingCisco Multilayer Switched Networks. Cisco Press.

1999. CCIE Fundamentals: Network Design and Case Studies,Second Edition. Cisco Press.

www.cisco.com

www.cis.ohio-state.edu/hypertext/information/rfc.html

26

Documents

Best Practices Enterprise IP LAN-WAN Design