OTJNR Mod 01

Module 1: Product Positioning and Hardware Operation 1-1

Module 1: Product Positioning and Hardware Operation

Operation & Troubleshooting Juniper Networks Routers


Module Objectives

l After successfully completing this module, you will be able to:

– Match Juniper Networks, Inc. products with typical applications in a service provider network

– List the major components of each Juniper Networks router (M5, M10, M20, M40, M40e, M160, T640)

– Describe Field Replaceable Unit (FRU) level maintenance procedures

– Operate the Craft Interface– Describe how packets flow through a Juniper Networks

M-series router

This Module Discusses:– Juniper Networks, Inc. products and their typical applications;– Major router components;– General maintenance issues;– Operation of the Craft Interface; and– Packet flow through an M-Series router.


Juniper Networks Role in the Internet

l Where we are going…– Networking hardware evolution– Juniper Networks: the company– Juniper Networks product line: M-Series routers

§ M5 and M10 routers§ M20 router§ M40 router§ M40e router§ M160 router§ T640 Internet Routing Node

– Product comparison

Juniper Networks Role in the InternetThe following pages discuss the topics listed below.

– Networking hardware evolution: How did routers evolve from a single CPU architecture?

– Juniper Networks—the company: What are the business, mission, market and products of Juniper Networks?

– Juniper Networks product line: What is the Juniper Networks product line and how does each product compare?


Networking Hardware Evolution

l The first routers were general-purpose computers– Single CPU, RAM, monolithic operating system – Low-speed serial interfaces

l Networking advancements:– More PCs attached to networks – Increased application bandwidth consumption– Increased network speeds – Single-CPU router architecture could not keep up!

l Juniper Networks broke tradition with: – Specialized operating system

§ Protected memory, multi-tasking– Hardware-based packet forwarding

§ Juniper Networks routers implement key functions on ASICs

General-Purpose ComputersThe first routers were general-purpose computers with a single CPU, RAM, and an operating system. These early routers had many low-speed serial interfaces. Juniper Networks took this hardware architecture and modified it to include a specialized operating system and many more physical interfaces.

Networking AdvancementsIn addition to the advancements in router hardware, networking advancements spawned the growth of more PCs attached to networks, increased application bandwidth requirements, and increased link speeds. Sophisticated software developers were able to keep this pace for a period of time; however, the single-CPU router architecture could not.

Breaking TraditionTo accommodate growing technology trends, Juniper Networks began modifying the hardware architecture. The invention of application-specific integrated circuits (ASICs) allowed software functions to be integrated into hardware. Juniper Networks began integrating many functions into ASICs in order to achieve wire-rate forwarding.


Juniper Networks: The Company

l Business:– Converts optical bandwidth into scalable, differentiable IP

services using a new class of integrated silicon- and software-based routing systems

l Mission:– To be the primary supplier of scalable, reliable,

high-performance IP systems for the new IP infrastructure l Market:

– Supplies systems to numerous worldwide markets that provide high-speed IP services

l Products:– Juniper Networks routers offer wire-rate throughput ranging

from 5+ Gbps to 640+ Gbps– JUNOS Internet software is an integral part of the

high-performance routing architecture

BusinessJuniper Networks, Inc. converts optical bandwidth into scalable, differentiable IP services using a new class of integrated silicon- and software-based routing systems. These Internet routers are designed and built by industry experts who have broad experience in IP infrastructure applications, as well as in-depth understanding of the future direction of IP infrastructures. MissionThe Juniper Networks mission is to be the primary supplier of scalable, reliable, high-performance IP systems for the new IP infrastructure. MarketJuniper Networks supplies systems to numerous worldwide markets that provide high-speed IP services. Applications include backbone bandwidth management, multiservices, content and Web hosting, public and private peering, and high-speed access. Customers include many of the world's leading service providers, including Cable & Wireless plc, VERIO, Inc., and UUNET, an MCI WorldCom company.ProductsThe Juniper Networks routers offer wire-rate throughput ranging from 5+ Gbps to 640+ Gbps. Router interfaces provide market-leading port density with speeds ranging from E1 and T1 to OC-192c/STM-64. Juniper Networks packet-forwarding technology is based on high-performance ASICs designed using the most advanced technology available today. For example, the Internet Processor II™ ASIC not only delivers wire-rate forwarding performance and unprecedented visibility into network operations, it also supports packet filtering, sampling, logging, rate limiting, and load balancing. The JUNOS Internet software is an integral part of the high-performance routing architecture. Its design meets the requirements for IP network routing, operations, and control in the world's largest networks.


Juniper Networks Product Line

l Family of router platforms that deliver:– Wire-rate performance– Solutions that scale easily – Market-leading port density– Flexible and manageable control over traffic– Optimal bandwidth efficiency

Product Line and Driving ForceThe key application driving the new IP infrastructure is the Internet, which is growing exponentially. The Internet has moved from a convenience to a mission-critical platform for conducting and succeeding in business. As reliance on the Internet grows, so do customer expectations for value-added services that require increased bandwidth. There is little doubt that these rising demands for new services and for the Internet will set the standard for the emerging IP infrastructure.To help you remain competitive and grow with the new IP infrastructure, Juniper Networks has developed a family of router platforms that deliver wire-rate performance, solutions that easily scale, market-leading port density, flexible and manageable control over traffic, and optimal bandwidth efficiency. This Juniper Networks family consists of M5, M10, M20, M40, M40e, M160 and T640 Internet Routing Node with JUNOS Internet software.


Product Line at a Glance

March 2000

Packet Forwarding Performanceper Rack Inch

Sep. 1998 Dec. 1999

M40M20

M160

A Growing Historyof Rapid InnovationA Growing History

of Rapid Innovation

l Product line all has common:

– ASIC technology– JUNOS software– Architecture

Sep. 2000

M5/M10

Feb. 2002

M40e

April 2002

T640 Internet Routing Node

M-Series Product LineThe expanded Juniper Networks product portfolio includes five M-series Internet routers, described below. The main differences are throughput, size, and number of Physical Interface Cards (PICs) per chassis.

– The T640 Internet Routing Node is Juniper Networks largest router, offering an aggregate throughput of 320+ Gbps and supports up to 128 OC-48/STM-16, 32 OC-192/STM-64, or 128 Gigabit Ethernet ports for the router.

– The M160 Internet router offering an aggregate throughput of 160+ Gbps. It supports up to 32 OC-48c/STM-16 PICs or up to eight OC-192c/STM-64 PICs per chassis.

– The M40 Internet router provides more than 40 Gbps aggregate throughput and supports up to 32 PICs per chassis.

– The M40e Internet router delivers M40 throughput while providing redundant System Control Board and Routing Engine capabilities.

– The M20 Internet router also provides aggregate throughput of 20+ Gbps, supporting up to 16 PICs per chassis.

– The M10 Internet router supports up to eight PICs per chassis with an aggregate throughput of 10+ Gbps.

– The M5 Internet router supports up to four PICs per chassis with an aggregate throughput of 5+ Gbps.

All Juniper Networks platforms employ purpose-built ASICs, including the Internet Processor II ASIC, which has the unique ability to process packets across all interfaces without compromising performance. This capability is essential to delivering quality of service (QoS), as well as being able to run multiple, private IP services over the same infrastructure simultaneously.


M5 and M10 Routers

l M5 and M10 routers used for:– Edge applications involving the aggregation of dedicated

access circuits– Core applications in locations where space and power are at

a premium

M5 and M10 OverviewThe M5 and M10 Internet routers deliver high performance and highly flexible interfaces in a space- and power-efficient design. These routers are designed using the same architecture, ASICs, and JUNOS Internet software as the already proven M-series Internet routers. This Internet-tested core technology is now available at your network edge, along with value-added services, such as packet filtering and sampling.The oversized forwarding performance of the Internet Processor II ASIC provides wire-speed forwarding with plenty of headroom. In fact, the M5 and M10 routers forward packets at an aggregate throughput rate of 5+ Gbps and 10+ Gbps, respectively.The M5 and M10 platforms are ideal for edge applications involving the aggregation of dedicated access circuits. They are also ideal for core applications in locations where space and power are at a premium, such as smaller metro POPs. The ability to connect a wide range of high-performance interfaces from T1 and E1 through OC-12c/STM-4 ensures that you can scale the network easily and cost efficiently.


M20 Router

l M20 router built for a variety of Internet applications:– High-speed access– Public and private peering– Hosting sites– Backbone core networks

M20 OverviewThe M20 Internet router is a high-performance routing platform built for a variety of Internet applications, including high-speed access, public and private peering, hosting sites, and backbone core networks.The M20 router leverages proven M-series ASIC technology to deliver wire-rate performance and rich packet processing, such as filtering, sampling, and rate limiting. It runs the same JUNOS Internet software and supports the same interfaces as other M-series routers, providing a seamless upgrade path that protects your investment. Its compact design (14 in./35.56 cm high) delivers market-leading performance and port density, while consuming minimal rack space.


M40 Routerl M40 router:

– Designed specifically for the needs of high-growth service providers

– Delivers the bandwidth required to grow networks to OC-48c/STM-16 speeds cost efficiently

M40 OverviewThe M40 Internet router is designed specifically for the specialized needs of high-growth Internet backbone providers. It features market-leading packet forwarding performance, unparalleled port density and flexibility, and best-in-class JUNOS Internet software. The router delivers the bandwidth required to grow backbones to OC-48c/STM-16 speeds, while providing rich packet processing and flexible Multiprotocol Label Switching (MPLS) tools for increased bandwidth efficiency and greater control over traffic.


M40e Router

l M40e router delivers:– Full hardware redundancy– The bandwidth required to grow

networks from DS-0 to OC-48c/STM-16 speeds cost efficiently

M40e OverviewLike the M40, the M40e Internet router is designed specifically for the specialized needs of high-growth Internet backbone providers while forwarding traffic at a rate of 40+ Gbps It is targeted primarily for the edge aggregation market. The M40e provides Routing Engine and Packet Forwarding Engine redundancy and the ability to hot-swap PICs.


M160 Routerl M160 router:

– Ideal for large networks requiring predictable performance for feature-rich infrastructures

– Built for large backbone cores, with features enabled for future migration to the backbone edge

M160 OverviewThe M160 Internet router is the first full-performance OC-192c/STM-64 platform. By providing aggregate forwarding rates of 160 Mpps and aggregate throughput exceeding 160 Gbps, this platform enables you to grow networks rapidly and reliably while taking advantage of increased optical bandwidth.An ASIC-based packet forwarding path enables full wire-rate forwarding over 10-Gbps circuits. Key to this performance is the Internet Processor II ASIC and the SONET Manager II ASIC.The M160 router offers a complete range of flexible and dense interfaces, allowing you to fit M160 routers into existing environments seamlessly. The platform supports up to 32 OC-48c/STM-16 Physical Interface Cards (PICs) per chassis (64 per standard rack) or up to 8 OC-192c/STM-64 PICs per chassis (16 per rack).The M160 router provides a scaleable solution using the same JUNOS Internet software, using the same ASICs, and supporting the same value-added services as the other M-series Internet routers.



l Goal: First system optimized for dense 10 Gbps – Density– Performance– Scalability– Serviceability– Reliability

T640 Internet Routing Node OverviewThe T640 Internet Routing Node Internet backbone router is a complete routing system that provides Gigabit Ethernet, SONET/SDH, and other high-speed interfaces for large networks and network applications, such as those supported by Internet service providers (ISPs). Application-specific integrated circuits (ASICs) are a definitive part of the router design; these ASICs enable the router to achieve data forwarding rates that match current fiber-optic capacity.The router accommodates up to eight Flexible PIC Concentrators (FPCs), which can each be configured with a variety of network media types—altogether providing up to 128 OC-48/STM-16, 32 OC-192/STM-64, or 128 Gigabit Ethernet ports for the router.In a stand-alone configuration, the router’s maximum aggregate throughput is 320 Gbps, full duplex. The router can forward traffic at line rate for any combination of PICs that does not exceed 40 Gbps on a single FPC3 (or 10 Gbps on an FPC2). The T640 supports any combination exceeding 40 Gbps, but this constitutes over subscription.The T640 Internet Routing Node is designed for the core of large service provider networks. It is targeted for roles requiring a high density of 10 Gbps interfaces.


Product Comparison

2 per rack

DC Only

8

12.8 Gbps

160+ Gbps

M160

2 per rack

DC only

8

40+Gbps

640+Gbps

T640 Internet Routing

Node

2 per rack

AC/DC

8

3.2 Gbps

40+ Gbps

M40e

15 per rack

AC/DC

1

3.2 Gbps

5+ Gbps

M5

15 per rack

AC/DC

2

3.2 Gbps

10+ Gbps

M10

2 per rack

AC/DC

8

3.2 Gbps

40+ Gbps

M40

5 per rackSize

AC/DCPower

4Slots

3.2 GbpsSlot Bandwidth

20+ GbpsFull Duplex Bandwidth

M20Feature

Product ComparisonOn all Juniper Networks routers, all slots can run at full bandwidth simultaneously. The model number for each of the routers is based on the full duplex bandwidth capabilities of that router. For example, an M40 is capable of processing 40 million packets per second. Also, all routers support both AC/DC power (but not both simultaneously), except the M160 and the T640, which support DC power only.


Juniper Networks Hardware Overview

l Where we are going…– General router architecture

§ Routing Engine§ Packet Forwarding Engine§ Relationship between Routing Engine and Packet Forwarding Engine

– Hardware overview§ Routing Engine§ Packet Forwarding Engine§ System Midplane§ Flexible PIC Concentrator (FPC)§ Physical Interface Card (PIC)§ System Control Boards (SCBs)

Juniper Networks Hardware OverviewThe following pages discuss the topics listed below.

– General router architecture: The general router architecture consists of the Routing Engine and the Packet Forwarding Engine. The Routing Engine and Packet Forwarding Engine interact through an Ethernet link to allow for communication.

– Hardware overview: The main hardware components of the M-series routers are the Routing Engine(s), Packet Forwarding Engine, FPC(s), PIC(s) and SCBs. On the M5/M10 the FPC and System Control Boards are combined.


l All Juniper Networks routers share the same basic design– Routing Engine– Packet Forwarding Engine

l Clean separation of control and forwardingl Routing Engine maintains routing table (RT) and primary copy of

forwarding table (FT)l Packet Forwarding Engine receives FT from Routing Engine

Routing Engine/Packet Forwarding Engine Interaction

Packet ForwardingEngine

Routing Engine

fxp1

Packets In Packets Out

FT RT

FT

JUNOS

CLI

Router ArchitectureArchitecturally, all Juniper Networks routers have two major components, which are described below.

– Packet Forwarding Engine: Forwards packets through the router. It is a high-performance switch capable of forwarding up to 160 Mpps in the M160 router for all packet sizes.

– Routing Engine: Performs routing updates and system management. It consists of routing protocol software processes running inside a protected memory environment on a general-purpose computer platform with a direct 100 Mbps connection to the Packet Forwarding Engine.

Separation of Control and ForwardingBecause this architecture separates control operations–such as routing updates and system management–from packet forwarding, the router can deliver superior performance and highly reliable Internet operation.

Routing and Forwarding Table InteractionJUNOS software routing protocol process controls the routing protocols. The routing protocol process starts all configured routing protocols and handles all routing messages. It maintains one or more routing tables, which consolidates the routing information learned from all routing protocols into common tables. From this routing information, the routing protocol process determines the active routes to network destinations and installs these routes into the Routing Engine’s forwarding table (FT).Continued on next page.


Routing Engine and Packet Forwarding Engine SynchronizationNext, the Packet Forwarding Engine receives the forwarding table from the Routing Engine. The Packet Forwarding Engine’s FT and the Routing Engine’s FT are always synchronized over the 100 Mbps Ethernet link, called fxp1. This synchronization ensures that a change in topology produces identical FTs in the Routing Engine and Packet Forwarding Engine.


Routing Engine Overview

l JUNOS software resides in flash memory– Backup copy available on hard drive

l Provides routing protocol intelligence to Packet Forwarding Engine

– Not directly involved with packet forwarding– Runs various routing protocols

l Implements command-line interface (CLI)l Manages Packet Forwarding Engine

JUNOS SoftwareThe primary copy of JUNOS software resides on the flash memory of the router. A backup copy is available on the hard drive. Also, you can keep an image on external media.

Routing Engine IntelligenceThe Routing Engine handles all the routing protocol processes, as well as other software processes that control the router’s interfaces, a few of the chassis components, system management, and user access to the router. These routing and software processes run on top of a kernel that interacts with the Packet Forwarding Engine.All routing protocol packets from the network are directed to the Routing Engine; hence, they do not delay the Packet Forwarding Engine unnecessarily.

Command-Line InterfaceThe Routing Engine provides the command-line interface (CLI). The CLI runs on top of the kernel; it is controlled by a software daemon. The CLI and its features are examined in more detail in future slides.

Packet Forwarding Engine ManagementThe Routing Engine controls the Packet Forwarding Engine by providing an accurate and up-to-date forwarding table as well as managing software daemons in the Packet Forwarding Engine microcode.


Routing Engine Characteristics

Fast, Intel-based, compact PCI platform

PCMCIAflash card

30+GB

128 MB

2 GB

600 MHz


PCMCIAflash cardExternal media

6.4+GBHard disk storage

80 MBSolid state flash storage

768 MBMemory

333 MHzProcessor clock

M5/10/20/40/40e160Feature

Routing Engine SpecificationsThe Routing Engines of the different router models have the same specifications, with the exception of the older M40 model. The legacy M40 model had a different clock speed, memory size, and external media than the other models. However, it can be upgraded to meet the current specifications. Current M40 routers ship with the same specifications as all other M-series routers.


Routing Engine Redundancy

l Configuring master and back-up Routing Engines – Capability available on M20, M40e, and M160 only– By default, Routing Engine in slot 0 is the master and slot 1 is the

back-up

set chassis redundancy routing-engine (0|1)(master|backup|disabled)

– Default failover is 5 minutes, range is 300-10,000 seconds

redundancy { failover on-loss-of-keepalives; keepalive-time seconds; routing-engine slot-number (master | backup | disabled); }

Default Settings and CaveatsFor routers with two Routing Engines (currently M20, M40e, and M160), you can configure which Routing Engine is the master and which is the backup. By default, the Routing Engine in slot 0 is the master and the one in slot 1 is the backup. To switch between the master and the backup Routing Engines, you must modify the configuration and then activate the configuration by issuing the commit command. For routers that have two Routing Engines, both Routing Engines must be running JUNOS Internet software Release 4.0 or later. Do not run JUNOS Internet software Release 3.4 on one of the Routing Engines and Release 4.0 on the other. (Note that Release 3.4 does not support Routing Engine redundancy, so if you are using this release of the software, only one Routing Engine can be installed in the router. It can be installed in either slot.) If you have Release 3.4 installed on one of the Routing Engines and Release 4.0 or later on the other, either remove the backup Routing Engine from the router or install Release 4.0 or later on that Routing Engine.


Routing Engine Timeline

l The sequence of events is: – After x seconds of keepalive loss, a message is logged– After another x seconds passes, the backup Routing Engine

trys to assume master status– An alarm is generated when backup is active– You must switchover Routing Engine manually if you want

the previous master to become active again

Sequence of EventsThe visual shows the Routing Engine’s sequence of events.


Routing Engine Synchronization

l To make tty connection to the other Routing Engine using the router's internal Ethernet network, issue the following command:

user@host> request routing-engine login re0 user@host> file copy /config/juniper.conf re1:/var/tmp/copied-

juniper.confuser@host> load replace /var/tmp/copied-juniper.conf

– Make sure you do not have identical IP addresses specified on fxp0 for re0 and re1!

l Recent versions of software use commit synchronize

Configuring File SynchronizationYou must ensure that both Routing Engines have the same configuration file. You can use either the console port or the management Ethernet (fxp0) port to establish connectivity between the two Routing Engines. You can then copy or FTP the configuration from the master to the backup, load the file, and commit it in the normal way. Make sure you change any IP addresses specified in fxp0 on Routing Engine 0 to addresses appropriate for Routing Engine 1. You can use configuration groups to ensure that the correct IP addresses are used for each Routing Engine and maintain a single configuration file for both Routing Engines. These pre-defined configuration group names take effect only on the appropriate Routing Engine.

commit synchronizeIf your router has two Routing Engines, you can direct one Routing Engine to synchronize its configuration with the other manually by issuing the commit synchronize command. The Routing Engine on which you execute this command(request Routing Engine) copies and loads its candidate configuration to the other (responding Routing Engine). Both Routing Engines then perform a syntax check on the candidate configuration file being committed. If no errors are found, the configuration is activated and becomes the current operational configuration on both Routing Engines.When you issue the commit synchronize command, you must use apply groups re0 and re1. The responding Routing Engine must use JUNOS release 5.0 or higher.


Process Redundancy

If software process fails three times in quick succession, switch to back-up source (other Routing Engine or alternative media)[edit system processes]lab@aurora# set ?Possible completions:> alarm-control Alarm control process+ apply-groups Groups from which to inherit configuration data> chassis-control Chassis control process> craft-control Front panel I/O control process> inet-process Inet process> interface-control Interface control process> mib-process MIB process> ntp Network time process> routing Routing process> sampling Traffic sampling control process> snmp SNMP process> watchdog Watchdog timerlab@aurora# set snmp failover ?Possible completions:

alternate-media On failing, reboot off alternate mediaother-routing-engine On failing, switch mastership to other Routing Engine

Ensuring Software Process Fail-OverFor routers with redundant Routing Engines only, switch to backup media if a process fails repeatedly. If a process fails three times in quick succession, the router reboots from the alternative media or from the other Routing Engine.


Packet Forwarding Engine Overview

l Custom ASICs– Implement forwarding path– Do not require a general-purpose processor– Provide integrated features, including multicast and special

queuingl Divide-and-conquer architecture

– Each ASIC provides a piece of the forwarding puzzle

Custom ASICSASICs enable the router to achieve data forwarding rates that match current fiber-optic capacity. Such high forwarding rates are achieved by distributing packet-processing tasks across four highly integrated ASICs. As a result, Juniper Networks routers do not require a general purpose processor, and the custom ASICs provide integrated features such as multicast and queuing.

Divide-and-Conquer ArchitectureEach ASIC provides a piece of the forwarding puzzle, allowing a single ASIC to perform its specific task optimally. Each ASIC’s function in forwarding is examined on future slides.


Internet Processor II ASIC

l Internet Processor I ASIC– Provides line-rate, longest-match packet lookup

l Internet Processor II ASIC– Adds packet-processing features: filtering, sampling, logging,

counting, and improved load balancing– Standard equipment on the M5, M10, M40e, and M160

platforms§ The Internet Processor II has been standard on the M20 and M40 since

the second quarter of 2000§ Field upgrade for IP I ASIC-equipped platforms

Internet Processor IThe Internet Processor ASIC, which first shipped with the M40 Internet router in September 1998, provides breakthrough technology that allows routers to forward traffic at wire-rate speeds. Performance tests in the lab, test networks, and on the Internet itself show a benchmark of forwarding 40-byte packets at 40 Mpps with an 80,000-prefix routing table.In addition to wire-rate forwarding, this ASIC supports industrial-strength routing protocol implementations, a routing policy definition language, reliable performance under stress, flexible MPLS traffic engineering, and class-of-service (CoS) queuing.

Internet Processor IIThe Internet Processor II ASIC continues to deliver best-of-class functionality in the network core. Juniper Networks is the first company to deliver these capabilities, and the M-series routers are the highest-performing systems on the market today.The experience of interacting with the world's largest service providers gives Juniper Networks a unique advantage in developing products and features that meet the increased requirements of the world's busiest and fastest networks. Juniper Networks translated the experience gained in the implementation and deployment of the first-generation Internet Processor ASIC into an expanded and scalable feature set on the Internet Processor II ASIC. Not only does the Internet Processor II ASIC deliver a 40 Mpps forwarding rate, it also adds the packet processing features you need to build a competitive advantage in a rapidly evolving industry. Offering rich packet processing, such as filtering, sampling, logging, counting, and load balancing, the Internet Processor II ASIC maintains high performance.The Internet Processor II ASIC is standard on the M5, M10, M40e, and M160 Internet routers and is optional on the M20 and M40 Internet routers. With the M-series centralized architecture, you only need to download software to activate Internet Processor II ASIC features on all interfaces.


Packet Forwarding Engine Components

l Physical Interface Card (PIC)– Provides Physical Layer interface and signaling

l Flexible PIC Concentrator (FPC)– Carrier that accepts PICs

l System Midplanel Control

– M5 and M10—Forwarding Engine Board (FEB): combined FPC and System Control Board

– M20—System Switching Board (SSB)– M40—System Control Board (SCB)– M40e and M160—Switching and Forwarding Module (SFM)

Physical Interface CardJuniper Networks M-series Internet routers provide a complete range of fiber optic and electrical transmission interfaces to the network. These space-efficient routers offer exceptional flexibility and leading port density by supporting a wide array of Physical Interface Cards (PICs). PICs provide a complete range of fiber optic and electrical transmission interfaces to the network.

Flexible PIC ConcentratorFlexible PIC Concentrators (FPCs) house the PICs and are installed in slots in the M20, M40, M40e, and M160 Internet routers. These intelligent, high-performance interface concentrators offer unparalleled interface density and flexibility.

System MidplaneThe System Midplane is the component of the Packet Forwarding Engine that distributes power and electrical signals to each card in the system.The Midplane forms the back of the card cage. All cards are installed into the System Midplane from the front of the chassis. The Routing Engine plugs into the Midplane from the rear. This Midplane is passive in all routers except the M40, which contains Distributed Buffer Manager ASICs.

System Control BoardsThe System Control Boards provide the route look-up component of the Packet Forwarding Engine. Each System Control Board on the M-series routers provide the same function although they have different names. The M5 and M10 routers however, contain the FPC components and the control components on the same physical circuit board, called the Forwarding Engine Board (FEB).


System Midplane

l M10 System Midplane– Built-in FPCs– Eight PIC slots

l M20 System Midplane– System Switching Board slot– Craft Interface slot– Four FPC slots

3210

Craft Interface

Primary SSBSecondary SSB

10

M5/M10 System MidplaneThe M10 and M5 System Midplane contain built in FPCs–two for the M10 and one for the M5. Each FPC can hold four PICs. Thus, the M10 can have up to eight PICS, and the M5 can have up to four PICs.

M20 System MidplaneThe M20 System Midplane can hold up to four FPC slots (0-3 counting from top to bottom). This Midplane also contains the System Switching Board slot as well as the Craft Interface slot.


System Midplane (Contd.)

l M40 System Midplane– Eight FPC slots– System Control Board slot– Distributed Buffer Manager

ASICs

l M40e and M160 System Midplane

– Connector Interface Panel (CIP)– Eight FPC slots

0 1 2 3 4 5 6 7

Con

nector In

terface Pan

el

SCB0 1 2 3 4 5 6 7

M40 System MidplaneThe M40 System Midplane can hold up to eight FPC slots (0-7 counting from left to right). This Midplane also contains the System Control Board (SCB) slot. In addition, the M40 router System Midplane provides the features listed below, which were later incorporated into the Packet Forwarding Engine control systems on all other routers.

– Management of shared memory on the FPCs: The Distributed Buffer Manager ASIC on the M40 router backplane uniformly allocates incoming data packets throughout shared memory on the FPCs.

– Transfer of outgoing data cells to the FPCs: A second Distributed Buffer Manager ASIC on the backplane passes data cells to the FPCs for packet reassembly when the data is ready to be transmitted.

M40e and M160 System MidplaneThese Midplanes can hold up to eight FPCs (0-7 counting from left to right). This Midplane also contains the connector interface panel slot.


Flexible PIC Concentratorl FPC features

– Four PIC connectors– Hot-swappable– PowerPC supervisory processor

§ Does not perform packet forwarding

l Built-in FPCs on M5 and M10l High throughput rates (full duplex)

– M5 router—6.4 Gbps– M10 router—6.4 Gbps per FPC– M20, M40, and M40e routers—6.4 Gbps

per FPC§ Four FPCs in an M20 router§ Eight FPCs in M40 and M40e routers

– M160 router—25.6 Gbps per FPC§ Eight FPCs per router§ Two types of FPCs

l 64, 128 or 256 MB packet memory– Pooled with other slots

Buf

fer

mem

ory

ASIC

FPC

PIC

PIC

PIC

PIC

FPC OverviewFPCs install into the backplane from the front of the chassis. Any FPC can be installed into any FPC slot; a specific order is not required. If an FPC does not occupy a slot, you must install a blank FPC carrier to shield the empty slot so that cooling air can circulate properly through the card cage. Each FPC has four connectors into which a PIC can be installed, yielding up to four PICs per FPC. When you install an FPC into a running system, the FPC requests its operating software from the Routing Engine, the FPC runs its diagnostics, and the PICs on the FPC slot are enabled. FPCs are hot-insertable and hot-removable. Each FPC is mounted on a card carrier. When you remove an FPC and install a new one, the backplane flushes the entire system memory pool before the new card is brought online, a process that takes about 200 milliseconds.

M5/M10 versus Other M-Series RoutersThe M5 and M10 routers have FPCs built into the chassis. PICs for the M20, M40, and M160 routers require FPCs. Each FPC holds four single-wide PICs. The exceptions are the M20 and M40 OC-48c/STM-16 and four-port Gigabit Ethernet PICs, and the M160 OC-192c/STM-64 SONET/SDH PIC, which are quad-wide and do not require an FPC.

Industry-Leading ThroughputThe M-series routers have a full duplex slot bandwidth of 6.4 Gbps per slot except for the M160 Router. This router has an increased full duplex bandwidth of 25.6 Gbps to allow for the OC-192 cards.Continued on next page.


Shared MemoryThe FPCs connect the PICs to the rest of the Packet Forwarding Engine so that incoming packets can be forwarded across the backplane to the appropriate destination port. FPCs contain shared memory, which is managed by the Distributed Buffer Manager ASIC, for storing data packets received by the PICs. The I/O Manager ASIC on each FPC sends packets from the PICs to the Distributed Buffer Manager ASICs.


PIC

l PICs support from 0 to 4 physical ports

– Some PICs support channelized options

– Tunnel PIC and Multilink Services PIC do not have any ports

l Custom ASIC for each media type

l Each port has status LED

l Hot-swappable on all M-series routers, except M20 and M40 routers

Physical Physical Interface Interface

CardCard

PICPIC

PICPIC

PICPIC

FPC

Buf

fer

Buf

fer

mem

ory

mem

ory

ASICASIC

PIC OverviewYou can install up to four PICs into slots on each FPC. Any PIC can be installed in any PIC slot. PICs provide the physical connection to various network media types. PICs receive incoming packets from the network and transmit outgoing packets to the network. During this process, each PIC performs appropriate framing and signaling for its media type. Before transmitting outgoing data packets, the PICs encapsulate the packets received from the FPCs. The Tunnel PIC and Multilink PICs do not contain ports. The Multilink PIC uses the Multilink Point-to-Point Protocol (MLPPP) and Multilink Frame Relay (MLFR, FRF 1.5) to group up to eight T1 or E1 links per bundle. The result is a service offering ranging from 1.5 Mbps through 12 Mbps (T1) or 2 Mbps through 16 Mbps (E1). With the Tunnel Services PIC, the routers can function as the ingress or egress point of an IP-IP unicast tunnel, and generic routing encapsulation (GRE) tunnel, or a Protocol Independent Multicast-Sparse Mode (PIM-SM) tunnel.

Custom ASICEach PIC is equipped with an ASIC, which performs control functions tailored to the PIC’s media type. For instance, an ATM PIC and a Fast Ethernet PIC each contain a unique ASIC.

LEDsEach PIC port has an LED that indicates the state of the port. Red indicates a serious problem that prevents traffic from flowing through that port. Yellow indicates that traffic is flowing but some of it might not be transmitted or received correctly. Green indicates a correctly functioning port.

Field ReplacementPICs are field replaceable. However, on the M40 and M20 routers, you must first remove the host FPC, which is hot-removable and hot-insertable. On the M160, M40e, M10, and M5 routers, you can remove a PIC without removing its FPC.


Control Systems

l All models– 200 MHz PowerPC 603e processor

§ Manages forwarding table updates§ Manages ASICs and environmental systems

– 64 MB EDO processor RAM– 8 MB of forwarding table SRAM– Internet Processor ASIC– Stratum 3 synchronization reference

l All except M40 router– Distributed Buffer Manager ASICs

The System Control Board’s Major Functions– Management of shared memory on the FPCs: The Distributed Buffer

Manager ASIC on the System Control Board uniformly allocates incoming data packets throughout shared memory on the FPCs. Note that the M40 SCB does not have a Distributed Buffer Manager ASIC.

– Route lookups: The Internet Processor ASIC on the System Control Board performs route lookups using the forwarding table stored in the synchronous SRAM (SSRAM). After performing the lookup, the Internet Processor ASIC informs the Midplane of the forwarding decision, and the Midplane forwards the decision on to the appropriate outgoing interface.

– Monitoring system components: The System Control Board monitors other system components for failure and alarm conditions. It collects statistics from all sensors in the system and relays them to the Routing Engine, which sets the appropriate alarm. For example, if a temperature sensor exceeds the first internally defined threshold, the Routing Engine issues a high tempalarm. If the sensor exceeds the second threshold, the Routing Engine initiates a system shutdown.

– Transferring exception and control packets: The Internet Processor ASIC passes exception packets to a microprocessor on the System Control Board, which processes almost all of them. The remaining packets are sent to the Routing Engine for further processing. Any errors originating in the Packet Forwarding Engine and detected by the System Control Board are sent to the Routing Engine using syslog messages.

Continued on next page.


The System Control Board’s Major Functions (contd.)– Controlling FPC resets: The System Control Board monitors the operation of

the FPCs. If it detects errors in an FPC, the System Control Board attempts to reset the FPC. After three unsuccessful resets, the SSB takes the FPC offline and informs the Routing Engine. Other FPCs are unaffected, and normal system operation continues. Note that the M5 and M10 routers do not have separate FPCs.

M40 CaveatThe M40’s Distributed Buffer Manager ASICs are contained on the System Midplane and not the System Control Board.


Platform Specifics

l Where we are going…– M5/10 components and connectors– M20 components– M40 components– M40e components– M160 components– T640 components

Platform SpecificsAlthough all of the routers in the M- and T-series have many similarities, there are enough differences between the various models to warrant discussion. The following pages discuss the platform specifics of the M5/10, M20, M40, M40e, M160 and T640 Internet Routing Node.


The M5 and M10 Routers

l Base chassis– Dual AC or DC power supplies

l Routing Engine– Processor– Solid-state PCMCIA storage media

l Packet Forwarding Engine– FEB has combined Control and FPC Boards – PICs

Power SuppliesThe maximum chassis power is 340 watts for M5 routers and 434 watts for M10 routers. Power supplies can be either AC or DC (but not both simultaneously); when two are installed, power is load balanced.

Routing EngineThe Routing Engine maintains the routing tables and controls the routing protocols, as well as the JUNOS software processes that control the router's interfaces, the chassis components, system management, and user access to the router. These routing and software processes run on top of a kernel that interacts with the Packet Forwarding Engine.

Packet Forwarding EngineThe Packet Forwarding Engine provides Layer 2 and Layer 3 packet switching, route lookups, and packet forwarding. The Internet Processor II ASIC forwards up to 40 Mpps for all packet sizes. The throughput is 5+ Gbps in an M5 router and 10+ Gbps in an M10 router.The Packet Forwarding Engine supports the same ASIC-based features supported by all other M-series routers. For example, CoS features include rate limiting,classification, priority queuing, random early detection (RED), and weighted round robin (WRR) to increase bandwidth efficiency. Filtering and sampling are also available for restricting access, increasing security, and analyzing network traffic.Finally, the Packet Forwarding Engine delivers maximum stability during exceptional conditions, while also providing a significantly lower part count. This stability reduces power consumption and increases mean time between failure (MTBF).


M5 and M10 Router Components

RoutingEngine

PICs

PowerSupplies

PCMCIAFlash Card

Fan

CraftInterface

M5 and M10 Internet Routers Hardware ComponentsBelow is a list of the hardware components for the M5 and M10 routers.

– Sheet metal chassis– Two power supplies, AC or DC– Fan assembly– Routing Engine– Forward Engine Board– Four (M5) or eight (M10) PICs


M5 and M10 Router Connectors

l Ports:– Management Ethernet

§ Provides access to router from POP administrative network

– Console§ Provides direct access to

JUNOS software

– Auxiliary console§ Provides modem access to

JUNOS software

PortsBelow is a list of the M5 and M10 router connectors.

– Ethernet management port: Connects the Routing Engine to an out-of-band management network.

– Console port: Connects a system console to the Routing Engine with EIA/TIA-232 serial asynchronous cable. Use the system console to access the command-line interface (CLI) to configure the attached router. This portis active by default.

– Auxiliary port: Connects a laptop or modem to the Routing Engine with EIA/TIA-232 serial asynchronous cable. You can also use this port to access the CLI. This port is disabled by default.


The M20 Router

l Base chassis– Dual AC or DC power supplies– 4 FPC slots

l Redundant Routing Engines– Processor– Solid-state PCMCIA storage media

l Packet Forwarding Engine– Redundant System Switching Boards (SSBs)– PICs– FPCs

Power SuppliesThe maximum chassis power is 24 A at -48 VDC, or 12 A at 120 VAC, 6 A at 240 VAC. Power supplies can be either AC or DC (but not both simultaneously); when two are installed, power is load balanced.

Routing EngineThe Routing Engine maintains the routing tables and controls the routing protocols, as well as the JUNOS software processes that control the router's interfaces, the chassis components, system management, and user access to the router. These routing and software processes run on top of a kernel that interacts with the Packet Forwarding Engine. You can install a redundant Routing Engine to ensure maximum system availability and to minimize mean time to repair (MTTR) in case of failure.

Packet Forwarding EngineThe Packet Forwarding Engine provides Layer 2 and Layer 3 packet switching, route lookups, and packet forwarding. The Internet Processor II ASIC forwards up to 40 Mpps for all packet sizes. The throughput is 20+ Gbps.The Packet Forwarding Engine supports the same ASIC-based features supported by all other M-series routers. For example, CoS features include rate limiting,classification, priority queuing, RED and WRR to increase bandwidth efficiency. Filtering and sampling are also available for restricting access, increasing security, and analyzing network traffic.Finally, the Packet Forwarding Engine delivers maximum stability during exceptional conditions, while also providing a significantly lower part count. This stability reduces power consumption and increases MTBF.


M20 Router Components

FrontBack

Primary SSB

RoutingEngines

FPCsPowersupplies

PCMCIAflash card

Fans

Craftinterface

0123

Secondary SSB

M20 Hardware ComponentsThe hardware components for M20 routers include:

– Sheet metal chassis;– Two power supplies (AC or DC);– Four fan assemblies;– One or two Routing Engines;– One or two SSBs; and– Up to four FPCs, each populated with up to four PICs for various interface

types, including OC-48, OC-12, and OC-3 SONET/SDH (including channelized OC-12); OC-12 and OC-3 ATM; and DS3, Gigabit Ethernet, and Fast Ethernet.


M5, M10, And M20 Cooling

l M20, M10, and M5 routers cool from side to side

– M20 router§ Three identical fan trays in

front§ One unique tray cools REs

in rear§ Integrated power supply

fans– M10 and M5 routers

§ Single fan tray

M20Router

Front view

Side-to-Side CoolingThe M5, M10, and M20 routers’ cooling systems are composed of one to four cooling trays. The M20 router's cooling system consists of the subsystems listed below.

– Front cooling subsystem: Three front fan trays that cool the FPCs and the SSB. These fan trays are located on the left front side of the chassis.

– Rear cooling subsystem: One rear fan tray that cools the Routing Engine. This fan tray is located immediately to the right of the Routing Engine.

– Power supply integrated fan: A built-in fan that cools each power supply.The four fan trays work together to provide side-to-side cooling. The fan trays plug directly into the router Midplane. Each front fan tray is a single, field-replaceable unit that contains three fans. The rear fan tray is a field-replaceable unit that contains two fans. Both front and rear fan trays are hot-swappable.The M5 and M10 routers’ cooling systems consist of a fan tray located along the side of the chassis, which provides side-to-side cooling. The fan tray is a single unit containing four fans. It is hot-removable and hot-insertable. It connects directly to the router Midplane.


The M40 Router

l Base chassis– Dual AC or DC power supplies– 8 FPC slots

l Routing Engine– Processor– LS-120 floppy disk storage media

l Packet Forwarding Engine– SCB– PICs– FPCs

Power SuppliesThe maximum chassis power is 35 A at -48 V. Power supplies can be either AC or DC (but not both simultaneously); when both are installed, power is load balanced.

Routing EngineThe Routing Engine maintains the routing tables and controls the routing protocols as well as the JUNOS software processes that control the router's interfaces, the chassis components, system management, and user access to the router. These routing and software processes run on top of a kernel that interacts with the Packet Forwarding Engine.

Packet Forwarding EngineThe Packet Forwarding Engine provides Layer 2 and Layer 3 packet switching, route lookups, and packet forwarding. The Internet Processor II ASIC forwards up to 40 Mpps for all packet sizes. The throughput is 40+ Gbps.The Packet Forwarding Engine supports the same ASIC-based features supported by other M-series routers. For example, CoS features include rate limiting, classification, priority queuing, RED, and WRR to increase bandwidth efficiency. Filtering and sampling are also available for restricting access, increasing security, and analyzing network traffic.Finally, the Packet Forwarding Engine delivers maximum stability during exceptional conditions, while also providing a significantly lower part count. This stability reduces power consumption and increases MTBF.



Front Back

Fans

SCBRoutingEngine

FPCs

PowerSupplies

LS-120

Fans

CraftInterface

CableManagement


– Sheet metal chassis;– Two power supplies (AC or DC);– Two impeller trays;– Three fan assemblies;– One Routing Engine;– One SCB; and– Up to eight FPCs, each populated with up to four PICs for various interface



The M40e Router

l Base chassis– Redundant power supplies– 8 FPC slots


l Packet Forwarding Engine– SFMs– PICs– Three types of FPCs

§ FPCe: 4 x M20/M40 PICs § FPC 1: 4 x medium-speed PICs§ FPC 2: 1 x high-speed PICs

– Redundant control systems

Power SuppliesThe maximum chassis power is 2600 watts with redundant, load-sharing DC power supplies.

Routing EngineThe Routing Engine maintains the routing tables and controls the routing protocols, as well as the JUNOS software processes that control the router's interfaces, the chassis components, system management, and user access to the router. These routing and software processes run on top of a kernel that interacts with the Packet Forwarding Engine. You can install a redundant Routing Engine to ensure maximum system availability and to minimize MTTR in case of failure. Each Routing Engine requires an adjacent miscellaneous control subsystems (MCS).



M40e Router Components

Front

Back

M40e Hardware ComponentsThe hardware components for M40e routers include:

– Sheet metal chassis;– Redundant cooling;– Two Routing Engines;– Two SFMs; and– Up to eight FPCs, each populated with up to four PICs for various interface



The M160 Router

l Base chassis– DC only power supplies– 8 FPC slots


l Packet Forwarding Engine– Redundant SFMs– PICs– Two types of FPCs

§ FPC 1: first-generation PICs § FPC 2: high-density M160 PICs

– Redundant control systems

Power SuppliesThe maximum chassis power is 2600 watts with redundant, load-sharing DC power supplies.

Routing EngineThe Routing Engine maintains the routing tables and controls the routing protocols as well as the JUNOS software processes that control the router's interfaces, the chassis components, system management, and user access to the router. These routing and software processes run on top of a kernel that interacts with the Packet Forwarding Engine. You can install a redundant Routing Engine to ensure maximum system availability and to minimize MTTR in case of failure. Each Routing Engine requires an adjacent MCS.




REsMCSsPCGs

PEMs

SFMs

Circuitbreakers

Fan

Fan

FPCs

Craftinterface

CIP

Airfilter


– Sheet metal chassis;– Two DC-only Power Entry Modules (PEMs);– Cooling components;– Redundant Routing Engines;– Redundant SFMs;– Up to eight FPCs, each populated with up to four PICs for various interface

types; – Connector Interface Panel (CIP);– Redundant MCSs, which provide redundant control of chassis power,

environmental control systems, online/offline of system components and Stratum 3 synchronization reference; and

– Redundant Packet Forwarding Engine Clock Generators (PCGs), which provide redundant synchronization signals to the hardware components in the Packet Forwarding Engine.


FPC1 and FPC2

Two Types of FPCs– FPC1: Supports single-port OC-12 and Gigabit Ethernet PICs.– FPC2: Supports higher-speed OC-48 and Tunnel PICs.

The router can operate with any combination of FPC1s and FPC2s installed.


M40, M40e, and M160 Cooling

l M40, M40e and M160 routers use front-to-rear cooling

l Field-replaceable:– Air filter– Fan components

l Fan speed and temperature sensors monitored by JUNOS software

M160Router

Front-to-Rear CoolingThe router’s cooling system consists of two separate subsystems–front and rear. The front cooling subsystem consists of an upper impeller and a lower fan tray; it cools the FPCs, the PICs and the Midplane. The front impeller and fan tray are both hot-insertable and hot-removable.The rear cooling subsystem consists of a pair of impellers that cool the SFMs, Routing Engine, MCS, the PCGs, and the power supplies. Each rear impeller is hot-insertable and hot-removable. The upper and lower impellers are not interchangeable.

JUNOS Software ControlAll Juniper Networks routers are designed to operate with missing or failed fans. The system can measure the presence and rotational speed of all fan components.Removing a fan triggers a minor alarm and the remaining fans continue to cool the system. Removing more than one fan triggers a major alarm.



l The T640 Internet Routing Node is the first platform to be built on Juniper Networks latest generation of packet forwarding hardware

l Represents the 3rd generation of Juniper Networks learning from the Internet classroom– Enhanced Internet Processor– New queuing architecture– Fundamental enhancements to system reliability and

availabilityl The system allows for scaling to hundreds of 10 Gbps

interfaces per router

New ASIC ChipsetThe T640 Internet Routing Node system is the first Juniper Networks product based on the T640 Internet Routing Node series set of ASICs. The ASIC set allows for the building of a distributed system providing up to 320 Gbps of forwarding capacity in half a seven foot rack. Each slot in a T640 Internet Routing Node chassis can contain up to two T640 Internet Routing Node PFEs. Each PFE provides 20 Gbps of forwarding capacity. With eight FPC slots in the chassis, this provides 320 Gbps of forwarding capacity per router.Distributed ArchitectureIn contrast to the current, centralized Juniper Networks architecture, the T640 Internet Routing Node systems have a distributed


What Has Not Changed

l Same architectural division of responsibility between Routing Engines and Packet Forwarding Engines as in our original systems – Juniper Networks engineers designed all ASICs– Hardware handles all transit packet forwarding

l Each JUNOS software release will continue to run on all platforms – Support continues for JUNOS software features across all

platforms

Leading Hardware DesignThe industry-leading hardware design is still in place for the T640 Internet Routing Node. The separation of the Routing Engine and Packet Forward Engine still exists. Thus, all transit packets only must look at the forwarding table in hardware to achieve high-speed forwarding. Also, although the ASIC chipset is a new generation, it was still designed by the same quality Juniper Networks Engineers as the previous M-series routers.

One Image Fits All JUNOS software will continue to run all platforms and will support all features across all platforms. This makes upgrades a trivial issue as no hardware-specific images to sort through exist.


Router Chassis—Front View

1500

Craft interface

Rack-mounting ears

Fan tray

FPCs

CIP

Air filter

ESD point

Fan tray

Air intakeAir intake

Air filterFan tray

ESD point

CIP

FPCs

Rack-mounting ears

Craft interface

Fan tray

Major Components—Front ViewBelow lists the major components of the router chassis from the front view.

– ESD Point: Consists of two electrostatic discharge points (banana plug receptacles)—one front and one rear.

– Flexible PIC Concentrators (FPCs): Up to eight FPCs install vertically in the front of the router. An FPC can hold up to four PICs. Each FPC contains one Packet Forwarding Engine. The Packet Forwarding Engines receive incoming packets from the PICs installed on the FPC and forward them through the switch planes to the appropriate destination port. Each FPC contains data memory, which is managed by the Queuing and Memory


Router Chassis—Rear View

15021501

Air exaust

Fan tray

SCGs

CBO

Routing Engines

CB1

SIBs

Air filter

Power supplies

ESD pointGrounding points

Fan tray

Air Exhaust

SCGs

CB0

CB1

REs

SIBs

Air filter

PowerSupplies

ESD pointGrounding points

Major Components—Rear View– Switch Interface Board (SIB): The T640

Internet Routing Node switching architecture is based on SIBs. Each of the systems five SIBs is connected to each FPC in the router. Four SIBs are in use at any one time, with the fifth provided for redundancy.

– Routing Engine (RE): T640 Internet Routing Node contains up to two identical REs. One RE is required for Line Card Chassis (LCC) operation and is a standard component of the base chassis. The second is required for redundant configurations. Each RE operates in a protected memory environment that limits the effect of a software process failing.


Flexible PIC Concentrators

l Eight FPC slots– Numbered 0–7, left to right

l Two types of FPCs– FPC2

§ Contains one Packet Forwarding Engine (PFE)§ Rated at 10 Gbps, full duplex§ Supports high bandwidth PICs used in M160 router platform

– FPC3§ Contains two Packet Forwarding Engines§ Rated at 40 Gbps, full duplex§ Supports higher speed, dedicated T640 Internet Routing Node router PIC

l Distinguished by type of Ejector on PIC

Eight SlotsUp to eight Flexible PIC Concentrators (FPCs) install vertically in the front of the router (see Figure 4). The FPCs are numbered left to right from FPC0 to FPC7. Each FPC has four connectors into which a PIC can be installed, allowing up to four PICs per FPC. An FPC can be installed into any FPC slot on the router, regardless of which PICs it contains.If a slot is not occupied by an FPC, an FPC blank panel must be installed to shield the empty slot and to allow cooling air to circulate properly through the router.FPCsFPC2 is rated at 5 Gbps full duplex and supports PICs also used in the M160 router. FPC3 is rated at 20 Gbps full duplex and


Routing Engine Components

l Each T640 Routing Engine consists of:– 600 MHz Pentium III CPU– 2 GB of SDRAM– 128 MB flash disk (primary storage device)– 30 GB hard disk storage– 128 MB removable PC card– PCI bus interface to the Control Board

Main ComponentsThe T640 Internet Routing Node uses a 600 MHz Pentium III processor running JUNOS software. The memory contains 2 GB of ECC SDRAM, which is used for Host Processor Subsystem processes such as routing protocols. The primary storage is a 160 MB or larger compact flash-based drive, which can hold two images, two configuration files, and microcode. The secondary storage is an IDE hard disk (30 GB or larger) for logs and for taking entire memory dumps and rebooting the system in the event of a flash disk failure. Tertiary storage consists of an externally accessible PC Card (PCMCIA) flash disk drive of at least 128 MB. It can be used for installing images and storing configuration files. The T640 Internet Routing Node has dual interfaces to the PFE Control Bus, which connect the RE to the other chassis modules.


Craft Interface

l Craft Interface features– LCD display (M40, M40e, and M160 routers only)– FPC online/offline buttons (M20, M40, M40e, and M160)– PIC online/offline buttons (M5 and M10)

Craft InterfaceThe Craft Interface is the collection of mechanisms on the Juniper Networks router that allows you to view system status messages and troubleshoot the router. The Craft Interface is located on the front of the chassis and contains system LEDs, FPC and PIC online/ofline buttons, and an LCD screen that provides status reporting for the entire system.


Craft Interface (Contd.)

l LEDs– OK

§ Blinking=starting§ Solid=running

– FAIL§ Solid=taken offline because of failure

l Buttons– Press and hold for three seconds to take FPC offline

System LEDs– FPC LEDs: Two LEDs exist–one green OK and one red Fail. These lights

indicate the status of each FPC. The two LEDs and a function button are located below each FPC module slot on the Craft Interface.

– Routing Engine LEDs: A red fail LED and a green OK LED on the Craft Interface indicate the status of the Routing Engines.

FPC Buttons– FPC offline buttons: These buttons allow you to take an FPC offline

gracefully. Press and hold the offline button near the FPC until the green OK LED extinguishes.


LEDs

l Red alarm– Major failure that affects service

l Yellow alarm– Minor failure that needs attention but does not affect service

Red AlarmThe red alarm LED indicates a system failure that is likely to cause an interruption in service. Examples of red alarms are listed below.

– Routing Engine failure– Cooling system failure– Interface loss of light or framing

Yellow AlarmThe yellow alarm LED indicates a system warning that is not likely to interrupt service, but that if left uncorrected might eventually cause a service interruption. Examples of yellow alarms are listed below.

– Maintenance alert– FPC with recoverable errors– Cooling system problem


LCD Display

l LCD display (M40 and M160 routers only)– Shows status when no alarms present– Shows alarm information when alarms are present

LCD DisplayThe Craft Interface has a four-line LCD screen with six navigation buttons. The LCD screen operates in one of two display modes are listed below.

– Idle mode: The default mode that displays the current system status until it is preempted by alarm mode.

– Alarm mode: The mode that displays alarm conditions whenever the red or yellow alarm LED is lit.


Status Display

l LCD display– Status messages– Alarms

godzillaUp 49+11:21:31

System Chassis OK

Status

Status DisplayIn normal operation when the system is functioning correctly, the LCD display shows a simple status display. This display is the default mode of the LCD screen. In this mode, the screen displays the current system status until it is preempted by an alarm.Basic status information displayed is listed below.

– Router’s name, on the first line.– Number of days, hours, minutes, and seconds that the system has been

running, on the second line.– Status messages, on the fourth line. These messages rotate at three-

second intervals. You can temporarily interrupt the message on this line during special situations involving hot-removal or hot-insertion of system components.


Alarm Display

l Alarms present– Number of alarms– Sorted by severity

Alarm

godzilla2 Alarms activeR: Fan failure Y: Change air filter

Alarm DisplayWhen a red or yellow alarm occurs, alarm mode preempts idle mode, displaying a message to alert you of serious alarm conditions. In alarm mode, the screen displays the information listed below.

– First line: Indicates the name of the router.– Second line: Indicates the number of alarms active on the router.– Third and fourth lines: Indicate individual alarms, with the most severe

condition shown first. Each line indicates whether the alarm is a red (R) or yellow (Y) alarm.


Dry Relay Contacts

l Activated with first alarm– Yellow and red alarms

l Can be disabled with cutoff button– New alarms reactivate relay– Alarm contacts supported on M20, M40 and M160 platforms

FAIL OK FAIL OK FAIL OK FAIL OK FAIL OK FAIL

0 1 2 3 4 5 6 7OK

FAIL

MANAGEMENTETHERNET

ROUTING ENGINE

CONSOLE

AUXILIARY

OKMENU

ENTER

ALARMREDALARM

ALARMCUTOFF

NCC

NO

NCC

NO

YELLOWALARM

FAIL OK FAIL OK

Dry Relay ContactsThe Craft Interface contains two sets of relay contacts–one set is activated by a system red alarm and one set is activated by a system yellow alarm. You can connect the alarm relay contacts to an external alarm device. Whenever a system alarm condition, such as fan failure or excessive temperature, triggers the red alarm on the Craft Interface, it also activates the red alarm relay contacts. Maintenance alerts, which trigger the yellow alarm on the Craft Interface, also activate the yellow alarm relay contacts.

Cutoff ButtonYou can silence external devices connected to the alarm relay contacts manually by pressing the alarm cutoff button, located at the bottom left of the Craft Interface front panel. Silencing the device does not remove the alarm messages from the display or extinguish the alarm LEDs. In addition, new alarms that occur after silencing an external device reactivate the external device. If no alarm is present, pressing the alarm cutoff button illuminates all LEDs on the Craft Interface.


PICs

l Where we are going…– Partial listing of common PICs– 4-port Fast Ethernet– 2-port ATM OC-3 – OC-192

PICsThe following slides examine a few of the common PICs. For a complete listing, see the attached Appendix.


Common PICs

l PICs– ATM– Channelized DS-3– Channelized OC-12 to DS-3 SMIR– DS-3, 4 port– E1– E3– Fast Ethernet– Gigabit Ethernet– SONET/SDH– T1– Tunnel services– Encryption services– Multilink services

PIC ListThe above visual shows a partial listing of the PICs supported. See the appendix in Volume 1 or https://www.juniper.net/products/pic-l2.html for a complete and current list.

https://www.juniper.net/products/pic-l2.html


4-Port Fast Ethernet PIC

4-port Fast Ethernet PIC, TX interface with RJ45 connector for M5 and M10 routers

Four-Port Fast Ethernet PIC The four-port Fast Ethernet PIC for the M-series Internet routers provides economic 100 Mbps performance with high reliability and low maintenance costs. The Fast Ethernet PIC is an attractive interface for a variety of applications, including intra-POP switching, content Web hosting, and both public and private peering.Additionally, each M-series router supports the Tunnel Services PIC, which enables the router to function as the ingress or egress point of an IP-IP unicast tunnel, a Generic Routing Encapsulation (GRE) tunnel, or a Protocol Independent Multicast-Sparse Mode (PIM-SM) tunnel. The Tunnel Services PIC does not have a physical interface due to its loopback function within the chassis.


2-Port ATM OC-3 PIC

2-port OC-3/STM-1 ATM PIC, single-mode, intermediate reach

Two-Port ATM OC-3 PIC The ATM OC-3/STM-1 and ATM OC-12/STM-4 PICs for the M-series Internet routers provide both the performance and the density to scale ATM-based backbones. They are useful for terminating ATM access circuits and for terminating ATM virtual circuits extending across a network backbone.


1-Port OC-192c PIC

1-port OC-192c/STM-64 SONET/SDH interface, for M160 router

One-Port OC-192 PIC The OC-192c/STM-64 PIC is advantageous when offering high bandwidth for inter-and intra-POP connections. The PIC also handles four 2.5 Gbps OC-48/STM-16 circuits over a single optical interface. It operates in both concatenated and nonconcatenated modes.


ASIC Functionality and Packet Flow

l Where we are going…– ASICs

§ PIC I/O Manager§ I/O Manager§ Buffer manager§ Internet Processor II

– Packet flow– Packet flow example

ASIC Functionality and Packet FlowThe following pages examine the Packet Forwarding Engine, which contains the ASICs for packet flow. The ASICS are the PIC I/O Manager, I/O Manager, Buffer Manager, and the Internet Processor II.


ASICs

Internet Processor II

ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

FPC

PICs

Packet Forwarding Engine System Controller

(For example, SSB and SFM)

MEM

MEM

MEM

PIC I/OManager

PIC I/OManager

ASIC Architecture An ASIC is a special-purpose chip designed for a specific application to accomplish identified tasks. An ASIC performs its task using hardware, which makes performance very fast; it differs from a general-purpose microprocessor chip that relies on software to accomplish a task. Examples of the Juniper Networks implementation of ASICs are the Distributed Buffer Manager and the Internet Processor.



ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

FPC

PICs

Packet Forwarding Engine System

Controller(SSB, SFM, etc.)

MEM

MEM

MEM

PIC I/OManager

PIC I/OManager

PIC I/O ASIC

l PIC I/O ASIC– Connects to FPC I/O ASIC– Manages Data Link Layer

framing, CRC, and bit stream– Detects Data Link Layer

errors– Generates Data Link Layer

alarms

l PIC I/O ASIC– Connects to FPC I/O ASIC– Manages Data Link Layer

framing, CRC, and bit stream– Detects Data Link Layer

errors– Generates Data Link Layer

alarms

PIC I/O ASICWhen a packet arrives on an input interface of the router, the PIC I/O Manager performs all the media-specific details such as framing and checksum verification. The PIC then passes a serial stream of bits into the I/O Manager ASIC on the FPC, which parses and removes the Layer 2 encapsulation.


MEM

Incoming Packets


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

FPC

PICs



MEM

MEM

PIC I/OManager

PIC I/OManager

l I/O Manager ASIC– Chops incoming

packets into 64-bytechunks

– Sends to buffer manager 1 ASIC

– Decodes Layer 2 encapsulation

– Decodes protocol level information

– Verifies packet integrity

l I/O Manager ASIC– Chops incoming

packets into 64-bytechunks

– Sends to buffer manager 1 ASIC

– Decodes Layer 2 encapsulation

– Decodes protocol level information

– Verifies packet integrity

I/OManager

I/O Manager ASICThe I/O Manager ASIC chops the packet into 64-byte pieces (called J-cells) and passes each J-cell to the inbound Buffer Manager ASIC. The I/O Manager ASIC also:

– Decodes and strips the Layer 2 portion of the packet, locating the beginning of Layer 3;

– Counts packets and bytes for each logical circuit;– Applies CoS rules to each packet; and– Performs basic packet integrity checks.


MEM


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

FPC

PICs

Packet ForwardingEngine System Controller


MEM

MEM

PIC I/OManager

PIC I/OManager

I/OManager

Buffer Manager ASICs

l Buffer manager ASICs– Manage packet memory shared across FPC slots – Extract address information from packets– Direct FPCs where to forward packets

l Buffer manager ASICs– Manage packet memory shared across FPC slots – Extract address information from packets– Direct FPCs where to forward packets

Buffer Manager ASICAll packets entering the router pass through the Buffer Manager 1 ASIC, which takes packet J-cells from each I/O Manager ASIC and writes them into the shared memory bank. The shared memory bank is distributed evenly across all the FPCs installed in the router. In parallel, the ASIC extracts information needed for route lookup from the packet and passes that information to the Internet Processor ASIC, which performs a lookup in its forwarding table and finds the outgoing interface and the specific next-hop address of the packet.The Buffer Manager 2 ASIC acts as an agent for each I/O Manager ASIC on each FPC. Once notified that packet needs transmission, the I/O Manager ASIC requests the packet’s stored J-cells from the Buffer Manager 2 ASIC. As the I/O Manager receives the J-cells, it transmits them to the PIC I/O Manager ASIC, which transmits them on the appropriate port.


FPC



MEM


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PICs

MEM

MEM

PIC I/OManager

PIC I/OManager

I/OManager

Internet Processor ASIC

l Internet Processor ASIC– Extracts next-hop

information from system forwarding table

– Passes next-hop information to buffer manager 2 ASIC

– Applies packet filtering and policy rules

– Collects exception packets for queuing to Routing Engine

l Internet Processor ASIC– Extracts next-hop

information from system forwarding table

– Passes next-hop information to buffer manager 2 ASIC

– Applies packet filtering and policy rules

– Collects exception packets for queuing to Routing Engine

Internet Processor ASICThe Internet Processor ASIC decides the ultimate destination for every packet that the router receives. The Internet Processor ASIC consults a forwarding table containing destination prefixes and corresponding next hops constructed by the Routing Engine.After the Internet Processor ASIC determines the next hop, it passes the results of the lookup to the second Distributed Buffer Manager ASIC, which passes it to the I/O Manager ASIC on the outgoing interface. (In the case of a multicast packet, multiple outgoing interfaces might exist.)


MEM


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

FPC

PICs

Packet Forwarding Engine System

Controller(SSB, SFM, etc.)

MEM

MEM

PIC I/OManager

PIC I/OManager

I/OManager

Outgoing Packets

l I/O Manager ASIC– Receives 64-byte

chunks from buffer manager 2 ASIC

– Adjusts any required protocol time-to-live values

– Encapsulates chunks inside appropriate Data Link Layer header

– Sends to PIC I/O Manager ASIC

l I/O Manager ASIC– Receives 64-byte

chunks from buffer manager 2 ASIC

– Adjusts any required protocol time-to-live values

– Encapsulates chunks inside appropriate Data Link Layer header

– Sends to PIC I/O Manager ASIC

Outgoing Packets and the I/O Manager ASICFor outbound packets, the I/O Manager ASIC gathers the 64-byte J-cells from the Buffer Manager 2 ASIC and feeds the resulting stream of bits to the PIC I/O Manager ASIC.The I/O Manager ASIC is responsible for queuing the packet, and applying CoS rules before output. However, the packet itself is never queued, but rather a pointer to the packet is queued. Each output port has four queues, each of which has some configured share of the port’s bandwidth.The I/O Manager ASIC on the outgoing interface can take a number of factors into account for queuing, such as the value of the Precedence bits, utilization of the input interface, destination address, and the RED algorithm. When the I/O Manager ASIC decides to dequeue the packet for transmission (when the packet reaches the front of the queue and is ready for transmission), the I/O Manager ASIC reads the memory J-cells from packet buffer memory, reassembles the packet, and finally passes it to the media-specific ASIC for transmission on the line.


Packet Flow Example (1)

Packet arrives at PIC on fiber-optic cable Internet Internet

Processor IIProcessor IIForwardingForwarding

TableTable

Buffer Buffer Manager 1Manager 1


I/OI/OManagerManager



PIC I/OPIC I/OManagerManager






MMEEMM

MMEEMM

MMEEMM



Packet ArrivalIn this example, a packet arrives at the router over a fiber-optic cable, where the PIC I/O ASIC receives it. This process remains the same for any physical medium.



PIC I/O ASIC extracts data and gives it to FPC I/O ASIC


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

MEM

MEM

MEM

PIC I/OManager


PIC I/OThe PIC I/O ASIC extracts data from the packet and sends the data to the FPC I/O ASIC.



FPC I/O ASIC chops up data and feeds it to buffer manager 1 ASIC

MMEEMM


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

MEM

MEM

PIC I/OManager

PIC I/OManager


I/O ManagerFrom the media-specific stage, a serial stream of bits is passed to the I/O Manager ASIC. The I/O Manager ASIC determines if the frame is IPv4, MPLS, Frame Relay, and so on, and identifies the beginning of the Layer 3 data. The I/O Manager ASIC also sets notification flags that can be used to support differentiated services. Finally, the I/O Manager ASIC chops the packet into 64-byte J-cells and passes each of the J-cells to the Distributed Buffer Manager 1 ASIC. (These J-cells are sized for efficient storage and retrieval from shared memory and are unrelated to ATM cells.)



Buffer Manager 1 ASIC saves the J-cells in shared memory

MMEEMM


ForwardingTable


Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

MEM

MEM

PIC I/OManager

PIC I/OManager

I/OManager

Buffer Manager 1 ASIC The Buffer Manager ASIC disperses the J-cells evenly across the shared memory in a round-robin manner. While the ASIC is doling out each of the J-cells to shared memory, the Distributed Buffer Manager ASIC extracts the route lookup key as the J-cells fly by and constructs a packet notification. For a unicast IPv4 frame, it determines the incoming interface, destination IP address, source IP address, protocol value, and the source and destination TCP/UDP port numbers.


MEM

MEM

MEM


Buffer Manager tells Internet Processor destination address and locations ofstored chunks


ForwardingTable


Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

I/OManager

Buffer Manager Notification CellAfter collecting all the information listed on the previous page, the Buffer Manager ASIC forwards the packet notification to the Internet Processor ASIC so it can make a forwarding decision.



Internet Processor looks up next hop and destination FPC and notifies output Buffer Manager and FPC ASICs

MEM


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

MEM

MEM

PIC I/OManager

PIC I/OManager

I/OManager

Internet Processor ASIC Look-UpWhen the packet notification arrives at the Internet Processor ASIC, it searches the IP forwarding table looking for the longest match for the given destination address. The result of this lookup returns the physical interface next-hop address for the destination. The Internet Processor amends the notification with the results and passes it to the Distributed Buffer Manager 2 ASIC. When the Distributed Buffer Manager ASIC receives the amended notification, it forwards the notification to the I/O Manager ASIC for the output interface.



I/O Manager ASIC requests chunks from shared memory and adds appropriate Data Link Layer encapsulation on way to PIC


ForwardingTable

Buffer Manager 1


I/OManager


PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

MMEEMM

PIC I/OManager

PIC I/OManager

I/OManager

MMEEMM

MMEEMM

I/O Manager OutgoingThe I/O Manager ASIC on the output FPC is responsible for managing packet queues. For transmission, the packet itself is not queued, but instead a pointer to the packet is queued while the packet (in other words, its component J-cells) remains stored in shared memory. The I/O Manager on the output interface can take a number of factors into account when deciding to queue a packet (for example, the value of the IP Precedence bits, utilization of the input interface, and the random early detect algorithm). When the packet notification reaches the front of its queue and is ready for transmission, the I/O Manager issues a request using the Distributed Buffer Manager 2 ASIC to read the J-cells from shared memory. The I/O Manager serializes the J-cells and sends the resulting bit stream to the media-specific ASIC on the output interface.



PIC ASIC adds Physical Layer framing, CRC, and sends bit stream out to the wire

MEM

MEM

MEM


ForwardingTable

Buffer Manager 1

Buffer Manager 2

I/OManager

I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager

PIC I/OManager


PIC I/OManager

PIC I/OManager

I/OManager

PIC I/O Manager OutgoingIn this example, the media-specific ASIC on the output interface performs SONET/SDH scrambling, places the bits in a SONET/SDH frame, and starts serializing the bits on the fiber. At this point, the packet leaves the Packet Forwarding Engine towards the destination network.


Exception Packets

l Exception packets– Local delivery– IP options

§ Source Route, Router Alert, etc.

l Processed by Packet Forwarding Engine control CPU– Performs rate limiting– Gives preference to control traffic – Sends local and control traffic to Routing Engine

Exception PacketsException packets are packets that cannot be processed by the Packet Forwarding Engine. They consist of the items listed below.

– Packets addressed to the box, such as routing protocol updates, telnet sessions, pings, traceroutes, and replies to any traffic sourced from the Routing Engine.

– Packets that need an ICMP error message generated.– An ICMP destination unreachable messages. If the Internet Processor has

no entry in the forwarding table for the packet's destination address, a destination unreachable ICMP message must be sent to the source. The Routing Engine must be notified so it can send these messages.

– IP packets with the IP Options field. Options in the packet's IP header are rarely seen, but the Packet Forwarding Engine was purposely designed notto handle IP options. They must be sent to the Routing Engine for processing.

Packet Forwarding Engine CPUThe Internet Processor II ASIC passes exception packets to the microprocessor on the Packet Forwarding Engine System Control Board, which processes almost all of them. Certain exception packets are also sent to the Routing Engine for further processing. The exception packets destined for the Routing Engine are sent by the PowerPC processor on the Packet Forwarding Engine System Control Board to the Routing Engine over the 100 Mb fxp1 interface. The packets are rate-limited by the PowerPC processor to protect the Routing Engine from denial-of-service attacks. During times of congestion, preference is given to the local and control traffic.


Review Questions

1. Which Juniper Networks routers are aimed at the Internet core? At the edge?

2. What are the primary responsibilities of the Routing Engine?

3. What are the primary responsibilities of the Packet Forwarding Engine?

4. What is the purpose of and relationship between FPCs and PICs?

5. What is the purpose of the Craft Interface?6. How do packets flow through a Juniper Networks

M-series platform?

This Module Discussed:– Juniper Networks products and their typical applications;– Major router components;– General maintenance issues;– Operation of the Craft Interface; and– Packet flow through an M-Series router.

Documents

OTJNR Mod 01