BSR64K R6.3.1 TroubleshootingGuide

  • Upload
    vartaxe

  • View
    238

  • Download
    0

Embed Size (px)

Citation preview

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    1/138

    BSR 64000

    Troubleshooting

    Guide

    Compass ID: 388931013 Version 4

    Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    2/138

    Notice

    EXCEPT AS INDICATED IN THE APPLICABLE SYSTEM PURCHASE AGREEMENT, THE SYSTEM,

    DOCUMENTATION AND SERVICES ARE PROVIDED "AS IS", AS AVAILABLE, WITHOUT WARRANTY OFANY KIND. MOTOROLA MOBILITY, INC. DOES NOT WARRANT THAT THE SYSTEM WILL MEETCUSTOMER'S REQUIREMENTS, OR THAT THEIR OPERATION WILL BE UNINTERRUPTED ORERROR-FREE, OR THAT ANY ERRORS CAN OR WILL BE FIXED. MOTOROLA MOBILITY, INC. HEREBYDISCLAIMS ALL OTHER WARRANTIES, EXPRESS OR IMPLIED, ORAL OR WRITTEN, WITH RESPECTTO THE SYSTEM AND SERVICES INCLUDING, WITHOUT LIMITATION, ALL IMPLIED WARRANTIES OFTITLE, NON-INFRINGEMENT, INTEGRATION, MERCHANTABILITY OR FITNESS FOR ANY PARTICULARPURPOSE AND ALL WARRANTIES ARISING FROM ANY COURSE OF DEALING OR PERFORMANCE ORUSAGE OF TRADE.

    EXCEPT AS INDICATED IN THE APPLICABLE SYSTEM PURCHASE AGREEMENT, MOTOROLAMOBILITY, INC. SHALL NOT BE LIABLE CONCERNING THE SYSTEM OR SUBJECT MATTER OF THISDOCUMENTATION, REGARDLESS OF THE FORM OF ANY CLAIM OR ACTION (WHETHER INCONTRACT, NEGLIGENCE, STRICT LIABILITY OR OTHERWISE), FOR ANY (A) MATTER BEYOND ITSREASONABLE CONTROL, (B) LOSS OR INACCURACY OF DATA, LOSS OR INTERRUPTION OF USE, ORCOST OF PROCURING SUBSTITUTE TECHNOLOGY, GOODS OR SERVICES, (C) INDIRECT, PUNITIVE,INCIDENTAL, RELIANCE, SPECIAL, EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING, BUT

    NOT LIMITED TO, LOSS OF BUSINESS, REVENUES, PROFITS OR GOODWILL, OR (D) DIRECT

    DAMAGES, IN THE AGGREGATE, IN EXCESS OF THE FEES PAID TO IT HEREUNDER FOR THE SYSTEMOR SERVICE GIVING RISE TO SUCH DAMAGES DURING THE 12-MONTH PERIOD PRIOR TO THE DATETHE CAUSE OF ACTION AROSE, EVEN IF COMPANY HAS BEEN ADVISED OF THE POSSIBILITY OFSUCH DAMAGES. THESE LIMITATIONS ARE INDEPENDENT FROM ALL OTHER PROVISIONS OF THISAGREEMENT AND SHALL APPLY NOTWITHSTANDING THE FAILURE OF ANY REMEDY PROVIDEDHEREIN.

    All Motorola Mobility, Inc. products are furnished under a license agreement included with the product. If you areunable to locate a copy of the license agreement, please contact Motorola Mobility, Inc.

    No part of this publication may be reproduced in any form or by any means or used to make any derivative work (suchas translation, transformation, or adaptation) without written permission from Motorola Mobility, Inc.

    Motorola Mobility reserves the right to revise this publication and to make changes in content from time to timewithout obligation on the part of Motorola Mobility to provide notification of such revision or change. MotorolaMobility provides this guide without warranty of any kind, implied or expressed, including, but not limited to, theimplied warranties of merchantability and fitness for a particular purpose. Motorola Mobility may make

    improvements or changes in the product(s) described in this manual at any time.

    MOTOROLA and the Stylized M Logo are trademarks or registered trademarks of Motorola Trademark Holdings,LLC. All other trademarks are the property of their respective owners.

    2011 Motorola Mobility, Inc. All rights reserved.

    Compass ID: 388931013 Version 4

    Release 6.3.1

    Published: 12/11

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    3/138

    Compass ID: 388931013 Version 4 iii

    Contents

    Contents

    Preface

    Scope ............................................................................................................................................. ixAudience........................................................................................................................................ix

    Documentation Set ........................................................................................................................ ix

    Conventions..................................................................................................................................xii

    Notes, Cautions, Warnings ..........................................................................................................xiii

    If You Need Help.........................................................................................................................xiii

    Telephone Support............................................................................................................xiii

    Online Support..................................................................................................................xiv

    Motorola BSR Customer Website......................................................................xiv

    1 Introduction

    Overview .....................................................................................................................................1-1

    Understanding Basic Troubleshooting ........................................................................................1-2

    Discovering Problems .................................................................................................................1-3

    Viewing Symptoms .....................................................................................................................1-3

    Isolating the Problem ..................................................................................................................1-4

    Solving the Problem ....................................................................................................................1-4

    Evaluating the Solution ...............................................................................................................1-5

    2 Checking Physical EquipmentOverview .....................................................................................................................................2-1

    http://bsr64k-r5.2.0-troubleshootingguidetoc.pdf/http://bsr64k-r5.2.0-troubleshootingguidetoc.pdf/
  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    4/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    iv Compass ID: 388931013 Version 4

    Checking Physical Network Connections ...................................................................................2-2

    Turning On the BSR 64000.........................................................................................................2-2

    Determining BSR 64000 Operational Status...............................................................................2-3

    Interpreting Resource Module LED Displays.............................................................................2-6

    Supervisor Resource Module LEDs .................................................................................2-7

    Module LEDs.....................................................................................................2-7

    Fan Status LEDs.................................................................................................2-7

    Alarm LEDs .......................................................................................................2-8

    2:8 CMTS Resource Module LEDs .................................................................................2-9

    Module LEDs.....................................................................................................2-9

    Per-Port LEDs..................................................................................................2-10

    2:8 CMTS Standby Resource Module LEDs .................................................................2-11

    Module LEDs...................................................................................................2-11

    Per-Port LEDs..................................................................................................2-12

    TX32 Resource Module LEDs .......................................................................................2-13Module LEDs...................................................................................................2-13

    Per-Port LEDs..................................................................................................2-15

    TX32 Standby Resource Module LEDs .........................................................................2-16

    Module LEDs...................................................................................................2-16

    Per-Port LEDs..................................................................................................2-17

    RX48 Resource Module LEDs.......................................................................................2-17

    Module LEDs...................................................................................................2-17

    Per-Port LEDs..................................................................................................2-19

    Diagnostic Ethernet Port LEDs........................................................................2-20

    RX48 Standby Resource Module LEDs.........................................................................2-20

    Module LEDs...................................................................................................2-20

    Per-Port LEDs..................................................................................................2-21

    Diagnostic Ethernet Port LEDs........................................................................2-21

    Ether-FlexTMResource Module LEDs............................................................................2-22

    Module LEDs...................................................................................................2-22

    GIG-E Port LEDs.............................................................................................2-22

    Ethernet (10/100) Port LEDs ...........................................................................2-23

    Rebooting an Individual Resource Module...............................................................................2-24

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    5/138

    Release 6.3.1 Contents

    Compass ID: 388931013 Version 4 v

    3 Troubleshooting the CMTS

    Overview .....................................................................................................................................3-1

    Using Flap Lists to Troubleshoot CM Problems.........................................................................3-2

    Viewing Flap List Statistics to Identify Network Health .................................................3-2

    Interpreting Flap List Statistics ........................................................................................3-5

    Tips for Administrating Flap Lists .................................................................................3-10

    Resolving HFC Network Performance Problems .....................................................................3-11

    Downstream Signal Reflected on Upstream Path ..........................................................3-11

    Slow Performance Detected on Upstream Port..............................................................3-11

    Too Many CPE Hosts on Subscriber CM ......................................................................3-15

    Resolving Problems on the Upstream Path ...............................................................................3-16

    Bad Upstream Signal-to-noise Ratio Detected ..............................................................3-16

    Upstream Power Level Too Low or High .....................................................................3-19

    Resolving Problems on the Downstream Path..........................................................................3-21

    Bad Downstream Signal-to-Noise Ratio Detected ........................................................3-21Downstream Power Level Too Low or High ................................................................3-23

    Resolving Cable Modem Problems ..........................................................................................3-24

    Misconfigured Authentication String or Key ................................................................3-24

    CM Does Not Reply to Station Maintenance Requests .................................................3-26

    CM is Offline .................................................................................................................3-26

    CM Cannot Obtain an IP Address ..................................................................................3-28

    Provisioning Problems Cause CMs Not to Register ......................................................3-29

    CM Does Not Respond to SNMP Messages .................................................................3-33

    4 Troubleshooting Basic Routing Problems

    Overview .....................................................................................................................................4-1

    Resolving Host Connectivity Problems ......................................................................................4-2

    Default Gateway Configuration Problems ......................................................................4-2

    Misconfigured or Missing Default Routes .......................................................................4-3

    Incomplete DNS Host Table ............................................................................................4-3

    DNS Not Running ............................................................................................................4-4

    Handling Routing Problems........................................................................................................4-4

    Problem Router ................................................................................................................4-4

    Misconfigured Router.......................................................................................................4-6

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    6/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    vi Compass ID: 388931013 Version 4

    Routing Interface Down ...................................................................................................4-7

    Handling a Misconfigured Access List ......................................................................................4-8

    Access List and Filter Misconfigurations ......................................................................4-10

    Handling UDP Broadcast Forwarding Problems .....................................................................4-11

    Missing or Misconfigured IP Helper-address Specification...........................................4-11

    UDP Broadcast Misconfiguration .................................................................................4-12

    Resolving PPP Link Over SONET Failures .............................................................................4-12

    5 Troubleshooting RIPOverview .....................................................................................................................................5-1

    Handling Routing Table Problems ..............................................................................................5-2

    Misconfigured or Missing Network Router Table Entries ...............................................5-2

    Misconfigured Route Filtering .........................................................................................5-2

    Split Horizon is Disabled .................................................................................................5-3

    Handling RIP Version Inconsistencies ........................................................................................5-4Misconfigured Version of RIP Running on BSR ............................................................5-4

    Misconfigured Version of RIP Running on Specified Interface.......................................5-5

    6 Troubleshooting OSPF

    Overview .....................................................................................................................................6-1

    Handling OSPF-designated Interface Problems..........................................................................6-2

    Handling Router Neighbor Misconfigurations ............................................................................6-2

    Misconfigured Router.......................................................................................................6-3

    Mismatched OSPF Parameters ........................................................................................6-3

    Mismatched IP MTU ........................................................................................6-4

    Resolving Missing Routes in Routing Table...............................................................................6-6

    RIP Routing Information Incorrectly Redistributed into OSPF.......................................6-6

    ABR Configured Without Area 0 Interface .....................................................................6-7

    7 Troubleshooting BGP

    Overview .....................................................................................................................................7-1

    Handling BGP Routing Problems ...............................................................................................7-2

    Missing Neighbor Table Entry ........................................................................................7-2

    Misconfigured Access List ..............................................................................................7-2

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    7/138

    Release 6.3.1 Contents

    Compass ID: 388931013 Version 4 vii

    Missing Network Destination Advertisement .................................................................7-3

    Handling BGP Peer Misconfigurations ......................................................................................7-4

    8 Troubleshooting Multicast Routing

    Overview .....................................................................................................................................8-1

    Hosts are Not Receiving Video ..................................................................................................8-1

    Check Connection Between IGMP and Hosts..................................................................8-2

    IGMP Configuration Problems .........................................................................8-2

    Misconfigured IGMP Access Group .................................................................8-2

    Some IGMP Hosts Are Not Receiving Multicast Services ...............................8-3

    Check Connections Between Multicast Routers .............................................................8-4

    9 Troubleshooting SONET

    Overview .....................................................................................................................................9-1

    Resolving Fault LED Issues........................................................................................................9-2LOS Determination ..........................................................................................................9-2

    LOS Resolution ................................................................................................................9-3

    LOF Determination ..........................................................................................................9-3

    LOF Resolution ................................................................................................................9-4

    LOP Determination ..........................................................................................................9-4

    LOP Resolution ................................................................................................................9-5

    Failed POS Module ..........................................................................................................9-6

    Fail LED Blinks and Lights Repeatedly...........................................................................9-6

    Handling Data Loss on SONET Link .........................................................................................9-6

    Data Loss Determination..................................................................................................9-6

    Data Loss Resolution........................................................................................................9-8

    A Cable ModemRegistration Process

    Introduction ................................................................................................................................A-1

    Scanning ..........................................................................................................................A-1

    Initial Ranging.................................................................................................................A-2

    Establishing IP Connectivity ...........................................................................................A-3

    Establishing Time of Day................................................................................................A-3

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    8/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    viii Compass ID: 388931013 Version 4

    TFTP Connectivity ..........................................................................................................A-3

    Registration......................................................................................................................A-4

    Baseline Privacy ..............................................................................................................A-4

    Periodic Ranging .............................................................................................................A-4

    Data Exchange .................................................................................................................A-5

    Index

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    9/138

    Compass ID: 388931013 Version 4 ix

    Preface

    Scope

    This document describes how to troubleshoot the Motorola Broadband Services

    Router 64000 (BSR 64000) including hardware, applications, servers, databases,

    and routing. This guide uses the term networkto refer to subscriber cable modems, theBSR family of products, cables and equipment, and host servers.

    Audience

    This document is for use by those persons who will install and configure the

    BSR 64000 product. Only trained service personnel should install, maintain, or

    replace the BSR 64000.

    Documentation Set

    The following documents comprise the BSR 64000 documentation set:

    n BSR 64000 Quick Start Guide

    The quick start guide provides a "roadmap" to the tasks involved in physically

    installing the BSR 64000 product, physically connecting it to your network/HFC

    infrastructure, and performing configuration tasks to enable the BSR 64000 to

    operate in your networking environment.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    10/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    x Compass ID: 388931013 Version 4

    n BSR 64000 Chassis Installation Guide

    This guide provides detailed instructions for physically installing the BSR 64000

    product including: procedures for rack mounting, making physical network cable

    connections, connecting DC power, and for determining the status of the BSR

    64000 after applying power to it. This document also provides a description of the

    BSR 64000 chassis, its hardware components and modules.

    n BSR 64000 Module Installation Guide

    This guide contains procedures for installing additional and replacement

    Resource and I/O Modules in a BSR 64000 chassis and for making physical cable

    connections to the modules.

    n BSR 64000 Command Line Interface Users Guide

    For users, this guide describes the structure of the BSR 64000 Command Line

    Interface (CLI) and its various command modes. It also provides rules and

    guidelines for navigating through the CLI.

    n BSR 64000 Command Reference Guide

    This guide contains individual descriptions of the entire set of commands that

    comprise the BSR 64000 Command Line Interface (CLI). These commands are

    used to interface with, configure, manage, and maintain the BSR 64000.

    n BSR 64000 System Administration Guide

    For system administrators, this guide provides detailed procedures for performing

    initial configuration tasks including setting up: user accounts and passwords;

    telnet and console access; system logging; and associated servers such as DHCP,

    DNS, etc.

    n BSR 64000 CMTS Configuration and Management Guide

    This guide provides the instructions and procedures for configuring and

    managing BSR 64000 CMTS operation.

    n BSR 64000 Routing Configuration and Management Guide

    This guide contains the instructions and procedures for configuring and managing

    BSR 64000 routing operation, including RIP, OSPF, and BGP.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    11/138

    Release 6.3.1 Preface

    Compass ID: 388931013 Version 4 xi

    n BSR 64000 SNMP Configuration and Management Guide

    This guide provides the instructions and procedures for configuring and

    managing BSR 64000 Simple Network Management Protocol (SNMP) operation.

    It also describes SNMP MIBs; provides information that describes standard and

    proprietary MIB support; describes how to walk MIBs; and how to compile and

    load SNMP MIBs.

    n BSR 64000 BGP/MPLS VPN Configuration Guide

    This guide provides the instructions and procedures for configuring and

    managing the BSR 64000 to support and implement Border Gateway Protocol/MultiProtocol Label Switching Virtual Private Networks (BGP/MPLS VPNs).

    n BSR 64000 Troubleshooting Guide

    This guide contains instructions and procedures for troubleshooting typical

    configuration problems that might be encountered using the BSR 64000. It also

    offers suggestions for information to record, and have available should the need

    arise to call Motorola support for assistance with BSR 64000 operationalproblems.

    n BSR 64000 Release Notes

    These documents are specific to each release of the BSR 64000 product (software

    and hardware). Release notes provide information about features not documented

    or incorrectly documented in the main documentation set; known problems and

    anomalies; product limitations; and problem resolutions.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    12/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    xii Compass ID: 388931013 Version 4

    Conventions

    This document uses the conventions in the following table:

    Convention Example Explanation

    angle brackets < > ping

    ping 54.89.145.71

    Arguments in italic and enclosed by angle

    brackets must be replaced by the text the

    argument represents. In the example,

    54.89.145.71replaces . When

    entering the argument, do not type the anglebrackets.

    bar brackets [ ] disable[level] Bar brackets enclose optional arguments. The

    example indicates you can use the disable

    command with or without specifying a level.

    Some commands accept more than one

    optional argument. When entering the

    argument, do not type the bar brackets.

    bold text cable relay-agent-option Boldface text must be typed exactly as it

    appears.

    brace brackets {} page{on| off} Brace brackets enclose required text. The

    example indicates you must enter either onor

    offafter page. The system accepts the

    command with only one of the parameters.

    When entering the text, do not type the brace

    brackets.

    italic text boot system Italic type indicates variables for which you

    supply values in command syntax descriptions.

    It also indicates file names, directory names,

    document titles, or emphasized text.

    screen display Wed May 6 17:01:03

    2000

    This font indicates system output.

    vertical bar | page{on| off} A vertical bar separates the choices when a

    parameter is required. The example indicatesyou can enter either command:

    page onorpage off

    When entering the parameter, do not type the

    vertical bar or the brace brackets.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    13/138

    Release 6.3.1 Preface

    Compass ID: 388931013 Version 4 xiii

    Notes, Cautions, Warnings

    The following icons and associated text may appear in this document.

    If You Need Help

    Support for your BSR 64000 hardware and software is available via telephone and the

    Internet.

    Telephone Support

    If you need assistance while working with the BSR 64000, contact the Motorola

    Technical Response Center (TRC):

    The Motorola TRC is on call 24 hours a day, 7 days a week.

    When calling for technical support, please have the following information available:

    n Your customer information, including location, main contact, and telephone

    number

    n BSR product and modules

    Note:A note contains tips, suggestions, and other helpful information, such

    as references to material not contained in the document, that can help you

    complete a task or understand the subject matter.

    Caution: The exclamation point, within an equilateral triangle, is intended toalert the user to the presence of important installation, servicing, andoperating instructions in the documents accompanying the equipment.

    Warning: This symbol indicates that dangerous voltage levels are present

    within the equipment. These voltages are not insulated and may be of

    sufficient strength to cause serious bodily injury when touched. The symbolmay also appear on schematics.

    U.S. 1-888-944-HELP (1-888-944-4357)

    International +215-323-0044

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    14/138

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    15/138

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    16/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    1-2 Compass ID: 388931013 Version 4

    Understanding Basic Troubleshooting

    The basic steps you need to troubleshoot network problems are as follows:

    1. Identify the cause or symptom of the problem, which can be any undesired result

    or behavior. SeeDiscovering Problems on page 1-3, to learn how to identify

    problems.

    2. Isolate the cause or symptom of the problem and try to determine its scope. For

    example, determine if it is the whole HFC network, a particular subnetwork on

    the HFC network, or just one subscriber that is experiencing the problem. See

    Isolating the Problem on page 1-4, for more information.

    3. Once the cause or symptom of a problem is isolated, make a list of

    troubleshooting procedures that you plan to use. Refer to subsequent chapters in

    this document for specific troubleshooting procedures you can use.

    4. Document the changes and effects of changes as you perform troubleshooting

    procedures, and note any new troubleshooting procedures that you use. This

    simple precaution helps to avoid repeating steps, allows for future reference in

    case the problem recurs, and is especially useful for troubleshooting intermittentproblems.

    5. Determine if the problem is solved. Ensure that the troubleshooting procedure did

    not cause new problems.

    6. If the problem is not solved, try to identify problem causes more clearly, isolate

    any additional causes, and perform additional troubleshooting procedures to

    correct the problem.

    Note: One or more symptoms or causes can be related to a single problem.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    17/138

    Release 6.3.1 Introduction

    Compass ID: 388931013 Version 4 1-3

    Discovering Problems

    Ensure that you thoroughly understand the network so that you can establish abaseline from which to work, and distinguish the differences between normal and

    abnormal activity on the network.

    Perform the following steps to determine the source of problems:

    Review release notes and technical bulletins to determine if there are any

    incompatibilities or known problems.

    Gather information for all the possible causes or symptoms to more quicklyisolate the problem.

    Discover if the problem relates to another problem that you must solve first.

    Record all configuration parameters that relate to the problem.

    Determine if the network configuration has changed recently, such as the addition

    or removal of components, upgrading, or reconfiguration.

    Identify aspects that causes or symptoms have in common to determine if they are

    related.

    Look for network patterns in the causes. What time of day did these problems

    occur? What events were logged? What network thresholds were transgressed?

    Record any changes that occurred since the network was last functioning

    correctly. Changes in network activity may relate to a configuration change.

    Record any changes that occurred since the last time the BSR was operating

    properly. Investigate any configuration changes that might be related to theproblem.

    Viewing Symptoms

    Perform the following tasks to view and compare symptoms that are related to a

    problem:

    Repeat the conditions that led to the symptom. Consider any errors or failures thatcan cause a particular symptom, and test them to see if they are causing the

    symptom.

    Determine if any symptoms are related. Are there unexpected or undesired results

    in more than one area? If so, find the areas in common and the variables that

    affect them. The source of the problem is often found in similar areas.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    18/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    1-4 Compass ID: 388931013 Version 4

    Focus on one symptom or a set of related symptoms of a problem. However, do

    not completely disregard other symptoms, because what may appear to be an

    unrelated problem may actually be related based on other symptoms.

    Isolating the Problem

    A problem can have one or more causes. To identify the cause of unwanted behavior,

    use the following techniques:

    Isolate the problem. For example, isolate a problem to one part of the network or

    to a specific access module. Find the functions that are working correctly.

    Retrace the steps that were taken, and return the network to its condition before

    the problem first appeared. Once the network is in a known condition, take

    incremental steps and observe the network to learn when and where the problems

    occur.

    Determine if there have been any additions, changes, or upgrades to the network.

    If so, consider any consequences the changes could have had on the network, and

    whether they affect the current situation.

    Solving the Problem

    Different problems require different actions and solutions, but follow these basic steps

    to fix any problem:

    1. Identify the course of action and the steps you plan to take.

    2. Decide what tools are necessary to fix the problem on the network. For example,

    you can use the CLI as a tool to look at events and set SNMP traps. You can use a

    cable tester to check physical media connections.

    3. Perform each step for the course of action.

    4. Verify the result of each action using the available CLI show commands. For

    example, if you enabled a port, use the appropriate show ip interface brief

    command in Privileged EXEC mode to verify that the port is enabled.

    5. If more than one possible action exists, select the easiest one first to quickly

    eliminate possibilities, or select the action that appears most likely to solve the

    problem first, even if it is the most time-consuming or difficult to perform.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    19/138

    Release 6.3.1 Introduction

    Compass ID: 388931013 Version 4 1-5

    Evaluating the Solution

    Once you find the solution, test it to ensure that no new symptoms or problems occur.If new symptoms or problems do occur, repeat the troubleshooting process to

    determine the cause. If problems or symptoms recur, create a standard test plan to

    evaluate fixes.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    20/138

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    21/138

    Compass ID: 388931013 Version 4 2-1

    2Checking Physical Equipment

    Overview

    This chapter describes how to turn on the BSR 64000 and observe system startup to

    determine if the system boots properly. The process also involves checking physical

    connections and observing LEDs on the BSR products. In addition, you should check

    power and network connections anytime you install new hardware or whenever a

    problem occurs.Topics in this chapter include:

    Checking Physical Network Connections

    Turning On the BSR 64000

    Determining BSR 64000 Operational Status

    Interpreting Resource Module LED Displays

    Rebooting an Individual Resource Module

    BSR 64000 T bl h ti G id R l 6 3 1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    22/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    2-2 Compass ID: 388931013 Version 4

    Checking Physical Network Connections

    Check network connections for loose, broken, or disconnected cables. Inspect thecable terminating connectors for damage.

    Use the following techniques to troubleshoot physical cables on the network:

    Use the Command Line Interface (CLI) showcommands to view port and slot

    information.

    Use a cable tester to test the network cables for damage.

    Verify that all the networks associated with the BSR are within properspecifications.

    Use cable testing equipment to measure or ensure that the correct distances for

    cable runs are in place.

    If the BSR 64000 is installed, ensure that all modules are seated correctly in the

    chassis.

    Replace any suspected defective modules or devices with known working spare

    modules or devices.

    Refer to the system documentation or service contract information for

    replacements and on-site spares.

    Turning On the BSR 64000

    To apply power to the BSR 64000, complete the following tasks:

    1. Verify that the electrical connections to the BSR 64000 are secure.

    2. Turn on the DC power supply connected to the BSR 64000.

    Release 6 3 1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    23/138

    Release 6.3.1 Checking Physical Equipment

    Compass ID: 388931013 Version 4 2-3

    3. Turn on each BSR 64000 Power Entry Module by placing its power switch in the

    ON (I) position. You can access the Power Entry Module power switches through

    the plastic shield that covers the DC power connections. See Figure 2-1.

    Figure 2-1 Accessing BSR 64000 Power Switches

    Determining BSR 64000 Operational Status

    To determine BSR 64000 status following power up do the following:

    Visually check the operational status of the cooling units. All fans in the bottomFan Module and the blowers in the top Blower Module should be turning and the

    Fan Status LED labeled OK on the front panel of the SRM should be lit green for

    both the top (TOP) Blower Module and bottom (BOT) Fan Module.

    Power Entry Module B

    Power Switch

    Power Entry Module A

    Power Switch

    bsr64k036a

    BSR 64000 Troubleshooting Guide Release 6 3 1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    24/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

    2-4 Compass ID: 388931013 Version 4

    Observe the LEDs on the SRM, CMTS, TX32, RX48, and NIM (HSIM) Modules

    after the booting process completes. The LED display on these modules will vary

    until the BSR 64000 is booted. When the booting process completes, the LEDswill display as described in Table 2-1.

    Table 2-1 BSR 64000 LED Display States Following Successful Booting

    Module LED LED Display State

    Supervisor Resource

    Module (SRM)

    Module LEDs:

    Fail Off

    Status Green (Active SRM)

    or

    Blinking Green (Standby SRM)

    (SRM Redundant Systems Only)

    Alarm Off

    Fan Status LEDs (TOP and BOT (bottom) Fan Module):

    OK Green

    Fail Off

    Alarm LEDs:

    MIN (Minor) Off

    MAJ (Major) Off

    CRIT (Critical) Off

    2:8 CMTS Resource

    Module

    Module LEDs:

    Fail Off

    Status Green

    Alarm Off

    Upstream and Downstream Port LEDs:

    Link Off (unconfigured) orGreen

    Fault Off

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    25/138

    Release 6.3.1 Checking Physical Equipment

    Compass ID: 388931013 Version 4 2-5

    2:8 CMTS Standby

    Resource Module

    Module LEDs:

    Fail Off

    Status Blinking Green

    Alarm Off

    Upstream and Downstream Port LEDs:

    Link Off Fault Off

    TX32 Resource Module Module LEDs:

    Fail Off

    Status Green

    Alarm Off

    Downstream Port LEDs:

    Link Off (unconfigured) orGreen

    Fault Off

    TX32 Standby Resource

    Module

    Module LEDs:

    Fail Off

    Status Blinking Green

    Alarm Off

    Downstream Port LEDs:

    Link Off

    Fault Off

    RX48 Resource Module Module LEDs:

    Fail Off

    Status Green

    Alarm Off

    Upstream Port LEDs:

    Link Off (unconfigured) orGreen

    Fault Off

    Table 2-1 BSR 64000 LED Display States Following Successful Booting

    Module LED LED Display State

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    26/138

    g

    2-6 Compass ID: 388931013 Version 4

    Interpreting Resource Module LED Displays

    The following sections describe the LED displays of the following BSR 64000

    Resource Modules:

    Supervisor Resource Module LEDs

    2:8 CMTS Resource Module LEDs

    2:8 CMTS Standby Resource Module LEDs

    TX32 Resource Module LEDs

    TX32 Standby Resource Module LEDs

    RX48 Resource Module LEDs

    RX48 Standby Resource Module LEDs

    Ether-FlexTMResource Module LEDs

    RX48 Standby Resource

    Module

    Module LEDs:

    Fail Off

    Status Blinking Green

    Alarm Off

    Upstream Port LEDs:

    Link Off Fault Off

    Ether-Flex Resource

    Module andEther-Flex

    Standby Resource

    Module

    Module LEDs:

    Fail Off

    Status Green

    Alarm Off

    GIG-E Port LEDs:

    Link Green

    ACT (Activity) Off orBlinking Green

    10/100 Port LEDs:

    Link Off (unconfigured) orGreen

    ACT (Activity) Off orBlinking Green

    Table 2-1 BSR 64000 LED Display States Following Successful Booting

    Module LED LED Display State

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    27/138

    Compass ID: 388931013 Version 4 2-7

    Supervisor Resource Module LEDs

    The SRM has the following groups of LEDs that indicate its operational status and the

    status of other chassis components:

    Module LEDs

    Fan Status LEDs

    Alarm LEDs

    The subsections that follow describe the display states of these LED groups.

    Module LEDs

    The SRM Module LEDs are visible on the module front panel and are labeled: Fail,

    Status, and Alarm.

    Table 2-2describes the possible display states of these LEDs during operation.

    Fan Status LEDs

    The SRM provides a set of Fan Status LEDs for each of the Fan Tray Modules

    installed in the BSR 64000 chassis. These LEDs are visible on the module front panel

    of the SRM and are labeled: OKand Fail. Table 2-3describes the possible display

    states of the LEDs. Separate LED status is available for the top (TOP) and bottom

    (BOT) Fan Tray Modules.

    Table 2-2 Module LED Display States for the SRM

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status. When operating in SRM redundant

    mode, indicates that the SRM is the Active SRM.

    Off Blinking

    Green

    Off Normal operating status. When operating in SRM redundant

    mode, indicates that the SRM is the Standby SRM.

    Off Green Red Failure. SRM is operating with an alarm condition.

    Red Off Off Indicates a module hardware failure.

    Red Off Red Failure. SRM is not operational.

    Red Green Red Reset. SRM is booting.

    Off Off Off Module is not receiving power.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    28/138

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    29/138

    Compass ID: 388931013 Version 4 2-9

    2:8 CMTS Resource Module LEDs

    2:8 CMTS Modules have two groups of LEDs that indicate the operational status of

    the module. These LED groupings include:

    Module LEDs

    Per-Port LEDs

    The following subsections describe the possible display states of these LED types.

    Module LEDs

    The Module LEDs are visible on the module front panel and are labeled: Fail,Status,

    and Alarm.

    Table 2-5describes the possible display states of these LEDs during non-redundant

    2:8 CMTS operation and Table 2-6describes the possible states of these LEDs during

    redundantoperation.

    Table 2-5 Module LED Display States for 2:8 CMTS Resource Module

    (Non-Redundant Operation)

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status.

    Off Green Red Failure. Module is operating with an alarm condition. Note: This

    sequence of LEDs occurs when an alarm condition is detected

    on individual upstream and downstream ports.

    Red Green Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    30/138

    2-10 Compass ID: 388931013 Version 4

    Per-Port LEDs

    Downstream ports and each upstream port have two LEDs associated with them to

    indicate their operational status. These LEDs are visible on the module front panel

    and are labeledLinkand Fault.

    Table 2-6 Module LED Display States for a 2:8 CMTS Resource Module

    (Redundant Operation)

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status. Module is providing service to the HFC

    network.

    Off Blinking

    Green

    Off A failure or other disruption has caused the 2:8 CMTS Standby

    Module to assume service to the HFC network. The module

    can be placed back in service by performing an administrative

    switch-over from the 2:8 CMTS Standby Module to this module.

    Off Blinking

    Green

    Red Failure. Module is operating with an alarm condition and service

    to the HFC network is switched over to the 2:8 CMTS Standby

    Module (if available).

    Note: This sequence of LEDs occurs when an alarm condition

    is detected on individual upstream and downstream ports.

    Red Blinking

    Green

    Off Indicates a module hardware failure and service to the HFC

    network is switched over to the 2:8 CMTS Standby Module.

    Red Off Red Failure. Module is not operational and service to the HFCnetwork is switched over to the 2:8 CMTS Standby Module (if

    available).

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Red Green Red Reset. Module is booting.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    31/138

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    32/138

    2-12 Compass ID: 388931013 Version 4

    Table 2-8describes the possible display states of these LEDs during operation.

    Per-Port LEDs

    The 2:8 CMTS downstream ports and each upstream port have two LEDs to indicate

    their operational status. These LEDs are visible on the module front panel and arelabeledLinkand Fault.

    Port LEDs are grouped vertically. A number to the right of each LED group indicates

    the channel number associated with the group. The two downstream channels are

    numbered 0 and 1, and the eight upstream channels are numbered 0 through 7.

    Table 2-8 Module LED Display States for the 2:8 CMTS Standby Resource Module

    Fail Status Alarm Interpretation

    Off Blinking

    Green

    Off Normal operating status. Module is in a Standby state.

    Off Green Off Normal operating status. Module has assumed service for a

    primary 2:8 CMTS module.

    Off Green Red Failure. Module is operating with an alarm condition.Note: Thissequence of LEDs occurs when an alarm condition is detected

    on individual upstream and downstream ports.

    Red Off Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Red Green Red Reset. Module is booting.Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    33/138

    Compass ID: 388931013 Version 4 2-13

    Table 2-9describes the possible display states of these LEDs when the module has

    assumed service for a primary 2:8 CMTS module. When the module is operating in

    Standby mode, the LEDs should all be Off.

    TX32 Resource Module LEDs

    TX32 Resource Modules have two groups of LEDs that indicate the operational status

    of the module. These LED groupings include:

    Module LEDs

    Per-Port LEDs

    The following subsections describe the possible display states of these LED types.

    Module LEDs

    The Module LEDs are visible on the module front panel and are labeled: Fail,Status,

    and Alarm.

    Table 2-9 2:8 CMTS Standby Module Downstream and Upstream

    Per-Port LED Display States

    Link Fault Interpretation

    Green Off Normal operating status when module has assumed service for

    a primary 2:8 CMTS module and the port is configured.

    Green Red Operating with an alarm condition detected. Note:An alarm

    condition detected for an individual port also causes the System

    Alarm LED to light.

    Off Red Failed port. Port is not operational.

    Off Off Port is not configured. Note: Check module LEDs to determine

    if the module is receiving power.

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    34/138

    2-14 Compass ID: 388931013 Version 4

    Table 2-10describes the possible display states of these LEDs during non-redundant

    operation and Table 2-11describes the possible states of these LEDs during

    redundantoperation.

    Table 2-10 Module LED Display States for the TX32 Resource Module

    (Non-Redundant Operation)

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status.

    Off Green Red Failure. Module is operating with an alarm condition. Note: This

    sequence of LED occurs when an alarm condition is detected

    on individual downstream ports.

    Red Green Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Off Off Off Module is not receiving power or is not secured in the chassisthrough its module ejectors and integrated ejector switch.

    Table 2-11 Module LED Display States for the TX32 Resource Module

    (Redundant Operation)

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status. Module is capable of providing service

    to the HFC network.

    Off Blinking

    Green

    Off A failure or other disruption has caused the TX32 Standby

    Module to assume service to the HFC network. The module can

    be placed back in service by performing an administrative

    switch-over from the TX32 Standby Module to this module.

    Off Blinking

    Green

    Red Failure. Module is operating with an alarm condition and service

    to the HFC network is switched over to the TX32 Standby

    Module (if available).

    Note: This sequence of LED occurs when an alarm condition is

    detected on individual upstream and downstream ports.

    Red Blinking

    Green

    Off Indicates a module hardware failure and service to the HFC

    network is switched over to the TX32 Standby Module (if

    available).

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    35/138

    Compass ID: 388931013 Version 4 2-15

    Per-Port LEDs

    Downstream ports each have two LEDs associated with them to indicate their

    operational status. These LEDs are visible on the module front panel and are labeled

    Linkand Fault.

    Port LEDs are grouped vertically. A number to the right each LED group indicates the

    channel number associated with the group. The eight downstream channels are

    numbered 0 through 7.

    Table 2-12describes the possible display states of these LEDs during operation.

    Red Off Red Failure. Module is not operational and service to the HFC

    network is switched over to the TX32 Standby Module (if

    available).

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Red Green Red Reset. Module is booting.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    Table 2-12 TX32 Downstream Per-Port LED Display States

    Link Fault Interpretation

    Green Off Normal operating status when configured.

    Green Red Operating with an alarm condition detected. Note:An alarm

    condition detected for an individual port also causes the System

    Alarm LED to light.

    Off Red Failed port. Port is not operational.

    Off Off Port is not configured.

    Note: Check module LEDs to determine if the module is

    receiving power.

    Table 2-11 Module LED Display States for the TX32 Resource Module

    (Redundant Operation)

    Fail Status Alarm Interpretation

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    36/138

    2-16 Compass ID: 388931013 Version 4

    TX32 Standby Resource Module LEDs

    TX32 Standby Resource Modules have two groups of LEDs that indicate the

    operational status of the module. These LED groupings include:

    Module LEDs

    Per-Port LEDs

    The following subsections describe the possible display states of these LED types.

    Module LEDs

    The Module LEDs are visible on the module front panel and are labeled: Fail,Status,and Alarm.

    Table 2-13describes the possible display states of these LEDs when the module has

    assumed service for a primary TX32 module. When the module is operating in

    Standby mode, the LEDs should all be Off.

    Table 2-13 Module LED Display States for the TX32 Standby Resource Module

    Fail Status Alarm Interpretation

    Off Blinking

    Green

    Off Normal operating status. Module is in a Standby state.

    Off Green Off Normal operating status. Module has assumed service for a

    primary TX32 module.

    Off Green Red Failure. Module is operating with an alarm condition.Note: This

    sequence of LED occurs when an alarm condition is detectedon individual upstream and downstream ports.

    Red Off Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Red Green Red Reset. Module is booting.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    37/138

    Compass ID: 388931013 Version 4 2-17

    Per-Port LEDs

    Downstream ports each have two LEDs associated with them to indicate their

    operational status. These LEDs are visible on the module front panel and are labeledLinkand Fault.

    Port LEDs are grouped vertically. A number to the right each LED group indicates the

    channel number associated with the group. The eight downstream channels are

    numbered 0 through 7.

    Table 2-14describes the possible display states of these LEDs during operation.

    RX48 Resource Module LEDs

    RX48 Modules have two groups of LEDs that indicate the operational status of the

    module. These LED groupings include:

    Module LEDs

    Per-Port LEDs

    The following subsections describe the possible display states of these LED types.

    Module LEDsThe Module LEDs are visible on the module front panel and are labeled: Fail,Status,

    and Alarm.

    Table 2-14 TX32 Standby Downstream Per-Port LED Display States

    Link Fault Interpretation

    Green Off Normal operating status when configured.

    Green Red Operating with an alarm condition detected. Note:An alarm

    condition detected for an individual port also causes the System

    Alarm LED to light.

    Off Red Failed port. Port is not operational.

    Off Off Port is not configured.

    Note: Check module LEDs to determine if the module is

    receiving power.

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    38/138

    2-18 Compass ID: 388931013 Version 4

    Table 2-15describes the possible display states of these LEDs during non-redundant

    operation. Table 2-16describes the possible display states of these LEDs during

    redundant operation.

    Table 2-15 Module LED Display States for the RX48 Resource Module

    (Non-Redundant Operation)

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status when configured.

    Off Green Red Failure. Module is operating with an alarm condition. Note: This

    sequence of LED occurs when an alarm condition is detected

    on individual upstream ports.

    Red Green Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Off Off Off Module is not receiving power or is not secured in the chassisthrough its module ejectors and integrated ejector switch.

    Table 2-16 Module LED Display States for the RX48 Resource Module

    (Redundant Operation)

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status. Module is capable of providing service

    to the HFC network

    Off Blinking

    Green

    Off A failure or other disruption has caused the RX48 Standby

    Module to assume service to the HFC network. The module can

    be placed back in service by performing an administrative

    switch-over from the RX48 Standby Module to this module.

    Off Blinking

    Green

    Red Failure. Module is operating with an alarm condition and service

    to the HFC network is switched over to the RX48 Standby

    Module (if available).

    Note: This sequence of LED occurs when an alarm condition is

    detected on individual upstream and downstream ports.

    Red Blinking

    Green

    Off Indicates a module hardware failure and service to the HFC

    network is switched over to the RX48 Standby Module (if

    available).

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    39/138

    Compass ID: 388931013 Version 4 2-19

    Per-Port LEDs

    Upstream port have two LEDs associated with them to indicate their operational

    status. These LEDs are visible on the module front panel and are labeledLinkand

    Fault.

    Port LEDs are grouped vertically. A number to the right each LED group indicates the

    channel number associated with the group. The eight upstream channels are numbered

    0 through 7.

    Table 2-17describes the possible display states of these LEDs during operation.

    Red Off Red Failure. Module is not operational and service to the HFC

    network is switched over to the RX48 Standby Module (if

    available).

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Red Green Red Reset. Module is booting.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    Table 2-17 RX48 Upstream Per-Port LED Display States

    Link Fault Interpretation

    Green Off Normal operating status.

    Green Red Operating with an alarm condition detected. Note:An alarm

    condition detected for an individual port also causes the System

    Alarm LED to light.

    Off Red Failed port. Port is not operational.

    Off Off Port is not configured.

    Note: Check module LEDs to determine if the module is

    receiving power.

    Table 2-16 Module LED Display States for the RX48 Resource Module

    (Redundant Operation)

    Fail Status Alarm Interpretation

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    40/138

    2-20 Compass ID: 388931013 Version 4

    Diagnostic Ethernet Port LEDs

    The RX48 front panel includes an RJ45 connector for use as a 1000BaseT diagnostic

    Ethernet Port. The RJ45 connector includes two integrated LEDs, one green and oneyellow. The Green LED provides Link status of the port. The yellow LED indicates

    port activity.

    RX48 Standby Resource Module LEDs

    RX48 Standby Modules have two groups of LEDs that indicate the operational status

    of the module. These LED groupings include:

    Module LEDs

    Per-Port LEDs

    The following subsections describe the possible display states of these LED types.

    Module LEDs

    The Module LEDs are visible on the module front panel and are labeled: Fail,Status,

    and Alarm.Table 2-18describes the possible display states of these LEDs when the module has

    assumed service for a primary RX48 module. When the module is operating in

    Standby mode, the LEDs should all be Off.

    Table 2-18 Module LED Display States for RX48 Standby Resource Module

    Fail Status Alarm Interpretation

    Off Blinking

    Green

    Off Normal operating status. Module is in Standby state.

    Off Green Off Normal operating status. Module has assumed service for a

    primary RX48 Module.

    Off Green Red Failure. Module is operating with an alarm condition. Note: This

    sequence of LED occurs when an alarm condition is detected

    on individual upstream ports.

    Red Green Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Off Off Failure. Power (-48 Vdc) is present at the midplane connector

    but not at the module. May indicate a blown fuse on the module.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    Release 6.3.1 Checking Physical Equipment

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    41/138

    Compass ID: 388931013 Version 4 2-21

    Per-Port LEDs

    Upstream port have two LEDs associated with them to indicate their operational

    status. These LEDs are visible on the module front panel and are labeledLinkandFault.

    Port LEDs are grouped vertically. A number to the right each LED group indicates the

    channel number associated with the group. The eight upstream channels are numbered

    0 through 7.

    Table 2-19describes the possible display states of these LEDs during operation.

    Diagnostic Ethernet Port LEDs

    The RX48 Standby front panel includes an RJ45 connector for use as a 1000BaseTdiagnostic Ethernet Port. The RJ45 connector includes two integrated LEDs, one

    green and one yellow. The Green LED provides Link status of the port. The yellow

    LED indicates port activity.

    Table 2-19 RX48 Standby Upstream Per-Port LED Display States

    Link Fault Interpretation

    Green Off Normal operating status.

    Green Red Operating with an alarm condition detected. Note:An alarm

    condition detected for an individual port also causes the System

    Alarm LED to light.

    Off Red Failed port. Port is not operational.

    Off Off Port is not configured.

    Note: Check module LEDs to determine if the module is

    receiving power.

    BSR 64000 Troubleshooting Guide Release 6.3.1

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    42/138

    2-22 Compass ID: 388931013 Version 4

    Ether-FlexTMResource Module LEDs

    The Ether-FlexTMResource Module has the following groups of LEDs that indicate its

    operational status and the status of other chassis components:

    Module LEDs

    GIG-E Port LEDs

    Ethernet Port LEDs

    The following subsections describe the possible display states of these LED groups.

    Module LEDsThe Module LEDs are visible on the module front panel and are labeled: Fail,Status,

    and Alarm.

    Table 2-20describes the possible display states of these LEDs during operation.

    GIG-E Port LEDs

    Ether-FlexTMResource Modules support two optical ports (either single-mode or

    multimode), labeled 0and 1. Each port has two LEDs associated with it to indicate the

    ports operational status. The LEDs are visible on the module front panel and arelabeledLinkand ACT.

    Table 2-20 Module LED Display States for the Gigabit Ethernet Resource Module

    Fail Status Alarm Interpretation

    Off Green Off Normal operating status.

    Off Green Red Failure. Module is operating with an alarm condition.

    Red Off Off Indicates a module hardware failure.

    Red Off Red Failure. Module is not operational.

    Red Green Red Reset. Module is booting.

    Off Off Off Module is not receiving power or is not secured in the chassis

    through its module ejectors and integrated ejector switch.

    Release 6.3.1 Checking Physical Equipment

    h b ( ) h i h f h i di h b

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    43/138

    Compass ID: 388931013 Version 4 2-23

    The number (0 or 1) to the right of each LED group indicates the port number

    associated with the group. Table 2-21describes the possible display states of these

    LEDs during operation.

    Ethernet (10/100) Port LEDs

    Ether-FlexTMResource Modules support eight 10/100 BASE-T Ethernet ports, labeled

    0through 7. Each Ethernet port on the module has two LEDs associated with it to

    indicate the ports operational status. The Ethernet port LEDs are visible on the

    module front panel and are labeledLinkand ACT.

    Port LEDs are grouped vertically. Table 2-22describes the possible display states of

    the 10/100 Ethernet Port LEDs during operation.

    Table 2-21 Gig-E Port LED Display States

    Link ACT Interpretation

    Green Flashing

    Green

    Normal operating status when configured and connected to an

    external device. Link is established, and the ACT LED flashes to

    show activity on the port.

    Off Off Port is not configured, has failed, is not operational, or is notconnected.

    Note:Check module LEDs to determine if the module is receiving

    power.

    Green Off Link without activity.

    Table 2-22 8-Port 10/100 Port LED Display States

    Link ACT Interpretation

    Green Flashing

    Green

    Normal operating status when configured and connected to an

    external device. Link is established, and the ACT LED flashes to

    show activity on the port.

    Off Off Port is not configured, has failed, is not operational, or is not

    connected.

    Note:Check module LEDs to determine if the module is receiving

    power.

    Green Off Link without activity.

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    44/138

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    45/138

    Compass ID: 388931013 Version 4 3-1

    3Troubleshooting the CMTS

    Overview

    This chapter provides troubleshooting solutions tosomecommon DOCSIS network

    problems with the following:

    Using Flap Lists to Troubleshoot CM Problems

    Resolving HFC Network Performance Problems

    Resolving Problems on the Upstream Path

    Resolving Problems on the Downstream Path

    Resolving Cable Modem Problems

    BSR 64000 Troubleshooting Guide Release 6.3.1

    Using Flap Lists to Troubleshoot CM Problems

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    46/138

    3-2 Compass ID: 388931013 Version 4

    Using Flap Lists to Troubleshoot CM Problems

    The BSR maintains a database of flapping CMs to assist in locating cable plant

    problems. The flap list feature tracks the upstream and downstream performance of all

    CMs on the network, without impacting throughput and performance between the CM

    and BSR, or creating additional packet overhead on the HFC network.

    Refer to theBSR 64000 CMTS Configuration and Management Guidefor more

    information on configuring flap-list settings.

    Viewing Flap List Statistics to Identify Network Health

    Flap lists are used to collect statistics for determining CM problems on the network.

    There are several different options for sorting flap list statistics. The CM flap list

    keeps track of the CM MAC address, up and down transitions, registration events,

    missed periodic ranging packets, upstream power adjustments on the BSR. This

    section describes the different sorting options and describes the command output

    fields.

    CMs appear in the flap list when any of the following conditions are detected: The CM re-registers more frequently than the configured insertion time.

    Intermittent keepalive messages are detected between the BSR and the CM.

    The CM upstream transmit power changes beyond the configured power adjust

    threshold.

    Follow these steps to view flap list statistics by using different sorting options:

    1. Issue the show cable flap-listcommand, in Privileged EXEC mode, to view allflap list statistics for CMs:

    MOT:7A#show cable flap-list

    The following is typical show cable flap-list command output:

    Figure 3-1 show cable flap-list Command Output

    MAC ID CableIF Hit Miss Ins CRC Pow Rng Flap Type Time

    000b.0643.3b60 4/0 U1 1469 7 0 3 0 1 1 Rng FRI NOV 05 11:59:39

    000b.0643.36c8 4/0 U1 1469 7 0 0 0 1 1 Rng FRI NOV 05 11:59:40

    000b.0643.3b78 4/0 U1 1469 6 0 0 0 1 1 Rng FRI NOV 05 11:59:40

    Release 6.3.1 Troubleshooting the CMTS

    2. Issue the show cable flap-list sort-flap command, in Privileged EXEC mode, to

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    47/138

    Compass ID: 388931013 Version 4 3-3

    2. Issue the show cable flap list sort flapcommand, in Privileged EXEC mode, to

    sort the flap list statistics in ascending order by the CM flap:

    MOT:7A#show cable flap-list sort-flap

    The following is typical show cable flap-list sort-flap command output:

    Figure 3-2 show cable flap-list sort-flap Command Output

    3. Issue the show cable flap-list sort-timecommand, in Privileged EXEC mode, to

    sort the flap list statistics in ascending order by the time at which the CM flapoccurred:

    MOT:7A#show cable flap-list sort-time

    The following is typical show cable flap-list sort-time command output:

    Figure 3-3 show cable flap-list sort-time Command Output

    4. Issue the show cable flap-list sort-interfacecommand, in Privileged EXEC

    mode, to sort the flap list statistics in ascending order by the cable upstream

    interface on which the CM flap occurred:

    MOT:7A#show cable flap-list sort-interface

    MAC ID CableIF Hit Miss Ins Pow Rng Flap Type Time

    0020.403d.a764 4/0 U0 5519 0 1 2877 0 2878 PAdj TUE JAN 13 07:58:24 2004

    0050.f112.2144 3/1 UO 65 0 1 2566 0 794 PAdj TUE JAN 13 07:19:47 2004

    0050.f112.2296 2/1 UO 18 0 1 1234 0 553 PAdj TUE JAN 13 07:21:05 2004

    0010.9504.a92b 3/0 UO 148 0 1 2344 0 181 PAdj TUE JAN 13 07:34:32 2004

    MAC ID CableIF Hit Miss Ins Pow Rng Flap Type Time

    0020.403d.a764 4/0 U0 5519 0 1 2877 0 2878 PAdj TUE JAN 13 07:58:24 2004

    0050.f112.2144 3/1 UO 65 0 1 2566 0 794 PAdj TUE JAN 13 07:34:32 20040050.f112.2296 2/1 UO 18 0 1 1234 0 553 PAdj TUE JAN 13 07:21:05 2004

    0010.9504.a92b 3/0 UO 148 0 1 2344 0 181 PAdj TUE JAN 13 07:19:47 2004

    BSR 64000 Troubleshooting Guide Release 6.3.1

    The following is typical show cable flap-list sort-interface command output:

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    48/138

    3-4 Compass ID: 388931013 Version 4

    g yp p p

    Figure 3-4 show cable flap-list sort-interface Command Output

    Table 3-1identifies the flap list command output column field identifications:

    Table 3-1 Flap List Command Output Identifications

    Field Identification

    MAC ID Lists the MAC addresses of the CMs sorted by the flap rate or mostrecent flap time. The first six digits in the CM MAC address indicate

    the vendor ID of the CM manufacturer, followed by six digits indicating

    a unique host address. Each CM MAC address is unique.

    Cable IF Indicates the cable interface up/down flap. This is the cable interface

    on the BSR 64000 CMTS module. It denotes the CMTS module slot

    number, the downstream and the upstream port number. The flap list

    data can be sorted based on the upstream port number which is

    useful when isolating reverse path problems unique to certaincombining groups.

    Hit The Hit and Miss column fields detect the intermittent upstream; the

    keepalive hits versus misses is the number of times CMs do not

    respond to the MAC layer keepalive messages. If there are a number

    of misses, this points to a potential upstream problem.

    Miss

    Ins The Insertions Link process is used by a CM to perform an initial

    maintenance procedure to establish a connection with the BSR. The

    Ins column is the flapping CMs (re-) insertion count and indicates the

    number of times the a CM starts and inserts into the network in an

    abnormal way. An abnormality is detected when the time between link

    re-establishment attempts is less than the user-configurable

    parameter. This function can identify potential problems in the

    downstream interface such as incorrectly provisioned CMs repeatedly

    trying to re-establish a link.

    MAC ID CableIF Hit Miss Ins Pow Rng Flap Type Time

    0020.403d.a764 4/0 U0 5519 0 1 2877 0 2878 PAdj TUE JAN 13 07:58:24 20040050.f112.2144 3/1 UO 65 0 1 2566 0 794 PAdj TUE JAN 13 07:19:47 2004

    0050.f112.2296 3/0 UO 18 0 1 1234 0 553 PAdj TUE JAN 13 07:21:05 2004

    0010.9504.a92b 2/1 UO 148 0 1 2344 0 181 PAdj TUE JAN 13 07:34:32 2004

    Release 6.3.1 Troubleshooting the CMTS

    Table 3-1 Flap List Command Output Identifications

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    49/138

    Compass ID: 388931013 Version 4 3-5

    Interpreting Flap List Statistics

    This section describes how to interpret flap list statistics in order to troubleshoot thecable network.

    CM activity follows the sequence below:

    Power-on

    Initial maintenance

    Station maintenance

    Power-off

    The initial link insertion is followed by a keepalive loop between the BSR and CM

    and is called station maintenance. When the link is broken, initial maintenance is

    repeated to re-establish the link.

    Initial maintenance @ Time T1

    Station maintenance

    CRC Displays the count of CRC errors for each cable modem on the

    flap-list. This count is also saved in the cable modem history record so

    that the count remains valid if cable modems flap. The count is a sum

    of all of the CRC errors for each service flow tied to a cable modem.

    Pow The Power Adjustment column field shows power adjustment

    statistics during station maintenance polling. This column indicates

    the number of times the BSR tells a CM to adjust the transmit power

    more than the configured threshold. If constant power adjustments are

    detected, an amplifier problem is usually the cause. The source of

    failure is found by viewing CMs either in front or behind various

    amplifiers.

    Rng Response to RNG-RSP messages from affected CMs. The cable

    modem is online and sent a periodic Ranging Request (RNG-REQ)

    message to the CMTS, but it received an Abort Ranging reply from the

    CMTS.

    Flap Total of Pow, Rng, and Ins values. High flap-count CMs have highSIDs and may not register.

    Type Type of error event which triggered the CM to flap.

    Time Indicates the most recent time a flap has occurred for a particular CM.

    Field Identification

    BSR 64000 Troubleshooting Guide Release 6.3.1

    Init maintenance @ Time T2

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    50/138

    3-6 Compass ID: 388931013 Version 4

    The Insand Flapcounters in the flap list are incremented whenever T2 T1 < N

    where Nis the insertion-time parameter configured using the cable flap-listinsertion-timecommand. The default value for this parameter is 60 seconds.

    Use the following cause or symptom observations to interpret flap list activity and

    solve CM problems:

    Table 3-2 Troubleshooting CM Problems

    Cause or Symptom Problem

    Subscriber CM shows a lot of flap list

    activity

    CM is having communication problems with

    the BSR.

    Subscriber CM shows little or no flap list

    activity.

    The CM is communicating with the BSR

    effectively, however there is still a problem.

    The problem can be isolated to the

    subscribers CPE computer equipment or the

    CM connection.

    Ten percent of the CMs in the flap listshow a lot of activity.

    These CMs are most likely having difficultiescommunicating with the BSR.

    CMs have a lot of power adjustment

    (P-Adj) errors.

    CMs have problems with their physical

    upstream paths or in-home wiring problems.

    Use corresponding CMs on the same physical

    upstream port interface with similar flap list

    statistics to quickly resolve problems outside

    the cable plant to a particular node or

    geographic location.

    All CMs are incrementing the insertion at

    the same time.

    There is a provisioning server failure.

    A CM has more than 50 power

    adjustments per day.

    The CM has a suspect upstream path.

    Corresponding CMs on the same physical

    upstream port interface with similar flap list

    statistics can be used to quickly resolve

    problems outside the cable plant to aparticular node or geographic location.

    A CM has roughly the same number of

    hits and misses and contain a lot of

    insertions.

    There is a problematic downstream path. For

    example, the downstream power level to the

    CM may have a power level that is too low.

    A high flap list insertion (Ins) time

    number.

    Intermittent downstream synchronization loss.

    DHCP or CM registration problems.

    Release 6.3.1 Troubleshooting the CMTS

    Table 3-2 Troubleshooting CM Problems

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    51/138

    Compass ID: 388931013 Version 4 3-7

    Low miss/hit ratio, low insertion, low

    P-adj, low flap counter and old

    timestamp.

    Indicates an optimal network situation.

    High ratio of misses over hits (> 10%) Hit/miss analysis should be done after the

    "Ins" count stops incrementing. In general, if

    the hit and miss counts are about the same

    order of magnitude, then the upstream may

    be experiencing noise. If the miss count is

    greater, then the CM is probably dropping out

    frequently and not completing registration.

    The upstream or downstream is perhaps not

    stable enough for reliable link establishment.

    Very low hits and miss counters and high

    insertion counters indicate provisioning

    problems.

    High power adjustment counter. Indicates the power adjustment threshold is

    probably set at default value of 2 dBadjustment. The CM transmitter step size is

    1.5 dB, whereas the headend may command

    0.25 dB step sizes. Tuning the power

    threshold to 6 dB is recommended to

    decrease irrelevant entries in the flap list. The

    power adjustment threshold may be set using

    the cable flap-list power-adjust threshold

    command from Global Configuration mode. Aproperly operating HFC network with short

    amplifier cascades can use a 2-3 dB

    threshold.

    Cause or Symptom Problem

    BSR 64000 Troubleshooting Guide Release 6.3.1

    Table 3-2 Troubleshooting CM Problems

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    52/138

    3-8 Compass ID: 388931013 Version 4

    High P-Adj (power adjustment) This condition can indicate that the fiber node

    is clipping the upstream return laser. Evaluate

    the CMs with the highest number of

    correcteds and uncorrecteds first. If the CMs

    are not going offline (Ins = 0), this will not be

    noticed by the subscriber. However, they

    could receive slower service due to dropped

    IP packets in the upstream. This condition will

    also result in input errors on the cableinterface.

    High insertion rate. If link re-establishment happens too

    frequently, then the CM is usually having a

    registration problem. This is indicated by a

    high Ins counter which tracks the Flap

    counter.

    Note: CMs go offline faster than the frequency hop period and can cause the

    frequency to stay fixed while CMs go offline. Reduce the hop period to 10

    seconds to adjust to the hop frequency period.

    Cause or Symptom Problem

    Release 6.3.1 Troubleshooting the CMTS

    Table 3-3describes how to interpret flap list statistics:

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    53/138

    Compass ID: 388931013 Version 4 3-9

    Table 3-3 Flap List Statistic Interpretations

    Field Description

    Hit and Miss The HIT and MISS columns are keepalive polling statistics between the

    BSR and the CM. The station maintenance process occurs for every CM

    approximately every 10 seconds. When the BSR receives a response

    from the CM, the event is counted as a Hit. If the BSR does not receive a

    response from the CM, the event is counted as a Miss. A CM will fail to

    respond either because of noise or if it is down. CMs which only log

    Misses and zero Hits are assumed to be powered off.

    Misses are not desirable since this is usually an indication of a return path

    problem; however, having a small number of misses is normal. The flap

    count is incremented if there are M consecutive misses where M is

    configured in the cable flap-list miss-thresholdparameter. The

    parameter value ranges from 1-12 with a default of 6.

    Ideally, the HIT count should be much greater than the Miss counts. If a

    CM has a HIT count much less than its MISS count, then registration is

    failing. Noisy links cause the MISS/HIT ratio to deviate from a nominal 1%or less. High Miss counts can indicate:

    Intermittent upstream possibly due to noise

    Laser clipping

    Common-path distortion

    Ingress or interference

    Too much or too little upstream attenuation

    P-Adj The station maintenance poll in the BSR constantly adjusts the CMtransmit power, frequency, and timing. The Power Adjustments (P-Adj)

    column indicates the number of times the CMs power adjustment

    exceeded the threshold value. The power adjustment threshold may be

    set using the cable flap-list power-adjust thresholdcommand with a

    value range of 0-10 dB and a default value of 2 dB. Tuning this threshold

    is recommended to decrease irrelevant entries in the flap list. Power

    Adjustment values of 2 dB and below will continuously increment the

    P-Adj counter. The CM transmitter step size is 1.5 dB, whereas the

    headend may command 0.25 dB step sizes. Power adjustment flap

    strongly suggests upstream plant problems such as:

    Amplifier degradation

    Poor connections

    Thermal sensitivity

    Attenuation problem

    BSR 64000 Troubleshooting Guide Release 6.3.1

    Table 3-3 Flap List Statistic Interpretations

  • 8/10/2019 BSR64K R6.3.1 TroubleshootingGuide

    54/138

    3-10 Compass ID: 388931013 Version 4

    Tips for Administrating Flap Lists

    Follow these suggestions for administrating flap lists:

    Write script(s) to periodically poll the flap list.

    Analyze and identify CM trends from the flap list data.

    Query the billing and administrative database for CM MAC address-to-street

    address translation and generate reports. These reports can then be given to the

    Customer Service Department or the cable plants Operations and Maintenance

    Department. Maintenance personnel use the reports to see patterns of flapping

    CMs, street addresses, and flap statistics that indicate which amplifier or feeder

    lines are faulty. The reports also help troubleshoot problems