Upload
derrick-senyo
View
65
Download
0
Embed Size (px)
DESCRIPTION
Hughes radio operations and maintenance
Citation preview
1 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Hughes Network Systems
AiReach Broadband 9000
Troubleshooting and Maintenance
Training Course
September 2011
3 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Reference Documents
Hub and Remote Terminal Site Preparation Guide Hub Terminal Installation and User Manual Remote Terminal Installation and User Manual EMS Reference Manual System Operations Manual Element Management System (EMS) Installation and
Maintenance Manual
All Manuals are online on the EMS and on the EMS software CD-ROM
4 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Course Topics
Troubleshooting– Isolating Problems– Installation Troubleshooting– System Maintenance– Operational Troubleshooting
Support Contact– TAC Problem Reporting– TAC Website Use
5 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Before You Start
PC Settings for LCT (Local Craft Terminal)– 9600, 8, none, 1– No handshaking
LCT Cable:
LCT Login– Username: lct– Password: hnslct
7
6
5
4
3
2
6
5
4
3
2
1
9
8
7
Correct LCT Cable
4 wire, 8 positionmodular
plug
DB-9FBackshell
Correct N/C
N/CN/C = No Connection
N/C
N/C
N/C
N/C
N/C
N/C
N/C
1
8
6
5
4
3
2
1
4 wire, 6 positionmodular
plug
N/C
N/C
N/C
N/C
6 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Troubleshooting
7 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Troubleshooting Tree
Troubleshooting Categories
InstallationAnd
Commissioning
OperationsAnd
Maintenance
For each category above:– The starting points are different, but– Some of the troubleshooting techniques overlap
8 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Isolating Problems Most problems identified during installation are more
likely to be physical/mechanical defects, problems created by human error at the time of the component configuration at the EMS, or parameters entered in the IDU during commissioning.
Most problems identified during normal operation tend to be categorized as network, RF, or specific service related failures that are recognized by the current alarms reported to the EMS.
It is possible to experience operational problems that may normally be classified as occurring during installation, and vice versa.
Regular system maintenance can help to eliminate some problems by identifying them as they develop rather than after they become a service impact.
9 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Installation Troubleshooting
10 Troubleshooting and Maintenance HUGHES PROPRIETARY II
HT Installation Troubleshooting
a) Power On
b) Start Up Diagnostics (SUDs)
c) Bootp request failed
d) Disabled Failed
e) Disabled Off Duty
f) Enabled Degraded
11 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Power On
Upon powering up the IDU the power LED should be solid green color
Verify input power is within the operating range:– +/- 48vdc: +/- 40.5vdc to +/- 57vdc
Does rectifier have enough current capability? If the IDU does not power on, remove the IFL cable
and try again. If it powers on, there is a short in the IFL cable.
12 Troubleshooting and Maintenance HUGHES PROPRIETARY II
SUDs (Start Up Diagnostics)
A failure during SUDs indicates one of the IDU self tests has failed.
If there is a failure the “status LED” will remain off, or be illuminated a solid color, and the LCT can not be accessed.
If the LCT is not accessible and the status LED is flashing, check your HyperTerminal settings (9600, 8, N, 1) and your serial cable.
Replace the IDU if SUDS fails
13 Troubleshooting and Maintenance HUGHES PROPRIETARY II
HT Bootup
On the PC LCT you should see the following (or similar) after power up.
It can take about 20 seconds to see some output from the IDU.
ROM_DISK_START address = 0xfe600000 ,size = 20971520ROMDISK OK. checking BSDBSD present in the ROMDISK
VxWorks System Boot
Copyright 1984-1998 Wind River Systems, Inc.
CPU: Motorola - PowerPC 8280Version: 5.4.2BSP version: 1.2/2Creation date: Suppressed Date/Time
14 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Bootp Request Fails
After exiting the “install mode” the HT will send a boot request to the EMS
booting...
boot device : lnunit number : 0 processor number : 0 file name : /aireach/ems/DBA/SWREL/R011307/HJBA1307.CPZinet on ethernet (e) : 10.10.10.10:ffffff00host inet (h) : 10.10.10.1gateway inet (g) : 10.10.10.1user (u) : anonftp password (pw) : anonflags (f) : 0x40
..........Bootp request failed after 10 retries...
15 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Bootp Request Fails
If after a few minutes the message “Bootp request failed after 10 retries” is received, there was no response from the EMS
Perform the following:– Verify HT MAC on EMS Hub Terminal screen.– On the hub site router:
• verify the “Helper Address” is defined to point the bootp broadcast message to the EMS IP address.
– Verify Network Management Connectivity (next slide)
16 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Verify Network Management Path
Verify LAN cable is connected to LAN port 3 on the 9800 IDU Verify the path to the hub router from the EMS by using the
“ping” command. Verify the path using “trace route” command. Determine if a point between the EMS and HT is not reachable. If the problem is in the DCN contact the DCN support staff. If the last router to the hub site is reachable, a site visit may be
required to verify the device interface cable integrity, or to verify the device is correctly configured.
TIP: You can use your laptop to verify connectivity from the Hub site (or router location). Set the IP of your laptop to the IP of any HT on the site. See if you can ping the near router interface, far router interface and the EMS IP address. Also, ask the EMS Operator to ping your laptop.
17 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Verify Network Management Path
If you are using E1 TDM in-band management:– Use the LCT to show the status of the E1 Physical link
– Check the E1 physical link status on the router
– Verify that the number of timeslots matches
– If there are any E1 alarms, the HT will not be able to boot
– While the HT is in the process of rebooting (after trying to get the boot reply from the EMS), the router will show LOS on the E1 line… this is normal. The E1 is only active when it is trying to send and receive the bootp messages.
If you are using ATM in-band management:– Use the LCT to show the status of the STM-1/E3 Physical link
– Check the STM-1 physical link status on the router or switch
– Verify that the management VP/VC is correct
18 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Disabled Off Duty
========================================================================== AB9800 Hub Terminal Main Menu Main MenuLCT Mode:Normal[RAM], PtP Mode:OFF, PtP Install Mode:OFFLocal Config Install:OFF, Local Config Normal:OFF, EMS Connectivity:LAN========================================================================== 1. Front Panel Display 2. Site Specific Information 3. EMS Port Information 4. Status 5. Statistics 6. Installation 7. Reset History 8. Reset 9. Software Revision 10. Set/Clear Configuration Mode 11. Local Configuration 12. Baud Rate Setting 13. Update Base Application in Flash 14. Update Boot ROM in Flash 15. Update FPGA in Flash 16. Shutdown 17. Logout
Enter your choice from the menu, OR 'p' for previous menu, 'm' for main menuYour Choice: 1
========================================================================== Front Panel Display========================================================================== HT Freq Disabled OffDutyPress Enter to continue...
19 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Disabled Off Duty
Disabled Off Duty indicates the IDU has not received all configuration from the EMS yet: it’s either in process of downloading it or it failed to complete FTP process with the EMS.
To find out if this is the case, logon to LCT and check “Status” – “Element Software”. If state is listed as “Unknown” or message says “Reconciling” and it stays that way for more than 10 minutes – most likely that FTP process has failed.
Check router configuration – make sure that:– “ip directed broadcast” and – “ip irdp” are setup on the Hub Router interface.
Check if company firewall or router access list maybe enabled and blocking FTP/ IPs range
20 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Disabled Failed
With the HT successfully downloaded from the EMS, (“Status” – “Element Software” shows “Reconciled” state) the final state shows:– HT Freq XX– Disabled Failed
Use the “Status option” on the LCT menu to identify the hardware failure. If ODU is failed…replace ODU. If IDU is failed…replace the IDU.
21 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Enabled Degraded
Enabled Degraded indicates the IDU is Enabled and transmitting, but one of the configured service ports does not have an interface cable installed or is not configured correctly.
22 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Other Issues
Sometimes if all other indications are correct, it is possible that the EMS database needs to be refreshed. On the EMS, the operator must run this command:– $ DescriptorRegenAll.ksh
This will force the HT configuration files to be rewritten and downloaded
Note: Please, be aware that if, for some reason, there were no communication between the EMS and HUB(s) at the time when configuration changes have been made at the EMS – changes will take immediate effect upon execution above command and/or when communication has been restored
23 Troubleshooting and Maintenance HUGHES PROPRIETARY II
RT Installation Troubleshooting
a) Power On
b) Start Up Diagnostics (SUDs)
c) Unable to read Frequency set
d) Stuck at Freq Acq in progress
e) Stuck at Ranging in progress
f) Install Acq Denied HT #
g) Bootp Request Failed
h) Disabled Failed
i) Disabled Off Duty
j) Enabled Degraded
k) Matching RF Planning data
24 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Power on
Upon powering up the IDU the power LED should be a solid green color
Verify input power is within the operating range:– +/- 48vdc: +/- 40.5vdc to +/- 57vdc
For DC power does rectifier have enough current capability?
If the IDU does not power on, remove the IFL cable and try again. If it powers on, there is a short in the IFL cable.
25 Troubleshooting and Maintenance HUGHES PROPRIETARY II
SUDs (Start Up Diagnostics)
A failure during SUDs indicates one of the IDU self tests has failed.
If there is a failure, the “status LED” will remain off or a solid color and the LCT can not be accessed.
If this happens, replace the IDU (note when replacing the IDU make sure to contact the EMS operator and report the new MAC address).
26 Troubleshooting and Maintenance HUGHES PROPRIETARY II
IDU Hangs After Entering Cable Loss
This occurs while the IDU is in the “install mode”. The LCT is stuck after inputting IFL cable/loss information and does not show the RT Frequency selection.
Enter your choice from the menu Your Choice(1-14): 1
========================================================================== IFL Cable Mode========================================================================== 1. Cable Loss 2. Cable Type/Len
Enter your choice from the menu Current Choice: 1Your Choice(1-2): 1
Enter the 'Rx Loss (0.1 dB)' default = 47 :
Enter the 'Tx Loss (0.1 dB)' default = 67 :
IDU Hangs
27 Troubleshooting and Maintenance HUGHES PROPRIETARY II
IDU Hangs After Entering Cable Loss
This indicates the IDU was unable to poll the ODU to verify it is the correct band radio for the frequency selected. This may be due to:– an open connection between the IDU and ODU…verify the
integrity of the IFL cable, and verify the IFL connections at the IDU and ODU have been properly made;
– HT ODU was mistakenly installed at RT site… verify for the correct ODU part number
– Unsupported ODU – Check the part numbers– Faulty ODU (rare) – replace ODU
If after inputting IFL cable loss/type information output screen displays wrong frequency set ( expected, for example A1-A20, displayed B1-B20) this indicates that the wrong band RT ODU has been installed at the site. Verify for the correct RT ODU by part number.
28 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Stuck at “Frequency Acq in progress”
========================================================================== Select 26C Freq========================================================================== 1. C1 2. C2 3. C3 4. C4 5. C5 6. C6 7. C7 8. C8 9. C9
Enter your choice from the menu Your Choice(1-9): 1
---------------------------------------- Freq Set C1 ----------------------------------------
---------------------------------------- Freq Acq In Prog---------------------------------------- Stays here for a
long time > 15 minutes
29 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Stuck at “Frequency Acq in progress” This indicates the RT is unable to synchronize to the HT.
Verify there are no other RTs having a problem in this sector. If this is the first RT in the sector, verify the HT is operational.
Another possible cause may be that the RT boresight alignment was performed incorrectly and the RT is not pointed well enough receive on either side lobe, or the main lobe. Verify the RT pointing.
If it is a point to point link, verify the HT pointing. It is possible the IFL cable integrity is not proper and a
poor quality IF signal is getting to the IDU. Verify the IFL cable sweeps and/or IFL cable connectors ( loose, improperly terminated)
(continued...)
30 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Stuck at “Frequency Acq in progress”
... Verify that the proper RT antenna is used (30cm used
instead of 60cm). Verify that the RT ODU is not installed with the incorrect
polarization Verify that correct Polarity antenna is installed at the Hub
Terminal Another possible cause is the ODU or IDU is failed…
replace the ODU or IDU
31 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Stuck at “Ranging in progress”
========================================================================== Select 26C Freq========================================================================== 1. C1 2. C2 3. C3 4. C4 5. C5 6. C6 7. C7 8. C8 9. C9
Enter your choice from the menu Your Choice(1-9): 1
---------------------------------------- Freq Set C1 ----------------------------------------
---------------------------------------- Freq Acq In Prog----------------------------------------
---------------------------------------- Ranging In Prog----------------------------------------
Stays here for a long time > 15
minutes
32 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Stuck at “Ranging in progress”
Ranging is performed at the demodulator in the IDU. The RT is able to receive signal from the HT well enough to synchronize, but there are too many errors on the link to perform the ranging process. Verify no other RTs in the sector are having problems. If so, check the HT site.
Since at this point the pointing voltage port is enabled on the ODU verify the RT antenna pointing to improve the receive signal quality.
Verify in the RF Planning data if a mechanical attenuator is required at this site. If so, verify it has been installed.
Verify the IFL cable sweeps.
If the link is Point to Point– Verify that you exited install mode at the HT– Verify that the range is set correctly
33 Troubleshooting and Maintenance HUGHES PROPRIETARY II
“Install Acq Denied HT#”
Install acquisition denied indicates that the RT ESN was not known by the HT. Verify the HT ID corresponds to the expected serving HT, and if so, check with the EMS operator that the ESN was properly input into the EMS database.
Another possible cause may be that, if this is a RT site on the sector fringe, the HT ID denying acquisition is not the expected serving HT but may be another HT operating on the same frequency as the desired HT. If so, the HT antenna may not be pointed properly to give the correct coverage footprint.
34 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Bootp Request Failed If the RT has successfully been taken out of the install
mode yet there is a failure to receive a reply to the bootp request, then the air link quality or the IF quality is suspect.
Put the RT back into the installation mode and repoint the antenna.
Review the actual IFL cable sweeps paying particular attention to the sweep cable characteristics.
Check the SQI – there may be interference on the link Check the Interface option on the EMS If the antenna pointing and IFL appears correct replace
the IDU (note when replacing the IDU make sure to contact the EMS operator and report the new ESN).
35 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Disabled Off Duty
Disabled Off Duty indicates the IDU has not received all software from the EMS yet: it’s either in process of downloading it or it’s failed to complete FTP process with the EMS.
To find out if this is the case, logon to LCT and check “Status” – “Element Software”. If state is listed as “Unknown” or message says “Reconciling” and it stays that way for more than 10 minutes – most likely that FTP process has failed.
Check EMS configuration ( subnet mask), check router configuration.
Check HT/RT Air Interface status – FTP process can get interrupted due to poor air link quality.
36 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Disabled Failed
With the RT successfully downloaded from the EMS, the final state shows:– RT Freq XX– Disabled Failed
Use the “status option” on the LCT menu to identify the hardware failure. If ODU is failed…replace ODU. If IDU is failed…replace the IDU.
37 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Enabled Degraded
Enabled Degraded indicates the IDU is Enabled and transmitting, but one of the configured service ports does not have an interface cable installed or is not configured correctly.
38 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Matching RF Planning Data
It is important to compare the RF operational characteristics of the site to the site RF plan.
Compare actual values of RSSI and Tx power to the RF Plan The actual RSSI keeping in mind that the RSSI displayed initially after the
install may be based on only the QPSK burst (messaging burst between HT and RTs). Unless this site or another RT is configured for the same modulation and a service is also configured on the sector, the RSSI is based on the QPSK burst only (the LCT indicates if the measurement is taken from a QPSK burst).
When comparing RSSI and Tx power, it is important to remember the accuracy of the values shown on the LCT, or reported on the EMS is affected by inherent variables in the system.
These variables include IFL cable loss, HT Tx power, specific ODU RF characteristics, and environmental conditions at the time (RF Planning data is usually shown based on clear sky conditions). It is fair to assume if actual RSSI and RF Planning data are comparable within +/-5dB then things are ok. If Tx power is within a few dB then it is ok to continue.
39 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Link Balancing
It’s a good idea to see if a link is “balanced” Balance means that uplink path loss closely match
downlink path loss A mismatch indicates a problem most likely in cable loss
values, IFL cable or could indicate an ODU problem
Uplink path loss = abs ( RT TX pwr – RSSI @ HT )
Downlink path loss = abs ( HT Tx pwr – RSSI @ RT )
HT RT
RSSI from RT
HT TX Power
RT TX Power
RSSI from HT
Uplink
Downlink
40 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Enhanced LCT – Normal Mode
Provides the ability to debug RF issues from the HT LCT (without EMS access)
“LCT Local Configuration” must be enabled on the EMS
If enabled, the HT LCT can be used to adjust:– Frequency (Channel)
– Clock Source
– ODU QPSK Transmit Power
– ODU 16-QAM Transmit Power
– Target RSSI
When disabled at the EMS, all values return to the EMS Configuration values
41 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Operational Troubleshooting
42 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Troubleshooting – Bottom Up
CPENetwork
Network Managment
HT Hardware
Port(s)
RT Hardware
Port(s)
Air Interface
Services
ODU ODU
Always start looking at the problem from the bottom up.
For example, if a service has errors you may:
1. Check hardware first – IDU, ODU
2. Check physical layer – ports, air interface
3. Check services
43 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Troubleshooting General Steps
Look at the following in the order shown:
– Current Alarms• Can the site be reached from the EMS? • If not, is it one HT or all?
– Alarm Log (Log History)
– HT/RT Summary Status
– HT/RT Detail Status – Look at overall status to see a failure has occurred
– HT/RT Air Interface – Air Interface must be good before looking at anything else
– HT/RT Port Status – Look at interface state, alarm bits, duplex mode, etc as applicable
– HT/RT Port & Service statistics – Look at cell/frame counts, ingress & egress errors, etc.
– Use Loopbacks to isolate problems
44 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Current Alarms
45 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Current Alarms
The current alarm screen identifies real time faults in the system.
Alarm severities range from:– Warning - Blue colored alarm showing details of
potential problems– Minor – Yellow colored alarm showing details of
system alarms that do not affect services– Major – Orange colored alarm showing details of
system alarms that could affect, or do affect services– Critical – Red colored alarm indicating details of
alarms having a major impact on services.
46 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Alarm Types
Communication alarms – Result from errors on the internal or external communication interfaces of the AB9000 network.
Quality of Service alarms – Related to service connections.
Equipment alarms – Result from equipment hardware or software failures.
47 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Acknowledged and Cleared Alarms
Each operator should track open alarms daily to monitor the progress of technical personnel dispatched to clear site faults.
Once an alarm is being worked on it should be acknowledged.
Admin Note: With multiple operators having the same privileges, the acknowledged alarm log should be reviewed by the system administrator to verify that all cleared alarms had the appropriate attention provided and fault resolutions implemented.
48 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Topology View - Alarms
The Alarm Summary shows ALL alarms from
the selected node, and below
The nodes are colored according to the highest severity
alarm on that node AND its children
Quick Navigation to debugging
screens: Detail Status, Alarms,
Summary Status
Link States:
Green line: Normal connected state (Summary Status is ok)
Orange line: Partial NM connectivity
Red dash line: not connected
49 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Current Alarm Screen
The Fault Element is the element of subcomponent where the fault actually is
The Reporting Element is the one where the alarm actually
came from
Sometimes they are the same. In this example,
the EMS is reporting that if can’t contact an RT (check the air link!)
50 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Current Alarms Details Screen
Current alarm details lead the operator to the fault element. These details combined with the use of status, statistics, and loopbacks help for quick drill down and fast problem
resolution.
Direct access to:Configuration
Details Status & ControlSummary Status
51 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Alarms Log
All Alarms are recorded in the Alarm Log to keep a historical record of Alarms
All Acknowledge and Clears are recorded
Alarm Log can be filtered to provide, for example, history of a particular service or element
52 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Operational TroubleshootingHT and RT Summary Status
53 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Summary Status – Hub Terminal1. Select the Hub
2. Press Read
Operational and Availability States
show you the current status for the HT
Admin State is the state configured
on the HT Configuration
screen
This shows you if a redundant HT is
switched in or switch out
Here you can see the last time the EMS
received a “heartbeat” message
from the HT
54 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Summary Status - Remote Terminal
By highlighting a row and clicking on the button ( to the right ), operator can “jump” and review more detailed data
1. Select the Hub and Sector
2. Press Read
Operational and Availability States
show you the current status for the RT
This shows you if a redundant RT is
switched in or switch out
Here you can see the last time the EMS
received a “heartbeat” message
from the RT
If Adaptive Modulation is in use,
you can see the configured and
current modulations
55 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Operational TroubleshootingHT and RT Detail Status
56 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Ping, Trace Route
Access these dialog boxes from HT and RT Detail
The screens are very useful
to determine if an HT or RT
is reachable...
... If not, how far into the network can we
reach? Is it a router or network
issue?
57 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Hub Terminal Detail Status & Control
This is a switched in Backup HT.
Hub Terminal can be reset from the EMS
Hub Terminal can be switched to the
primary or backup HT
The main tab provides an overview of current status
of HT components
58 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Hub Terminal Detail Status – Air InterfaceRT Status shows
Modulation...
... and Uplink and Downlink RSSI and Tx Power
Entered Cable Loss values
Channel and bandwidth
ODU Type
59 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Hub Terminal Detail Status & Control
Current Alarm tab displays all alarms existing at the HT at
this moment. Probable cause and additional text would help
to resolve the issue.
60 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Remote Terminal Detail Status & Control
Remote Terminal can be reset from the EMS
The main tab provides an overview of current status
of RT components
61 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Remote Terminal Detail Status – Air Interface
Remote Terminal Modulation can be
switched (AM)
Primary & Backup HT signal Strength
Downlink RSSI and SQI
Tx Power
Current Modulation
62 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Operational TroubleshootingAir Interface
63 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Air Interface Troubleshooting
Use the HT Air Interface Statistics screen to look at UPLINK errors
Use RT Air Interface Statistics screen to look at DOWNLINK errors
Is the problem isolated to a single RT?– If yes, then concentrate on the RT side
Is the problem on some or all RTs in the sector?– If yes, then look at the HT side
64 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Air Link Statistics – HT (Uplink)Solicited Mode
= Real TimeCurrent Recording
Interval
This RT has pre-FEC uplink errors – OK?
Maybe...
...but it also has post-FEC errors -
BAD
Is SQI ok for the selected
Modulation?
65 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Air Link Statistics – RT (Downlink)Solicited Mode
= Real TimeCurrent Recording
Interval
This RT has pre-FEC downlink errors –
OK? Maybe...
...but it also has post-FEC errors -
BAD
Is SQI ok for the selected
Modulation?
66 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Air Interface Troubleshooting
Pre-FEC errors are ok as long as they are corrected– “Bit Error Count” shows pre-FEC errors
Is the link (UPLINK or DOWNLINK) experiencing post-FEC errors?– “Sync Errored Frames” and “FEC Errored Frames”
MUST be zero– Look at SQI (see next slide)– Look at Rx & Tx Levels – Do they match the planning
data?
67 Troubleshooting and Maintenance HUGHES PROPRIETARY II
What is SQI? SQI is Signal Quality Indicator The value of SQI is derived from the modem. It is a
measure of the received signal with respect to the “ideal” or perfect received signal.
Minimum acceptable SQI to get acceptable BER is about:– 12 dB for QPSK– 18 dB for 16QAM– 22 dB for 64-QAM
In clear sky, it’s possible to see SQI values at the RTs (downlink) in the mid 20's or more because the HT is transmitting at high power. – RTs close to the HT will see a stronger signal and will have
better SQI than RTs very far from the HT. In the uplink, the SQI values (on the HT) won't be as high
as the downlink SQIs because RTs only transmit at a level high enough to meet the configured target RSSI level. However, uplink SQI should still be close to or higher than the baseline numbers above.
68 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Operational TroubleshootingPorts and Services
69 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Port States & Status
Check the HT and RT Configuration screens to make sure ports that have services configured are in “Unlocked” state.
“Unlocked” means that alarms are reported to the EMS.
Remember, by default, all ports are created as “Locked for test” – they carry traffic but DO NOT report alarms
Two kinds of ports:– Physical : E1, DS3, STM-1– Virtual : IMA, HDLC, PPP
70 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Hub Terminal Detail Status – DS3 Port
STM-1 Port is similar. Look at Alarm bits on the interface.
Status of port: Configured vs. Operational
Is the port Unlocked?
Is the port failed?
Check Alarm bits
Problem on near end or far end?
71 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Remote Terminal Detail Status – E1 Port
Status of each port: Configured vs. Operational
Is the port failed?
Check E1 Alarm bits
Problem on near end or far end?
Loopback status
72 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Remote Terminal Detail Status – LAN Port
This port is Locked for Test – there will be no alarms reported to the
EMS
Does the speed and duplex match?
73 Troubleshooting and Maintenance HUGHES PROPRIETARY II
RT IMA Group Status
Don’t forget to check “virtual ports” like IMA groups
IMA Group State & Status
Individual E1 status
74 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Service States & Status
Check the Service Configuration screens (or service list screens) to make sure services are in “Unlocked” state.
Service states are on top of Port states
“Unlocked” means that alarms are reported to the EMS for that service
75 Troubleshooting and Maintenance HUGHES PROPRIETARY II
CES Service Detail Status
If no port number is given – than all circuits configured for the corresponding SSI
will be shown
HT Side of the service.
RT Side of the service.
76 Troubleshooting and Maintenance HUGHES PROPRIETARY II
LAN Service Detail StatusSelect HT side or RT side if
desired. Default is from both sides.
This service is “Unlocked” but Control Status
shows “Locked for Test” which means LAN port is “Locked
for Test”
Availability State is FAILED – LAN cable
unplugged? We should have already caught that before
coming here!
77 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Loopbacks
To help isolate a traffic problem it is possible to perform various loopbacks from the EMS. Note - Enabling any of the loopbacks will affect all services. Any changes made on the EMS database require a short time to download to the site. Loopbacks may require a maintenance window to perform.
Do not to use combinations of loopbacks simultaneously.
78 Troubleshooting and Maintenance HUGHES PROPRIETARY II
External Port Loopbacks
External port loopbacks are useful to verify physical connectivity
The administrative state of the port must be “Locked for test” to enable a loopback.
External port loopbacks can be applied to the HT or the RT.
Network HT RT
CPE
Traffic
Network HT RT
CPE
Traffic
RT Port External
Loop
HT Port External Loop
79 Troubleshooting and Maintenance HUGHES PROPRIETARY II
External Port Loopbacks – E1
If the port supports it, payload loopback can be used to isolate a problem
First start with External Line and then try External Payload
IDU
Tx
Rx
IDU
Framer
Rx
Tx
External line loopback loops the incoming data at the line
interface unit.
Set the BERT to internal clock.
External payload loopback loops the data at the framer
(Structured mode only).
Set the BERT to recovered clock.
80 Troubleshooting and Maintenance HUGHES PROPRIETARY II
External Port Loopbacks – E1 at HT
IDU
Tx
Rx
External Line Loopback – clocking is provided by the test equipment connected to that port.
81 Troubleshooting and Maintenance HUGHES PROPRIETARY II
External Port Loopbacks – E1 at HT
IDU
Framer
Rx
Tx
External Payload Loopback – clocking is provided by the system. Note that Payload
loopback is only available when the port is in Structured mode
82 Troubleshooting and Maintenance HUGHES PROPRIETARY II
External Port Loopbacks – STM1 at HT
IDU
Tx
Rx
IDU
Tx
Rx
Make sure that clocking settings are coordinated with the
test equipment settings.
83 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Internal Port Loopback
Internal loopback offers the ability to test the quality and integrity of a round trip service connection within the AB9000 system.
The loopback can be done at the HT or the RT. The administrative state of the service must be
“locked for test” to enable a loopback.
Network HT RT
CPE
Traffic
Network HT RT
CPE
Traffic
RT Port Internal Loop
HT Port Internal Loop
84 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Internal Port Loopback – E1 at RT
IDU
Tx
Rx
Internal line loopback at RT port is sending data back to HT.
Set the BERT to recovered clock.
85 Troubleshooting and Maintenance HUGHES PROPRIETARY II
OAM Test
Use this screen to perform Operations, Administration, and Management (OAM) testing on a selected ELCI to verify the integrity of the connection without affecting traffic on the circuit.
You can click to Start or Stop the OAM testingon the selected ELCI
86 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Service Troubleshooting
If the problem is related to a service, make sure that the configured services on the AIReach system match the connected equipment
For example:– Are the VP/VC’s the same (ATM)?– Are the bandwidths assigned the same (ATM, TDM,
LAN)?– Do MAC address or VLAN ID match (LAN)?– Do the port physical characteristics match?
• Framing, coding, etc (TDM)
• Scramble mode (ATM)
• Duplex setting (LAN)
87 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Is Site Visit Required? - Prerequisites
Questions to ask before going to site:
Can you ping HT or RT from EMS? Can you ping router at hub site from EMS? Can you ping the HT Air Interface IP address? Did I write down the IFL Cable loss values from HT
or RT Configuration screens? What type of unit is installed on the site?
– Do I have the correct spare IDU and ODU?– For example: 16 E1+STM-1+LAN + 26GHz B-Band
88 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Long Term Performance Monitoring
89 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Status and Performance Screens
Operators should baseline the site status and RF performance data of Hub and Remote sites during both clear sky and severe weather conditions.
They should perform a regular review of the Hub and Remote Air Link performance and check that RF performance is consistent.
Periodic monitoring of RSSI levels, Tx power, SQI, pre-FEC & post-FEC errors may help to identify problems as they develop, and before they become service affecting.
90 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Statistics
Operators should review HT and RT Air Link statistics and Service statistics for each site’s traffic bearing service type.
The historic log files can be reviewed by the operator or planner looking for unusual or intermittent unexplained RF and service anomalies that may have occurred during the past.
Statistics can be exported into Excel and plotted showing trends.
91 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Support Contact
92 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Contact Numbers Contacting TAC:
Phone: +1 301 601 7499
E-mail: [email protected]
Website https://customergateway.hns.com/
(Note: Access to customer‑specific and
product information is restricted by User ID and PIN.)
Mail: Hughes Network Systems, LLCTMD TAC 11717 Exploration LaneGermantown, MD 20876
93 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Hours of Operation Normal US business days - Monday though Friday,
8 a.m. to 5 p.m. EST.
– TAC engineers are on duty at the Germantown facility.
– In the event incoming calls cannot be answered by the on duty engineer during normal business hours, calls will automatically forward to an answering service, and the on duty engineer(s) will be paged.
US observed holidays, weekends, and normal business days after 5p.m. to 8a.m. EST.
– An automated attendant answers the TAC phone, and through menu selection, a customer can connect directly to the mobile phone of an on-call engineer.
94 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Hours of Operation (cont.)
– In the event the engineer’s mobile phone is not available, the caller can page the engineer through the answering service.
– During special customer situations outside of normal hours, such as, a planned major software load or system expansion, special assistance may be requested. To schedule this support contact the Director of the TAC at least one week in advance of the planned event.
– Every attempt is made to provide an adequate level of staffing during periods of inclement weather in Germantown, MD.
95 Troubleshooting and Maintenance HUGHES PROPRIETARY II
CRM Web Interface
Customer Resource Management (CRM) web-based interface
Customer portal for:– Opening trouble tickets (TAC Cases)– Viewing & Searching TAC Cases– Viewing documentation– Downloading software
96 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Login Page
Sign in
You must get a username and password from Hughes TAC
97 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Signed On
Site Navigation
Access Case history, Documents and other functions
here.
Click on Documents and Technical Info
98 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Documents By Category1. Click on Reports & Tech Info 2. Click on All
Documents with Folders
Here you will see a list of all document categories available
to you.
Click on a folder to open it and
navigate down
99 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – All Available Documents
1. Click on All Documents with Folders
Here you will see a complete list of
documents available.
You can enter a search term to quickly find a document.
This is a full text search so it
searches inside documents too.
100 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Report a New Case1. Click on BCN Case
Management
2. Click on Report a
Case
3. Select the Site Name. Hughes
typically does not track your site names, so you may see only
one entry here.
4. Fill in all of the required
information and provide as
much detail as possible
5. Attach screen
captures, database
dumps, logs, etc.
6. Submit the case
101 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Report a New Case
A new case is created and TAC is
notified.
Your case number is shown here.
102 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Search for Cases
1. Click on Case Search
2. Select a site. You can
select ALL SITES if you
are not sure of the site.
3. Other information is
optional
4. Click Search
103 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Search Results
Click on the case number to view the
case
104 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Case Overview
Case status is displayed here
You can add notes and
attachments to the case anytime
Hughes TAC updates are
shown here as well
Click here to show the case
description details
105 Troubleshooting and Maintenance HUGHES PROPRIETARY II
TAC Online – Case Details
Case description
details
106 Troubleshooting and Maintenance HUGHES PROPRIETARY II
Conclusion
From this course you should:– Understand the distinction between:
• Installation Troubleshooting• Operational Troubleshooting
– Know how to use the LCT to troubleshoot– Know how to use the EMS to troubleshoot
– Understand how to contact Hughes TAC for issues that you can’t resolve
Questions?