Download pdf - RF Troubleshooting Guide_EVDO(R27) (1).pdf

ALL RIGHTS RESERVED ALCATEL-LUCENT 2006 FOR INTERNAL USE ONLY USE PURSUANT TO COMPANY INSTRUCTIONS

1 X E V- D O R F P E R F O R M A N C E A N A LY S I S

& T R O U B L E S H O O T I N G

G U I D E

REL 27.0

RF TOOLS & SERVICES SYSTEMS ENGINEERING GROUP

TA B L E O F C O N T E N T S

0. CHANGE HISTORY.............................................................................................................................. 8

1. INTRODUCTION................................................................................................................................. 12

2. USING THIS GUIDE ........................................................................................................................... 14

3. OVERVIEW OF 1XEV-DO SYSTEM AND OPERATION............................................................. 16

3.1 SYSTEM FUNCTIONAL BLOCKS ..............................................................................................16 3.1.1 Access Network ........................................................................................................... 17 3.1.2 Access Terminal .......................................................................................................... 20

3.2 1XEV-DO SYSTEM OPERATION ...........................................................................................20 3.2.1 Forward link................................................................................................................ 21 3.2.2 Reverse Link................................................................................................................ 22

4. SUMMARY OF PERFORMANCE RELATED FEATURES IN PRODUCT RELEASES .......... 23

4.1 PERFORMANCE RELATED FEATURES IN R26.0...................................................................23 4.2 PERFORMANCE RELATED FEATURES IN R27.0...................................................................24 4.3 PERFORMANCE RELATED FEATURES SCHEDULED IN R28.0 ............................................26

5. KPI OPTIMIZATION AND TROUBLESHOOTING ...................................................................... 28

5.1 ESTABLISHED CONNECTION RATE ......................................................................................29 5.1.1 Call setup process ....................................................................................................... 30

5.1.1.1 Fast Connect Calls ........................................................................................................ 32 5.1.2 RF access failures ....................................................................................................... 33

5.1.2.1 Lack of RF coverage ..................................................................................................... 34 5.1.2.2 Excessive Pilots/Non-dominant Pilots .......................................................................... 35 5.1.2.3 Poorly constructed Neighbor Lists................................................................................ 36 5.1.2.4 Improper PN planning................................................................................................... 38 5.1.2.5 External Interference..................................................................................................... 39 5.1.2.6 Inadequate Search Window Size................................................................................... 41 5.1.2.7 Edge of coverage........................................................................................................... 42 5.1.2.8 RNC border issues ........................................................................................................ 43

5.1.3 Failures due to faulty hardware.................................................................................. 44 5.1.4 Heavy Reverse Link traffic .......................................................................................... 44 5.1.5 TP overload................................................................................................................. 46 5.1.6 Maximum number of connections reached.................................................................. 47 5.1.7 Traffic Channel Activation failure .............................................................................. 49 5.1.8 Authentication error.................................................................................................... 49 5.1.9 High Processor Occupancy rates................................................................................ 50

5.2 DROP CALL RATE ...................................................................................................................53 5.2.1 Lack of RF coverage ................................................................................................... 55 5.2.2 Excessive pilots/Non-dominant pilots ......................................................................... 56 5.2.3 Poorly constructed neighbor lists ............................................................................... 58 5.2.4 Improper PN planning ................................................................................................ 59

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

ALL RIGHTS RESERVED ALCATEL-LUCENT 2006 3

5.2.5 External interference................................................................................................... 61 5.2.6 Insufficient search window size................................................................................... 62 5.2.7 Failures due to faulty hardware.................................................................................. 63 5.2.8 Heavy reverse link traffic ............................................................................................ 65 5.2.9 Maximum number of legs reached .............................................................................. 67 5.2.10 No resources at the candidate leg............................................................................... 68 5.2.11 High Processor Occupancy rates................................................................................ 69 5.2.12 Handoff failures .......................................................................................................... 70 5.2.13 Reverse link lost due to hybrid tuneaways .................................................................. 71

5.3 DATA THROUGHPUT ...............................................................................................................73 5.3.1 Lack of RF coverage ................................................................................................... 74 5.3.2 Excessive pilots/Non-dominant pilots ......................................................................... 75 5.3.3 Poorly constructed neighbor lists ............................................................................... 77 5.3.4 Improper PN planning ................................................................................................ 79 5.3.5 External interference................................................................................................... 80 5.3.6 Insufficient search window size................................................................................... 82 5.3.7 Failures due to faulty hardware.................................................................................. 83 5.3.8 Heavy reverse link traffic ............................................................................................ 84 5.3.9 High forward link traffic ............................................................................................. 86 5.3.10 Hybrid mode................................................................................................................ 88 5.3.11 Backhaul restrictions .................................................................................................. 89 5.3.12 Handoff rejects ............................................................................................................ 90 5.3.13 Core IP network issues................................................................................................ 92

APPENDIX A METRIC CROSS-REFERENCE .............................................................................. 93

APPENDIX B OVERVIEW OF SPAT3G TOOL............................................................................. 96



L I S T O F F I G U R E S FIGURE 1 FUNCTIONAL BLOCKS IN A ALCATEL-LUCENT 1XEV-DO SYSTEM............................... 16 FIGURE 2 FUNCTIONAL BLOCKS IN A ALCATEL-LUCENT 1XEV-DO OVERLAY ON A ALCATEL-

LUCENT 3G1X HIGH SPEED DATA NETWORK ....................................................................... 17 FIGURE 3 PROTOCOL STACK DIAGRAM FOR ALCATEL-LUCENT 1XEV-DO SYSTEM [9] .............. 20 FIGURE 4 CALL FLOW FOR AN AT INITIATED CONNECTION .......................................................... 31

APPENDIX FIGURE 1 EXECUTIVE SUMMARY ................................................................................. 97 APPENDIX FIGURE 2 SM SUMMARY WINDOW AND PEG COUNT DETAILS .................................... 97 APPENDIX FIGURE 3 SM MAP ....................................................................................................... 98 APPENDIX FIGURE 4 SM TREND.................................................................................................... 99 APPENDIX FIGURE 5 ROP SUMMARY ALARMS ANALYSIS ....................................................... 100 APPENDIX FIGURE 7 TRX SUMMARY WINDOW .......................................................................... 101 APPENDIX FIGURE 8 NL ALERT SUMMARY ................................................................................ 102

L I S T O F TA B L E S TABLE A-1 ESTABLISHED CONNECTION RATE METRIC CROSS REFERENCE ...................... 93 TABLE A-2 DROP CALL RATE METRIC CROSS REFERENCE ................................................ 93 TABLE A-3 DATA THROUGHPUT METRIC CROSS REFERENCE ............................................ 94



List Of Abbreviations 2G Second Generation (IS95A/IS95B) 3G1x Third Generation 1.25MHz (IS2000) 3GPP2 Third Generation Partnership Project 2

A10 Interface that carries user traffic between PCF and PDSN A11 Interface that carries signaling between PCF and PDSN A12 Interface that carries signaling (authentication) between PCF and AAA A13 Interface that carries signaling between source and target PCF AAA Authentication, Authorization and Accounting ACK Acknowledgement AHO Access Handoff AN Access Network AP Application Processor APHO Access Probe Handoff AR Assistance Request AT Access Terminal

BTS Base Transceiver Station

CAMSHO Channel Assignment into Soft Handoff CBR CDMA Baseband Radio CC Control Channel CDM CDMA Digital Module CLGC Closed Loop Gain Control CRC CDMA Radio Controller CTU Common Timing Unit

DQOS Data Quality of Service DRC Data Rate Channel

EMS Element Management System EV-DO Evolution Data Optimized EV-DO Rev 0 Evolution Data Optimized Revision 0 EV-DO Rev A Evolution Data Optimized Revision A EVM EV-DO Modem

FCA Failed Connection Attempts FER Frame Error Rate FLM Forward Link Modem FMS Flexent Mobility Server FTP File Transfer Protocol

GPS Global Positioning System GRE Generic Routing Encapsulation

HCS HDR Cell Site



HDLC High Level Data Link Control HDR High Data Rate HOM Handoff Matrix HRPD High Rate Packet Data

ID Identification IM Inter Modulation IP Internet Protocol

KPI Key Performance Indicator

LCP Link Control Protocol LIU Line Interface Unit LWS Lucent Worldwide Services

MAC Medium Access Control MCC Main CDMA Controller MCR Multi Carrier Radio MIP Mobile Internet Protocol MR Modification Request

NAK/NACK Negative Acknowledgement NVM Non Volatile Memory

OA&M Operations, Administration & Maintenance OHM Overhead Channel Manager OMC-RAN Operations and Maintenance Center Radio Access Network OMP Operations and Management Platform OOS Out of Service

PCF Packet Control Function PDA Personal Digital Assistant PDSN Packet Data Serving Node PPP Point-to-Point Protocol PSK Phase Shift Keying PSMM Pilot Strength Measurement Message PTT Push-to-Talk

QAM Quadrature Amplitude Modulation QoS Quality of Service QPSK Quadrature Phase Shift Keying

RAB Reverse Activity Bit RATI Random Access Terminal Identifier RF Radio Frequency RLM Reverse Link Modem RLP Radio Link Protocol RNC Radio Network Controller ROP Read Only Printer



RSSI Receive Signal Strength Indicator

SBEVM Single Board EV-DO Modem SCI Slot Cycle Index SM Service Measurements SNR Signal to Noise Ratio SNMP Simple Network Management Protocol SIP Simple Internet Protocol SPAT3G System Performance Analysis Tool 3G SU Software Update

TCA Traffic Channel Assignment TCC Traffic Channel Complete TCP Transmission Control Protocol TDU Time Delay Unit TP Traffic Processor

UATI Unicast Access Terminal Identifier UCR UMTS/CDMA Radio UDP User Datagram Protocol UNL Undeclared Neighbor List URC Universal Radio Controller

VoIP Voice over Internet Protocol



0. CHANGE HISTORY

Version Changes Date R21 Initial Release October 2003

R22 Updates reflecting new service measurement counts and metrics, reference web sites, metrics cross-reference table and SPAT3G information for Release 22.

October 2004

R23 Updates reflecting new service measurement counts and metrics for Release 23. Updates include:

Updated the guide with information about the new reverse link overload control translations and the corresponding peg counts

Introduced new chapter summarizing performance related features in R23 and R24.

Added new information to the hybrid mode section about SCI and MSM6500 chipset performance.

Added a new subsection on high processor occupancy rates Added a link for EV-DO release letters Updated appendix B with information about EV-DO specific features

in SPAT3G Updated acronyms list, references and the metrics cross-reference table

February 2005

R24 Updates reflecting new service measurements counts and metrics for Release 24.

Added information about Celnet Xplorer in the introduction section Updated section 4 with performance related features in R24 and 25 Removed negotiation failures from the Established Connection Rate

equation Modified equation for Fast Connect Failure rate Added section 5.1.2.7 discussing connection failure rate issues in edge-

of-coverage areas Added section 5.1.2.8 discussing connection failure rate issues in inter-

RNC border areas Updated section 5.1.6 with details from Alcatel-Lucent Alert 05-0493

(peg count decrementing issue) Updated equation for forward throughput in section 5.3 Added section 5.3.13 discussing core IP network issues affecting data

throughput Updated appendix B (SPAT3G overview) with screenshots for ROP

call processing failure analysis Updated table of contents, references and acronyms list Updated metrics cross reference list

August 2005

R25 Updates reflecting new service measurements counts and metrics for Release 25 Added information about Handoff Matrix, Undeclared Neighbor list

and Multiple carriers in the introduction section Updated section 4 with performance related features in releases 25 and

March 2006



26 Updated the equation for the Established Connection Rate Updated the metric name Established Call Rate to Established

Connection Rate Added information about 1xEV-DO cell processor overload counts Updated appendix B (SPAT3G overview) with screenshots for Handoff

Matrix and Undeclared Neighbor List analysis Updated table of contents, references and acronyms list Updated metrics cross reference list

R26 Updates reflecting new service measurements counts and metrics for Release 26 Updated section 4 with performance related features in releases 26 and

27 Added information about reverse link lost due to tuneaways Added information about new SM counts for critical system edge

metric preservation

September 2006

R27 Updates reflecting new service measurements counts and metrics for Release 27 Added information on service measurements data for all six sectors by

default Changed Lucent entries to Alcatel-Lucent where relevant Updated section 4 with performance related features in product releases

27 and 28 Added information about LUNAR availability for service providers Updated acronyms list and metrics cross-reference table Added information on SM changes in R27 SU1 and Soft/Softer handoff

peg count issues Added information on PCMD analysis is SPAT3G in Appendix B

December 2006



References [1] 1xEV-DO RF Translation Application Note # 0 Introduction

[2] 1xEV-DO RF Translation Application Note # 1 Timing, Delay and Access Parameters

[3] 1xEV-DO RF Translation Application Note # 2 Forward link

[4] 1xEV-DO RF Translation Application Note # 3a Reverse Power control

[5] 1xEV-DO RF Translation Application Note # 3b Reverse Overload control

[6] 1xEV-DO RF Translation Application Note # 4 Handoff

[7] 1xEV-DO RF Translation Application Note # 5 High level Protocol Stack Parameter Optimization

[8] 1xEV-DO RF Optimization Guidelines

[9] SRD 1853 1x EV-DO Packet Data Call Processing

[10] SRD 1880 OA&M Requirements for the Network (MSC) portion of a 1xEV-DO System

[11] 1XEV-DO and 3G1X Service Measurement Counts Cross Reference Memo 12/07/02 ( http://nwswww.wh.Alcatel-Lucent.com/fjiang/1XEVSMcrossRef.doc )

[12] 1xEV-DO Troubleshooting Guide ( http://nwswww.ih.Alcatel-Lucent.com/apxlib/cgi-bin/srchdoc.cgi?entry=d26453+ )

[13] 1xEV-DO Performance Expectations document

[14] White Paper on Base Station and Network Configuration for 1xEV-DO July 30, 2003 (http://nwswww.wh.Alcatel-Lucent.com/yyang7/hdr/do_wp_2.3.pdf)

[15] CDMA RF Performance Analysis and Troubleshooting Guide

[16] 1xEV-DO RNC, AP Operations, Administration and Maintenance (Doc # 401-614-102)

[17] 1xEV-DO RF Translation Application Note # 6 Flexible Scheduler

[18] Alcatel-Lucent 1xEV-DO System Architecture, http://nwswww.wh.Alcatel-Lucent.com/gaby/do-work.htm

[19] 1xEV-DO Technology Overview (Physical Layer and Signaling) http://nwswww.wh.Alcatel-Lucent.com/gaby/do-work.htm

[20] Call processing exception reporting for 1x EV-DO: OHM, SFM and PCFA11 CP failure error list document (d32365)

[21] 1xEV-DO Service Measurements Manual (Doc # 401-614-326)



Useful Web Pages 1xEV-DO Current Engg http://nwswww.wh.Alcatel-Lucent.com/hdrce/ 1xEV-DO FOA http://nwswww.wh.Alcatel-Lucent.com/1xevfoa/index.html 1xEV-DO Perf Mgmt team http://nwswww.wh.Alcatel-Lucent.com/sobers/DO_Perf_CFT/ 1xEV-DO Release Letters http://nwswww.wh.Alcatel-Lucent.com/hdrce/ APX Library http://nwswww.ih.Alcatel-Lucent.com/apxlib/ Ask Alcatel-Lucent http://askAlcatel-Lucent.web.Alcatel-Lucent.com/ Cares Tickets (ARs) http://cares.web.Alcatel-Lucent.com/ External Technical Support https://wireless.support.Alcatel-Lucent.com/amps/ FID/FDD Lookup http://wngpjm.web.Alcatel-Lucent.com/cpdr/ LWS Knowledge Store http://servility.ih.Alcatel-Lucent.com/msstpi/ Alcatel-Lucent Alerts http://mss.web.Alcatel-Lucent.com/bulflash/bulflash.html Alcatel-Lucent Navigator http://navigator.web.Alcatel-Lucent.com/ Alcatel-Lucent Product Documents http://www.cic.Alcatel-Lucent.com/cicmulti.html SRD website http://nwswww.wh.Alcatel-Lucent.com/seamps/livelink/ Wireless Service Creation http://rfcoresupport.wh.Alcatel-Lucent.com/ & Performance Department Wireless University https://training.Alcatel-Lucent.com/SabaWeb



1. INTRODUCTION

This document has been compiled to serve as a guide for understanding concepts and techniques necessary to analyze, troubleshoot and monitor performance in a commercial 1xEV-DO network. The primary intent is to present information on ways to assess current levels of service quality offered by the system as well as ways to augment it by identifying and resolving specific performance issues.

There are two main ways to approach the task of performance troubleshooting and monitoring in any network. The approach often depends on the phase of network operation. The two approaches/phases are described below.

For a new network deployment, single-user based drive test techniques are often employed effectively to optimize performance before turning over the network for commercial operation. It involves characterizing RF performance under controlled conditions with use of several test mobiles. There is little real-user traffic to base system level analysis on.

Once in commercial mode, several parameters influence performance over time such as increase in subscriber usage, shift in traffic pattern, cell additions, carrier growth, hardware problems, environment changes, etc. that make the ongoing performance management critical. In this phase, network performance can be gauged via the use of Service Measurements (SM) and system level tools that derive measurements off the abundant commercial traffic. Controlled single user tests may be needed to supplement system level analysis, mainly in localized areas to fine tune performance.

With the above background in mind, it is important to clarify the approach adopted in this guide. It mainly addresses a 1xEV-DO commercial market with the number of cells and traffic large enough to offer a statistically meaningful analysis based on system level indicators.

The document follows a top-down approach in that it looks at system level performance and problem resolution techniques via Key Performance Indicators (KPI) based on available service measurements for 1xEV-DO1. For each KPI, it supplements the system wide performance management by selective drive test based optimization techniques.

In terms of the scope of the document, another point to note is that this guide primarily addresses performance associated with the sub-system involving Access Terminal (AT), Base Transceiver Station (BTS) and Flexent Mobility Server (FMS) components (explained in Section 3). There is a separate document for end-to-end network troubleshooting using single user call traces and associated diagnostics in more detail [12].

1xEV-DO has been deployed extensively by large service providers like Verizon Wireless and Sprint PCS with additional deployments expected by China Unicom and Reliance. With each 1xEV-DO product release, additional service measurements peg counts as well as new performance data sources are introduced enhancing the capability to use live traffic measurements

1 Service Measurement counts available through R27.0



to troubleshoot and improve 1xEV-DO network performance. For example, with R25, Handoff Matrix and Undeclared Neighbor List data is available to optimize neighbor lists.

This guide is written by applying general concepts of performance management from 2G/3G1x systems due to many similarities with 2G/3G1x CDMA systems. As the knowledge base grows from experience in commercial 1xEV-DO networks, additional troubleshooting techniques/principles will be incorporated.

One of the key assumptions made in this guide is that most 1xEV-DO systems will be single carrier. At this point, no multi-carrier deployments are on the immediate horizon, and this simplifies many of the performance considerations typically associated with multi-carrier systems. However, with R25 multiple carriers are supported for 1xEV-DO.

There are several tools available to retrieve and analyze system level performance data. One such tool is Vallent Corporations Prospect. It primarily allows creating reports (Microsoft Excel output) based on Service Measurements data. Another tool that allows a more integrated analysis of not just SM data, but also other system level information, such as ROP alarms and call processing failures, Translations/Configuration summary, Neighbor List analysis and Handoff Matrix/Undeclared Neighbor List analysis, is the Alcatel-Lucent internal tool, SPAT3G (System Performance Analysis Tool 3G).

Celnet Xplorer is a recently developed tool that was used during performance optimization of Verizon Wireless markets. Celnet Xplorer collects data for each data flow and contains detailed information about RF conditions, mobile type and call status. .



2. USING THIS GUIDE

The document is intended for Alcatel-Lucent internal use. Typical users will be comprised of LWS personnel specializing in RF performance and troubleshooting as well as Customer Technical Advocates.

To get the most from the guide, familiarity with 2G/3G1x performance management will prove helpful, but not required.

Before listing different ways this guide can be employed, it is important to highlight the fundamental questions the guide addresses related to system performance management. These are:

KPI/Metrics: How to define/characterize service quality offered to the end user?

Factors: What are some of the factors that can influence these metrics?

Pointers: How does one detect/isolate the performance impacting factor/root cause?

Resolution: How to solve/mitigate the performance impact?

This type of information can be applied effectively in several ways, mainly:

Audit system level performance

It is important to assess network performance on a continual basis. Long-term trending of system level performance indicators helps identify issues related to system bottlenecks, malfunctioning hardware, possible translation entry errors, etc as they develop. The information in the guide will provide useful pointers to aid in the effort.

Note that before embarking on KPI studies, it is important to thoroughly scrub translations per recommended values (Refer to [1]-[7], [17]). Over time, recommendations may change, or translation values may get altered mistakenly (such as fine-tuning some other parameter), or not kept up to date with configuration changes (such as when new resources are provisioned). Cleaning up translation settings often turns out to be a low hanging fruit to pick when it comes to improving network performance.

Targeting root cause for a specific problem request

This may be in response to a trouble ticket opened by the customer, citing degradation in a particular service quality indicator. The goal here is to try to focus on the root cause rather than performing a system level audit of all metrics.

The basic assumption in the document is that most performance issues can be funneled down to a few key manifestations. It is left to the user how to categorize the reported anomaly into one of these few KPIs.

After deducing the impacted KPI, the underlying component or failure cause can be identified using the repository of information offered in this document. It often requires eliminating other



causes in order to narrow down to the real one; therefore, it is important to carefully consider all underlying components. For example, reverse overload may be caused by external interference or 1xEV-DO user load and each has a different resolution strategy.

General Reference

An RF engineer new to the performance management area can learn a great deal on steps to maintain and troubleshoot network performance using the principles outlined in the document. The person may or may not have any experience in the area on prior technologies.

KPIs presented in the document are a good way to start delving into the basics of performance monitoring. The information on failure components can also lead to better understanding of how to apply/make use of secondary performance metrics. This version of the CDMA 1X EV-DO RF Performance Analysis & Troubleshooting Guide only reflects troubleshooting for Lucent 1X EV-DO equipment. While some references of the document have been updated to reflect the newly created Alcatel- Lucent, most of the references to the hardware and software are still to Lucent or Lucent Technologies hardware or software. These references will be updated in future versions of this document as their nomenclature is updated.



3. OVERVIEW OF 1XEV-DO SYSTEM AND OPERATION

Before delving into the core subject of this document, it is worthwhile to present a basic view of the 1xEV-DO system in terms of architecture and operational principles. These are discussed in the subsequent sections.

3.1 System Functional Blocks

Figure 1 illustrates functional blocks in a standalone Alcatel-Lucent 1xEV-DO system. Figure 2 shows the same for an Alcatel-Lucent 1xEV-DO overlay on an Alcatel-Lucent 3G1x network; color codes differentiate the shared from unique components in each system.

1xEV AN

1xEV Element Management

System

FIGURE 1 FUNCTIONAL BLOCKS IN A ALCATEL-LUCENT 1XEV-DO SYSTEM

A 1xEV-DO system can generically be categorized into 2 macro segments. These are:

AN Access Network is the equipment providing data connectivity between a packet switched data network (typically the Internet) and the access terminals.

AT Access Terminal is the device providing data connectivity to a user. An access terminal may be connected to a computing device such as a laptop personal computer or it may be a self-contained data device such as a personal digital assistant.

Next, we present some of the components in each segment of the network.



1xEV EMS

FIGURE 2 FUNCTIONAL BLOCKS IN A ALCATEL-LUCENT 1XEV-DO OVERLAY ON A ALCATEL-LUCENT 3G1X HIGH SPEED DATA NETWORK

3.1.1 Access Network

PDSN Packet Data Serving Node is one of the key components in a wireless Data network. In the downlink direction (Internet to AT), it converts IP packets received from the Internet into PPP stream and passes on to the 1x EV Controller. In the uplink direction, it terminates the PPP layer, structures PPP packets coming from PCF into IP packets, and directs them away from the DO network to the Internet. AAA server AAA stands for Authorization, Authentication and Accounting. The AAA server stores the database that contains service and protocol information for each user registered in the 1xEV-DO network. This information is returned to the PDSN using a secure protocol when the user first establishes a session in the network. The PDSN uses this information for authentication. AAA also administers the billing function by storing accounting information obtained from the PDSN.

1xEV-DO Flexent Mobility System (FMS) It serves as the heart of Alcatel-Lucent 1xEV-DO call processing and management. Its main hardware components include a bank of servers, known as APs and TPs, described below.

AP Each FMS frame can contain up to 8 Application Processors (AP). These are actually 4 pairs of APs - both APs in a pair run in full active/backup mode, meaning either one can take control if the other goes down. AP server is based on the Solaris operating system. Each AP has 2 Traffic Processors (TP) and several cells to manage.



TP TP runs off VxWorks operating system. A Data session served by a FMS has an associated TP. TP handles the direct traffic processing for the data flow between the cell sites and PDSN.

Each FMS contains 2 Cajun switches for traffic exchange between the APs, TPs and the router (described later).

An Alcatel-Lucent 1xEV-DO FMS can also be classified into software functional blocks. The specific functions are mainly implemented on the AP and the TP:

1xEV-DO Controller It is responsible for call management and RLP (Radio Link Protocol) control functions in the AP and TP. During setup of a Data session, AP assigns and binds a TP for rest of the session until and unless an idle (dormant) mode inter-PCF transfer occurs. Among several functions, TP is responsible for physical layer channel assignment and managing virtual handoffs between cells. From Data protocol stack perspective, it receives GRE (Generic Routing Encapsulation) packets from PCF, sequences GRE stream into RLP packets to pass on to the serving cell in the forward link. In the other direction, it performs frame-selection to choose the best received physical layer frame from various cells in the Active set of the call, terminates RLP packets and performs GRE wrapping to transport PPP packets to the associated PCF for the session. Another important function it handles is the RLP recovery mechanism. On the reverse link, it is responsible for verifying the integrity of RLP frames received over the physical layer, and requesting re-transmission of RLP frames received in error via NAKs. On the forward link, it buffers and re-transmits RLP packets in response to RLP NAKs received from the AT.

PCF The term Packet Control Function (PCF) refers to the general PCF functionality, which includes both the A10-PCF and the A11-PCF functionality. The A10-PCF is responsible for the data traffic (A10) connection and resides physically on the TP (Traffic Processor). The A11-PCF handles the signaling (A11) interface and resides on the AP within the FMS hardware architecture. The primary function of the PCF is to buffer and transport PPP packets between the PDSN and FMS using GRE (Generic Routing Encapsulation) protocol. As it is evident from the protocol structure in Figure 3, the GRE encapsulated PPP packets are carried over an Ethernet connection (which could consist of one or more routers) using IP datagrams; this is a local IP network separate from the end-to-end IP packet delivery mechanism between PDSN and AT. Also, note that the interface between PCF and PDSN, is also often called A10/A11 connection, per IS2001. The PCF is also responsible for generating accounting information needed by PDSN for each call.

Router The router performs an important function of distributing traffic from the controller to the destination cell(s) and aggregating uplink traffic from the cell to the Controller. In addition, it also routes traffic between PCF and the PDSN.

1xEV-DO Element Management System (EMS) The EMS interfaces with FMS to provide OA&M facilities for 1xEV-DO cells and the FMS. It resides on the standard Alcatel-Lucent OMP server platform. Its OA&M tasks include configuration management on 1xEV-DO cells and FMS



translation change and verify via the EMS GUI2 (Graphical User Interface), software installation, reboot/restart/status, network provisioning; fault management alarm monitoring, fault detection and recovery, diagnostics; and, performance management service measurements gathering and storage.

1xEV-DO Base Transceiver station (BTS) It is the cell site servicing over-the-air physical connections with the AT to transport RLP frames in either direction. It has an EVM (1xEV-DO Modem) hardware unit (similar to Channel Card Unit or CCU in 2G/3G1x BTS), which consists of Forward Link and Reverse Link Modems also known as FLMs and RLMs to transmit and receive physical/MAC layer frames in the two directions. Another important circuit pack is CDMA Radio Controller (CRC). It has two functional subunits one is Line Interface Unit (LIU) whose primary function is to terminate the HDLC protocol over T1/E1 backhaul, and the other is Main Cell Controller (MCC) which is responsible for BTS wide OA&M control, such as, alarm management, interface to other circuit packets, NVM updates and diagnostic testing. CBR (CDMA Baseband Radio) is another circuit pack responsible for digital and RF signal inter-conversion.

On the forward link, at any given instant, AT is tuned to traffic channel from any one cell. On the reverse link, there can be multiple cells in the Active set that are actively receiving, decoding and passing on traffic channel frame contents to the Controller. In addition, cell transmits certain Control Channels to assist AT with the idle mode operation and call set-up process. It includes broadcast messages (similar to overhead messages such as System and Access Parameters messages in 3G1x) and mobile specific messages (such as page, traffic channel assignment, etc.). Another key element is its assignment of MAC Indices. MAC Indices are similar to Walsh Codes. There are a total of 64 MAC Indices, one reserved for each of the overhead channels and the rest assigned to active calls. MAC Indices are important to distinguish signaling messages as well as reverse power control bits sent on the forward link for individual calls. Finally, BTS is also responsible for managing the inner and outer closed loops of reverse power control with the AT.

2 EMS GUI is the interface in the EMS that allows entering / modifying translation parameter values for various

components in the BTS and the FMS.



PCF PDSN 1xEV BTS

1xEV AT

Call Control

Airlink T1/E1

IP IP

802.3 T1 T1

HDLC HDLC

GRE GRE PPP PPP

IP IP

TCP UDP

TCP UDP

Fixed End System

RLP RLP

Internet

1xEV Controller

100 Base T

IP IP

802.3 802.3

Router

IP IP

A10 Interface

AAA

Frame Selector

RLP

UDP IP

UDP IP

PHY PHY 802.3 MAC MAC

TCP

Ethernet

802.3

TCP

FIGURE 3 PROTOCOL STACK DIAGRAM FOR ALCATEL-LUCENT 1XEV-DO SYSTEM [9]

3.1.2 Access Terminal

Also known as the mobile, it can operate in one of 2 different modes, based on the layers of the protocol stack it manages.

Connected to laptop: In this split function mode, AT terminates physical, MAC and RLP layers in the Protocol stack. All higher layers (PPP, TCP/UDP, IP, Application, etc) are terminated in the laptop itself.

Self-connected unit (such as PDA): In contrast to the above mode, AT terminates all the layers in itself. All the functions mentioned on the AN side have peers in the AT (e.g., physical layer channel establishment, reverse power control, RLP recovery, sequencing/de-sequencing between RLP packets and PPP stream, protocol conversion between PPP and IP, as well as between TCP/UDP and IP, etc).

The analysis and troubleshooting discussions in the remainder of the document mainly involve AT, BTS and 1xEV Controller components of the network.

3.2 1xEV-DO System Operation

Unlike 3G1x technology (defined by IS2000 standards), which can support both Voice and Data calls in a standard 1.25Mhz carrier, 1xEV-DO is a Packet Data only technology. It is defined by IS856 standards. The 1xEV-DO air interface, mobile and network components have been



carefully designed to offer much higher data rates/throughputs to the end-user on the forward link compared to 3G1x systems. The reverse link is largely similar to that of 3G1x.

3.2.1 Forward link

The 1xEV-DO forward link is the key differentiator that sets the technology apart from 3G1x. It offers peak rates of 2.4 Mbps compared to 153.6 kbps in 3G1x. The various rates include 38.4, 76.8, 153.6, 307.2, 614.4, 921.6, 1228.6, 1843.2 and 2457.2 kbps. The forward link is time multiplexed; there is no concept of Walsh code sharing among logical channels/users. There is also no concept of separate fundamental and supplemental traffic channels per user. Each user is allotted entire sector-carrier power at any instant without any power control for the transmission at a given rate.

The forward link transmission is in terms of slots; each slot is 1.66ms. The Pilot channel structure also differs considerably compared to 3G1x systems. All cells, for a small portion within every slot, send the Pilot channel synchronously. The mobile computes received Ec/Io (or Signal to Noise Ratio) for each Pilot during this synchronous interval. This SNR measurement is one of the key parameters used by the mobile to select and request a specific rate. Unlike 3G1x systems, the rate decision process is in the mobile and not in the cell. This allows for quicker rate adaptation to changing RF conditions, quicker transition to a stronger sector to receive forward link data, and minimal signaling overheads, compared to 3G1x systems. The mobile selects a specific rate to meet a target packet error rate (hard-coded to 1% in current mobiles). It is important to note that the transmitted cell site power to the user stays constant, so the mobile has to adjust rates to maintain the QoS (in 3G1x systems, cell has to adjust rate and transmit power to maintain QoS). 1xEV-DO uses a more sophisticated modulation technique to support high-speed data rates 16-QAM or 8-PSK modulation (lower rates use conventional QPSK). The other distinguishing 1xEV-DO attribute is the use of Incremental Redundancy on forward link. Basically, for multiple slot transmissions (except the higher couple rates, all other rates span over multiple slots), the cell sends data first and then redundancy bits in an incremental manner only if the mobile is having trouble decoding the contents. This results in high utilization of the physical layer.

On the forward link, only one sector is transmitting all the data to a given user. As mentioned above, it is the mobile that requests cell to transmit on a specific rate. While in an active call, the mobile constantly sends these requests via DRC (Data Rate Channel) bits on the reverse link. It addresses these DRCs to a specific cell from which it desires transmission. Such a cell is called DRC pointed cell. When another pilot in its Active set becomes stronger, the mobile starts pointing its DRC to this new cell. This process of switching active sector transmission is called Virtual Soft handoff. It is similar to the anchor transfer of forward link supplemental channel performed by the cell site in Alcatel-Lucent 3G1x systems; in case of 1xEV-DO, the process is managed by the mobile and inherently faster.

The initial Alcatel-Lucent 1xEV-DO system uses a proportionally fair scheduler, which attempts to maximize the aggregate sector throughput while managing delay (data starvation) experienced by the disadvantaged users. Cell evaluates DRCs requests from all mobiles and chooses the one with the highest rate. Due to short-term RF variations, different users experience SNR peaks at different times; the one experiencing strong signal gets the data - this is the proportional part where user served and hence, the assigned rate is based on signal strength.



Considering long-term averages, everybody gets roughly equal transmission time (number of slots) this is the fair part.

3.2.2 Reverse Link

As mentioned earlier, there are many similarities in the reverse link between 1xEV-DO and 3G1x systems. The supported rates are 9.6, 19.2, 38.4, 76.8 and 153.6 kbps. There is also a Pilot channel transmitted by each mobile on the reverse channel to aid coherent demodulation at the cell site. The soft/softer handoff process and the Active set maintenance procedures are similar (Tadd, Tdrop, Tcomp, TTdrop concepts of IS2000). The mobile can enter into a maximum of 6-way soft handoff, depending on translation settings. The mobile transmission is power controlled by the cell site to maintain a 1% reverse frame error rate. Unlike on the forward link where the target error rate is hard coded at the mobile, the one on the reverse link can be controlled by a translation called Reverse Target Frame Error Rate on the Sector EMS GUI page.

There are some differences, however. The physical layer frames are 26.66ms in duration (20ms frames used for 3G1x). The reverse link carries DRC bits to report forward link requested rates. There is no concept of separate fundamental and supplemental channels. There is only one logical traffic channel per mobile the mobile dynamically changes rate based on available data backlog and a set of rate transition probabilities. The mobile starts with low transmission rate and ramps it up based on these probabilities. It includes Reverse Rate Indication information on each frame to indicate the corresponding transmission rate.

As mentioned above, each mobile decides the transmission rate on its own. To control the reverse link integrity and prevent RF overload, cell has two options. One is to transmit a Rate Limit message on the forward link control channel limiting the maximum rate used by the mobiles. The other mechanism is to set the Reverse Activity Bit (RAB) on. RAB is sent on the Medium Access Control channel the MAC channel contains RAB, reverse power control commands and DRC lock bit for Active calls on the sector. When the RAB bit is on, some of the users can probabilistically halve their rate on the next frame. The percentage of such users is specified in translations. If the bit is off, the users can double their transmission rate unless they are already at maximum rate (153.6kbps).



4. SUMMARY OF PERFORMANCE RELATED FEATURES IN PRODUCT RELEASES

This section summarizes the key performance related features that are part of the current release (R27.0) and the previous release (R26.0). Features that are scheduled for the next release (R28.0) are also discussed to provide the user with information about future performance impacting features in the product. For a comprehensive list of features available in each release, the user is encouraged to refer to the release letters.

4.1 Performance related features in R26.0

The following performance related features are available in Cell/RNC release 26.0.

FID-8219.11 Support for Multiple 1xEV-DO Carriers IFHO: This feature supports IFHO for the 1xEV-DO multiple carrier feature. IFHO was originally defined to be part of 8219.1 to handle situation where there is carrier frequency discontinuity across the cells.

FID-10519.1 HRPD RNC Performance and Capacity Improvements: The current RNC supports 130 3-sector carriers. This feature will enable the FMS RNC to support 260 3-sector carriers to through several software improvements. These improvements will be applicable for both FMS and FBP platforms. The software improvements will be made for the Traffic Processor and Applications Processor (both OA&M and Call Processing) subsystems.

FID-12402.0 Short interval service measurements for 1xEV-DO Rev 0: This feature will provide a pre-determined set of 1xEV-DO service measurements in intervals of 15 minutes and will be stored in OMP in pre-designated files in the same format as the 60-minute interval service measurements. There will be separate count groups for shorter interval reporting. The 1xEV-DO RNC software will determine which count groups are reported in which file. The counts reported for shorter intervals will not be part of 60-minute interval SM count group. These measurements include the SMs that cannot be reported via the 1xEV-DO Per Call Measurements Data (PCMD). The initial set of SMs that will be include for the shorter interval reporting are: Cajun Switch performance statistics such as packets transmitted, received, dropped etc., Average and peak AP processors occupancy, Average and peak users per AP and Average and peak T1 occupancy v. Processors overload count and duration.

FID-10607.3 1xEV-DO AP Processor Overload Control: This feature will include overload control actions when the AP Processors go into overload. The AP supports Call Processing and OA&M subsystem functionality. The overload detection mechanism is Processor Occupancy based. It will identify when the time critical AP Processor Occupancy (%) crosses a certain threshold (set in translations). The overload control mechanism is intended to control the overload for the entire AP. FID-10968.0 1xEV-DO Product Enhancements to address Hybrid Mode scenarios: 1xEV-DO/3G-1X hybrid mode ATs introduce new Call Processing scenarios for the 1xEV-DO product that were not previously



envisioned. This feature addresses these scenarios and introduces a new service measurement and four new ROP messages to account for hybrid AT tuneaway from 1x EV-DO to 3G-1X.

FID-54321.0 1xEV-DO Critical System Edge Metric Preservation: This feature provides the short-term solution for preserving the usefulness of service measurements in poor RF coverage areas by allowing the service provider to optionally deny connection setup requests from ATs in those areas. A long-term solution is under discussion for a future release. At the edge of the Alcatel-Lucent Technologies 1xEV-DO system, many ATs are autonomously attempting to start up new sessions in extremely poor RF conditions outside of the nominal service area. These attempts use up Reverse Link resources often taking them away from other ATs that have a better chance of establishing a data connection. Battery life of the ATs in question is also most often needlessly diminished. The number of such requests is sufficient to skew Service Measurements so that they no longer provide an accurate account of the service in the valid coverage area.

FID-12269.0 PCMD for 1xEV-DO: This feature will collect Per Call Measurement Data (PCMD) on CDMA 1xEV-DO calls. It will be a similar but separate feature from the existing CDN-based PCMD feature in that it will collect additional information about 1xEV-DO calls including 1xEV-DO RNC resources.

FID-10898.0 1xEV-DO Neighbor sector Form enhancements: With this feature, the Neighbor Sector page of the EMS Configuration Management (CM) GUI screens will provide the user the ability to automatically populate some Neighbor Sector attributes. The values for these attributes are retrieved by EMS from other CM tables in the database. Only creation of a Neighbor Sector instance is affected by this feature. The feature is intended to act as a job-aid in pre-populating the data already exists and known to the system.

FID-12435.0 Additional 1xEV-DO translation parameters for optimization: This feature enables the following parameters be provisionable via the EMS: - RateLimitUserInHandoff - ControlChannelRate (38.4 kbps or 76.8 kbps) The request was originated from KTF and the parameters were intended to be changed during optimization. The associated RDAF is 707323. Also note that the RDAF requests for the change be done on the EMS. If this feature is to be done post R26, the OMC-RAN instead of the EMS should be modified.

In addition, there are several features available in R26.0 in support of the 1xEV-DO Rev A technology. These features include Support of Rev 0 calls on Rev A capable hardware (SB-EVM/SB-EVMm), Enhanced Reverse and Forward Links using Enhanced MAC Protocols with Rev A Subtype 2 Physical Layer, Generic Attribute Update Protocol (GAUP) for Dynamic Parameters Update in DO Rev A, Inter and Intra-RNC Handoff Enhancement and Application based Quality of Service (QoS) support for HRPD Rev A.

4.2 Performance related features in R27.0

The following performance related features are available in Cell/RNC release 27.0.

FID-8219.12 1x EV-DO multiple T1/E1 uplink: This feature covers the platform work to support sending reversed link traffic data upstream over multiple T1s/E1s in 1xEV-DO, with



proper priority control. This capability is important for the support of multiple carriers and the support of 1xEV-DO RevA.

FID-9138.16 Session Timeout in the EVDO RNC: This security feature provides session time-out for inactive unix sessions in EVDO RNC. Inactivity is the lack of user input to the session from a keyboard. The session is terminated along with its associated processes when the inactivity timeout value is exceeded. The inactivity timeout value is configurable by the security administrator. The value of this parameter establishes the timeout value for all users on all the RNC APs in the service node. The security administrator can configure a user to be exempt from this timeout limitation. The security administrator makes changes to the configuration at the OMP, which runs scripts to implement the changes in the subtending RNC APs. The relevant interfaces impacted by the inactivity timeout include the EVDO RNC console port, an EVDO RNC IP accessible port providing unix-like remote login (telnet/ssh) capability, and an EVDO RNC application supported by unix-like remote login (telnet/ssh) capability. This feature is dependent on SSH for secure provisioning. This feature has been requested by Sprint.

FID-9253.1 1x EV-DO performance monitoring tools enhancements: This FID provides the following two 1xEV-DO tools support capabilities: Special Engineering Studies tool to provide histogram counts of performance data for capacity planning purposes including Reverse Link Short and Long Term RSSI, Forward Link Traffic Percent Busy Slots and Datalink Utilization and a tool to identify active users per EVM (i.e., provide a Reverse Link Channel Element Status Display capability - RDAF #706248)

FID-12269.2 Enhancements to EVDO Per Call Service Measurements: This feature includes requirements listed in SRD 5677 as "deferred", Per-Call Measurement Data (PCMD) support for basic Rev A EVDO, and general PCMD and Service Measurements Enhancements. The deferred requirements from 5677 include the option of reporting either no RUM data, or reporting the data from the first and last RUM received during a PCMD Interval, and enhancements of the RUM data reported. The PCMD requirements to support Basic Rev A will be to generate PCMD Records for Personality Changes (GAUP) and optionally for flows in Multi-Flow Packet Applications (MFPA).

FID 12528.0 Rev 0 RNC to handle idle handoff from Rev A session: FID-12528.0 will add software changes for a Rel 0 target RNC to close a Rev A session idle handoff from a Rev A source RNC. For all three idle handoff methods, Assign First, Color Code and Prior Session, once the target RNC receives the old UATI of an AT idle handoff to it, it will send the A13-Session Information Request message to the source RNC and wait to receive the A13-Session Information Response message. If the target RNC is with Rel 0 software releases and the A13-Session Information Response message contains any unknown protocols indicating the AT is not with a Rel 0 session, the target RNC will close the session. The Rel 0 RNC is required to retrieve the Enhanced Idle State Protocol attributes from the A13-Session Information Response message. The information is needed in case the A13-Session Information Response message come back late and the Rel 0 RNC will have to send the SessionClose message to the AT during its awake time defined in the Enhanced Idle State Protocol.



4.3 Performance related features scheduled in R28.0

The following performance related features are scheduled to be available in Cell/RNC release 28.0. It must be noted that the feature content may change by the R28 GA time frame.

FID-9053.3 Increasing the UATI Sessions per OHM: This optional feature increases the maximum number of UATI sessions per OHM beyond 65K (to 100K). The feature also adds the OA&M functionality needed to view the memory size of the equipped AP CP2140 cards in the 1xEV-DO RNC (a continuation of 9053.2).

FID-9123.2 Bulk Provisioning Capability for 1x EV-DO with OMC-RAN: This feature is to ensure that the EVDO bulk provisioning capability via the OMP-FX is available regardless of whether the Element Manager is the EMS or OMC-RAN.

FID-11003.25 1x EV-DO Standalone support with FMS based RNC phase 1: This feature implements support with the FMS RNC, of a standalone EV-DO system using the converged (RCS based) cell OA&M implemented in R25 for mixed mode networks. This feature includes all required functions outside of FMM development covered in FID-11003.2. This includes changes to support Generic Retrofit, and documentation.

FID-12078.12 Data Over Signaling Protocol: This feature enables the support of transmission of data over the access channel (reverse link) using the Data Over Signaling protocol defined in the HRPD RevA standard. The support of forward link DoS is provided in 12078.36. This feature is useful for application such as sending messages for PTT, etc. where real time performance is critical.

FID-12078.13 Enhanced Idle State Protocol: This feature supports the Enhanced Idle State protocol specified in the HRPD RevA standard. The Enhanced Idle State protocol enhances the control of idle state monitoring and provides benefits such as reduced paging delay and better battery life to the mobile users.

FID-12373.1 Prior Session: Allow Session Transfer from/to Serving RNC: Prior to this feature, the system will block any prior session requests that are made by the AT on the RNC where it currently has its previous session. In the field, this apparently occurs frequently, especially in areas of confused or poor RF, such that, in some areas, over 40% of all prior session requests are for sessions from the serving RNC and are thus blocked. The same-RNC prior session request blocking was put in place so that the software would not perform a prior session request for sessions that were already removed by the existing hardware ID matching software. This feature will allow the previous session with matching hardware IDs to persist long enough to be transferred, and will remove the same-RNC restriction, allowing the same-RNC prior session transfers to occur. If the new session with a matching hardware ID does not request the matching prior session during session negotiation, the system will then continue on and remove the older orphaned session, as before.

FID-12456.1 RNC Grouping to Improve Idle Handoff Performance: This feature allows multiple RNC's to form a group in which AT can be served by one RNC but controlled by another during in a dormant state. The PCF entity with the A10 termination will be able to send page messages across RNC boundaries to any cell within the multiple RNCs in a Group. This feature provide the capability so that AT does not need to wake up at RNC boundary or



it wakes up across the RNC boundary but it can still keep its session. This feature will improve the overall system performance to avoid the idle handoff ping-ponging and RATI ping-ponging at RNC boundaries. The above functionalities will be available to customers in 12456.1 and this feature will provide the Infrastructure for 12456.1.

FID 12458.01 Per OHM Service Measurements in 1x EV-DO: This feature changes the existing AP service measurement counts to per OHM service measurement counts. OHM failover alarms will also be provided with this feature. Internally this feature also enhances the session memory management infra-structure to enable the support of 40K UATI sessions for the release and to allow the memory to be used more efficiently on "unused" sessions (sessions without R-P connections).

FID-12766.0 Customer Critical Service Measurements and Translations: This feature will allow fast turnaround for features that require SM or Translations on EVDO. It contains: 1. Spare EMS translations, whose values are delivered to, and are accepted by the RNC, and are available for use by the software, but are not used by this feature, and 2. Spare Service Measurements, which are available at the RNC to be pegged, but are not actually pegged by this feature, and delivered to the OMP, and to entities using SMs.

FID 13378.0 Service Measurements Re-architecture: Prior to FID-13378.0, 1x EV-DO RNC sector-carrier service measurements always contained data for six sectors for every carrier, regardless of how many sectors the carrier may have. Feature 13378.0 modifies the 1xEVDO RNC sector-carrier SM reporting, the system only generates RNC SM data for six sectors if sectors numbered 4, 5, or 6 have service measurements except when the modem that supports sectors 4, 5, and 6 was OOS for the entire SM collection interval. In this case, only three sectors RNC SM data will be reported. If both modems were OOS for the entire SM collection interval, then no SMs for the cell will be reported.



5. KPI OPTIMIZATION AND TROUBLESHOOTING

At a higher level, the performance of a 1xEV-DO system can be quantified via a few primary metrics, named here, the KPIs. When used collectively, they are indicative of the quality of service experienced by a typical end user. In a more general sense, they may also be applied to characterize performance in any wireless Data network.

These KPIs are:

Established Connection Rate

Drop Call Rate

Data Throughput

As may be evident from the names, the KPIs together help gauge the quality of a network in terms of its ability to offer successful call set-ups, minimal unintended call interruptions and fast download speeds to an end user all essential ingredients for a rich Data experience.

In the following sections, we treat the KPIs individually. For each KPI, we first provide a high level definition and then, explore the underlying elements/parameters that may negatively influence it along with the associated troubleshooting/rectification techniques.

One key point to note is that the ability to isolate a factor influencing any of the KPIs hinges on the degree to which the problem manifests. The latter itself depends on many factors such as the extent of the problem area, amount of call activity in the problem area, usage patterns, persistence of the problem, etc. In general, larger than normal degradation in a metric stands out much easier and hence, facilitates its own detection and resolution.

In 1x EV-DO R27, irrespective of how many sectors are equipped, service measurements are always reported for all six sectors. However, with the service measurements re-architecture feature in R28, service measurements data for sectors 4, 5 and 6 will be reported only if those sectors have any SM data or if the modem that supports those sectors is in OOS condition.



5.1 Established Connection Rate

In essence, this KPI represents the success rate associated with setting up a Data call. Mathematically, it can be expressed as follows:

RequestsConnectionFailuresConnectionquestsConnectionRateConnectiondEstablishe Re

The metric can be computed on a per sector basis.

The various terms in the above equation are computed from SM peg counts collected in the AP and the TP on a per-sector basis as follows:

Connection Failures sum of (counts below are pegged in the TP): AT-init Connection Attempt failures-Reverse link not acquired AN-init Connection Attempt failures-Reverse link not acquired AT-init connection attempts failures no response to traffic channel assignment AN-init connection attempts failures no response to traffic channel assignment AN-initiated connection attempt failures no resources available AT-initiated connection attempt failures no resources available AT-initiated Connection Attempt Failures other failures (R24 and before) AN-initiated Connection Attempt Failures other failures (R24 and before) AT-initiated Connection Attempt Failures other reasons pre-TCA (R25 and later) AN-initiated Connection Attempt Failures other reasons pre-TCA (R25 and later) AT-initiated Connection Attempt Failures other reasons post-TCA (R25 and later) AN-initiated Connection Attempt Failures other reasons post-TCA (R25 and later)

Connection Requests sum of (counts below are pegged in the AP): AT-initiated Connection Request AN-initiated Connection Request

A Connection Request is an event where the AT makes a request to establish a call with the AN, or vice versa. If the cell is able to assign a Traffic channel, the connection may still fail before establishing a stable traffic link for several reasons, which are aggregated as Connection Failures.

The cell may not be able to assign Traffic channel for several reasons. One reason could be it experiences trouble activating traffic channel element typically due to hardware error in the EVM.

The other reason could be that the cell is already processing maximum number of connections and cannot free up resources to serve additional users. Any type of call processing overload (such as both the TPs of an AP going in overload due to excessive traffic load) can also prevent Traffic channel allocation. These two causes essentially represent call blocking due to resource shortage. Note that per the current implementation, the call is never denied due to any power overload conditions unlike in 2G/3G1x networks.



Once assigned a traffic channel, the connection may still fail for several reasons. A call setup failure in this mode is primarily linked to some type of RF impairment (poor RF coverage, pilot pollution, excessive RF loading, etc.)

The Established Connection Rate may alternatively be viewed as the deficit between Connection Requests and the calls that did manage to come up stable on the Traffic Channel. A complementary metric commonly used is called Access Failure rate. It can be represented as (1 Establish ConnectionRate) and signifies failed call or access attempts.

Service providers define their own metrics to track performance. Verizon wireless has created Failed Connection Attempts (FCA) as an equivalent metric for Established Connection Rate. Alcatel-Lucent engineers might encounter this metric when attending to Verizon wireless specific performance issues. At this point, the FCA metric equation is the same as Alcatel-Lucents Established Connection Rate equation.

To improve the Established Connection Rate, it is important to understand the constituent factors influencing the deficit. We catalogue and explain these in the subsequent sections. But, first we briefly describe the call setup process.

It must be noted that R27 SU1 changes RNC to handle the connection close by closing the connection. This is expected to release the call drop rate but since the call is not considered established and therefore will increase the failed connection attempts. In R27 SU0, this scenario is pegged as a drop call. The connection close is considered as TCC received and therefore the call is considered as established. In R27 SU1, compared to R26, the AT/AN initiated connection attempts failed due to no TCC received peg count is expected to decrease. Compared to R27 SU0, the AT/AN initiated established connections as well as the Connection Release due to RLL are expected to decrease.

5.1.1 Call setup process

Figure 4 illustrates a typical flow of messages between AT and the AN for an AT initiated call. It bears many similarities with a typical origination call attempt in a 2G/3G1x network.

When the AT receives the command to establish a call (such as, from the laptop connected to AT when user tries to establish a fresh PPP session or reactivate a dormant3 session), AT first checks to see it is current with the overhead messages, and if not, updates them.

On the next available access channel slot, the AT sends a Connection Request message via an access probe. Along with the Connection Request message, the AT always sends a Route Update message listing the received pilot strengths. If a response is not received within a finite time, the

3 The first time AT establishes traffic channel with the AN, it is called a new session. The session may last for several

hours or days depending on the AN configuration. Within a session, the AT can switch between active and dormant modes numerous times. Data is exchanged between AT and AN in active mode over traffic channels. After a short period (typically of the order of a few seconds depending on the AT/AN setting) of inactivity, the AT goes into dormancy, that is, tears the traffic link and goes into idle mode. When a fresh set of data appears, either side (AT or AN) can initiate a connection out of dormancy (generically termed reactivation) depending on who has data to send. Note that a new session is always initiated by the AT. The reactivation can either be from the AT or AN.



AT will retransmit the probe. It will keep sending the probes (based on the Number of Access Probes configuration parameter value in the Sector and Service Node forms), each at incrementally higher power, until it receives AcAck (Acknowledgement) from the cell.

Once the cell is ready with the necessary resources and once the cell receives the DRC Cover Indication from the controller, it sends a Traffic Channel Assignment message (TCA) to the AT. If there were multiple pilots in the Route Update message, cell will transmit TCA on all the sectors.

On receipt of the TCA, the AT initiates reverse link traffic channel transmission. There is no traffic channel information sent at this time, only DRC and reverse pilot bits. If there were multiple pilots in the TCA, the mobile will include all these Pilots in its Active set. The AT will gradually increase its transmit power until it receives acknowledgement from the cell site or it times out.

FIGURE 4 CALL FLOW FOR AN AT INITIATED CONNECTION AT BTS Controller

Route Update & Connection Request Message

Allocate Traffic Channel Request

Allocate Traffic Channel Response AcAck

DRC Cover Indication

Send Traffic Chn Assignment Traffic Channel Assignment

Send DRC + Pilot and ramp up Reverse Traffic Channel Mobile Acquired Indication

Send RTC Ack RTC Ack

Traffic Channel Complete Traffic Channel Complete

Ack

Configuration Negotiation Procedures

Command received by the AT to initiate a call. AT updates overhead messages and waits for the next Access Slot to transmit



When cell site acquires the reverse link transmission, it sends an acknowledgement called RTCAck to the AT. Once the mobile receives RTCAck message, it sends a Traffic Channel Complete (TCC) message to the cell site. Cell site acknowledges it back with an Ack message.

If there are any negotiation procedures to be completed, a set of messages is exchanged next to arrive at the proper configuration. For example, there are certain parameter values that the mobile assumes by default per standards. This saves the control channel bandwidth (in 2G/3G1x networks, such information is transmitted via overhead messages such as via System Parameters message and Access Parameters message). If the cell site is configured with non-default parameter settings, it must negotiate (exchange) these with the mobile. After the negotiation is finished, the call setup is complete at this time and the actual Data contents can now be carried over the air. However, any negotiation failure does not contribute to the Established Call Failure rate metric.

Many of the access enhancements first introduced by IS95B standards are also available in a 1xEV-DO system. For example, after sending the first probe and while waiting for the AcAck, the mobile can switch to a stronger Pilot. This is similar to Access Probe Handoff (APHO) specified in IS95B/IS2000 standards. After receiving the AcAck and while waiting for the TCA, the mobile can again switch to the previous or different stronger Pilot. This is similar to Access Handoff (AHO) specified in IS95B/IS2000 standards. Finally, the TCA can contain multiple Pilots that the mobile can place in its Active set as it enters Traffic channel. This is similar to Channel Assignment into Soft Handoff (CAMSHO) specified in IS95B/IS2000 standards. Notes that these features are implemented as core functionalities in Alcatel-Lucent 1xEV-DO system and cannot be disabled via translations.

The process for AN initiated call (akin to a mobile termination attempt in 2G/3G1x network) is similar to the above except that the cell site first pages the mobile to invoke Connection Request message from the AT. In a wireless Data network, all AN initiated calls are typically reactivations from dormancy; the network does not initiate a fresh session with the mobile. A mobile on the other hand is capable of initiating a new session or reactivation from dormancy.

5.1.1.1 Fast Connect Calls

Fast Connect is a special type of AN initiated call. When the AT has been dormant for 5 seconds or less (controlled by the tunable parameter called Suspend Timer) and the AN has a fresh set of data to transmit to this AT, the AN uses this procedure to bring a dormant session back onto the Traffic channel. Basically, it just sends the TCA to the AT, which consists of Pilots in the ATs Active set at the time prior call release. The idea is that 5 second is a relatively short interval and the AT must have been in the vicinity, so it is safe to locate it on the prior set of Pilots. It is appropriately called Fast Connect since there is not much time spent on the Control/Access channel message exchange and the attempt is reactivate the Traffic channel quickly.

The AN sends the TCA once it has ascertained necessary resources on the AN. This results in the peg for Fast Connect Requests. If the cell times out waiting for TCC from the mobile, it pegs it as Fast Connect Failure No response to traffic channel assignment. If the cell does not acquire the reverse link after the TCA was sent, then the Fast Connect Failure Reverse link



not acquired count is pegged. Both the above counts are pegged on the AP on a per AP basis. Per sector granularity is not available. These counts are not pegged towards any of the counts for AN Initiated calls.

For such calls, Fast Connect Failure rate metric (for AT/AN Initiated calls) can be derived. It can be defined on a per AP basis as follows:

RequestsConnectFastacqnotRLtFailuresFastConnec

TCAtoespNoFailuresConnectFast

rateFailureConnectFast ___R_

One may want to compute a composite metric for Established Connection Rate consisting of counts for AT Initiated, AN Initiated and Fast Connect calls together at the per sector-carrier level. However, since the pegs for Fast Connect do not have the granularity down to the sector-carrier basis, the Established Calls rates will have to be computed separately for AT/AN Initiated calls (per sector-carrier) and Fast Connect calls (per AP).

Many of the factors commonly impact AT/AN Initiated calls as well as Fast Connect Calls. These are discussed in the subsequent sections.

5.1.2 RF access failures

As the name suggests, this type of failures impacts access performance due to RF performance limiting factors. By definition, the RF access failures influencing the deficit between Connection Request and successful call setup occur after a Traffic channel is assigned4 by the AN, but before Traffic Channel Complete message is received from the AT. The failures represent the inability to complete call setup due to inadequate signal strength received by the AT and/or AN. Basically, the RF impairments break down the message flow and/or the signal acquisition resulting in an incomplete call setup.

Call setup is a multi-stage process, and hence there are various places where RF breakdown may occur. There are separate SM counts to trap failures (for both AN initiated and AT initiated connections) at each stage. These are:

After sending a Traffic Channel Assignment Message to the AT, AN times out waiting to acquire DRC channel from the AT. The SM count to trap this is called Connection Attempt Failure Reverse link not acquired.

After AN successfully acquires AT on the reverse link, it sends an Acknowledgement message, called RTCAck, to the AT. AT is expected to Ack it back with the TCC message. If AN does not receive the TCC after 3 attempts of transmitting the RTCAck, it will result in a failed call attempt. The SM count to capture this event is called Connection Attempt Failure - No response to Traffic Channel Assignment. Note that

4 From AT perspective, failure may occur much sooner but AN will not know until it times out waiting for response to

the Traffic Channel Assignment.



the peg count is a misnomer; it conceptually represents no receipt of TCC, not no response to TCA.

All the above stages apply regardless of an AT or AN initiated Connection Request. Accordingly, there are separate SM counts for connection requests (that is, AT or AN initiated Connection Request), connection failures and separate metrics (AN initiated RF failure rate and AT initiated RF failure rate) to track them.

Next, we elaborate on potential RF factors influencing the failures along with some mitigation strategies. Their impact may be felt anywhere along the call setup process although with varying degrees. It is not uncommon to see similar symptoms for problems induced by more than one of the RF factors. Thus, a comprehensive understanding of all RF factors is called for to be able to eliminate unrelated RF factors and single out the root cause.

5.1.2.1 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage area (such as AT receive power less than 105 dBm and best Pilot SNR < -10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss resulting in its inability to close one or both the links. Typically, the reverse link runs out of coverage first given that the 1xEV-DO forward link has roughly 10 dB advantage (Pilot channel transmitted at full power). Areas suffering from heavy shadowing such as in-building locations or stretch of route heavily blocked by building, tree, etc can likely experience this type of failure.

Failure Symptoms and Identification strategies

Such problems are best detected via drive tests. The drive test area can be narrowed to within the coverage of a few cells by identifying cells via SM, which tend to exhibit unusual degradation in many QoS indicators, including:

Low Established Connection Rate High Drop Call rate (defined in section 5.2) High Page Failure rate High RLP re-transmission rates (ratio of RLP Octets Re-transmitted to RLP Octets

Transmitted on the forward link, ratio of Missing RLP Octets Requested to RLP Octets Received on the reverse link these counts are collected in the TP for the DRC pointed sector).

Suggested Mitigation Techniques

Having identified select areas, drive test based RF optimization techniques can be employed as detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case



adding repeater5 (especially for in-building environments) or cell to fill in low signal strength areas.

Improving the coverage may also result in secondary effects such as overall performance improvement as interference is mitigated due to reduced traffic channel power requirement.

5.1.2.2 Excessive Pilots/Non-dominant Pilots


Presence of too many pilots in an area often impacts forward link performance due to interference it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in such a problem area, that is, there are several pilots with weak comparable strengths, often accompanied by constant pilot thrashing (dominant sector changing quickly). Another scenario is where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Call setups are susceptible to failure in multiple pilot regions since AT has no single strong pilot to tune to. While AT can enter into Traffic Channel with multiple pilots in the Active set, there may still be some degradation since AT can tune to only a single pilot at any given time on the forward link. Any delay in completing the call set-up (for example, AT may miss the first RTC_ack message before it tunes to a stronger pilot, the message will be resent a few frames later) could impair the Established Connection Rate.

Failure Symptoms and Identification strategies There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more Pilots provide one such indication. There are also separate SM counts to flag Connection Requests with 2 Pilots and Connection Requests with 3 Pilots events. All these counts are reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple pilot counts] reflect the number of Connection Requests with single pilot. The following four metrics in SPAT3G can verify the existence of the pilot pollution problem:

Connection Request with One Pilot (%) Connection Request with Two Pilots (%) Connection Request with Three Pilots (%) Connection Request with Four or more Pilots (%)

Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT) 5 Current 1xEV-DO deployments are based on regular cells; generic repeaters may first require thorough

characterization to ensure they maintain decent transmit SNR in the forward direction.



when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can also indicate pilot pollution problems.

Another pointer, though a weak one6, is excessive activity for Soft and/or Softer handoff attempts. It is likely that other KPIs such as Dropped call and Data throughput may also exhibit impairment in these areas.

For one-to-one overlay on a Alcatel-Lucent 2G or 3G1x network, data and tools from the underlying 2G/3G1x infrastructure may also be employed for identification of excess-pilot areas. Note, however, that this type of comparative analysis will be valid only if the antennas are shared across technologies, or in physical proximity with similar attributes (azimuth, downtilt, height, beam width)

2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage, or Total Walsh Code relative to Primary Erlangs, are potential indicators to excess pilot regions.

Handoff Matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlying 2G/3G1x network to flag cells overshooting from a distant region. Every effort must be made to contain their coverage/reduce their interference in order to not only benefit call set-up rate, but also to improve overall performance (drop call rate, throughput, etc).

Suggested Mitigation Techniques Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve such problem areas. Typical RF optimization techniques include adjusting cell site power and/or antenna downtilt of one or more neighboring cells to create a dominant pilot or fewer visible pilots. Antenna downtilt usually are more effective in controlling coverage of overshooting sectors. Refer to [8] for more details.

5.1.2.3 Poorly constructed Neighbor Lists


Not having a fine tuned neighbor list is a common source of access failures. In essence, it prevents AT from being on the best pilot during the access process. A poor neighbor list may typically be missing a nearby pilot to which AT will otherwise handoff to, or may have integrity issues, s