Specification+ +Latest+Version

EDGE BSS Performance

TEST SPECIFICATION

v. 3.0.0 (for S13)

For Internal Use

INTERNAL HISTORY

Archive

Location: PI database:

http://esdoc04nok.ntc.nokia.com/urn.htm?version=current&id=0b006c37802d6be9&DMW_DOCBASE=espoo11

Filename: EDGE BSS Performance Test Specification (this document)

EDGE BSS Performance Test Matrix

History

Date Version Author Change Note No./Notes

16.09.2004 0.0.1 Ari Kosonen Initial Draft

12.10.2004 0.0.2 Ari Kosonen Template changed

08.11.2004 0.0.3 Ari Kosonen Delivered for 1st email review, mainly related to Chapter 5 Test Items.

26.11.2004 0.1.0 Ari Kosonen Updated after 1st email review (12.11.2004) comments

02.12.2004 0.1.1 Ari Kosonen New test item for Es/N0 added

03.12.2004 0.1.2 Ari Kosonen Delivered for 2nd review

27.12.2004 0.2.0 Ari Kosonen Updated after 2nd review

14.01.2004 0.2.1 Ari Kosonen Test Cases updated to comply with Test Matrix.

31.01.2005 0.2.1 Ari Kosonen EDAP configuration and KPIs updated

10.02.2005 0.2.2 Ari Kosonen Delivered for 3rd (email) review

23.02.2005 0.3.0 Ari Kosonen Updated after 3rd (email) review

03.03.2005 1.0.0 Ari Kosonen Approved (VPK)

14.06.2006 1.1.0 Ari Kosonen First Draft for S12

http://esdoc04nok.ntc.nokia.com/urn.htm?version=current&id=0b006c37802d6be9&DMW_DOCBASE=espoo11%20

http://esdoc04nok.ntc.nokia.com/urn.htm?version=current&id=0b006c37802d6be9&DMW_DOCBASE=espoo11%20

NSN/RA/EDGE SFP

BSS EDGE Performance Test Specification v.3.0.0

21.07.2005 1.1.1 Ari Kosonen Test cases updated

23.08.2005 1.1.2 Ari Kosonen Minor updates according to email comments

08.09.2005 1.1.3 Ari Kosonen Test items related to PBCCH removed

19.09.2005 1.1.4 Ari Kosonen

03.10.2005 1.1.5 Ari Kosonen Delivered for comments

19.10.2005 1.1.6 Ari Kosonen Updated acc.to. comments

20.10.2005 1.1.7 Ari Kosonen EQoS cases added

21.10.2005 1.1.8 Ari Kosonen Delivered for review (28.10.2005)

27.10.2005 1.1.9 Ari Kosonen DTM cases added

28.10.2005 1.1.10 Ari Kosonen Updated with initial email review comments

31.10.2005 1.2.0 Ari Kosonen Updated after review 28.10.2005

31.10.2005 2.0.0 Ari Kosonen Approved (VPK)

18.04.2006 2.0.1 Ari Kosonen Parameters updated

05.05.2006 2.0.2 Ari Kosonen KPIs updated, few new test items.

25.07.2006 2.1.0 Ari Kosonen Updated after email review (19.05.2006)

02.03.2007 2.2.0 Ari Kosonen First draft for S13

16.04.2007 2.2.1 Ari Kosonen Updated to comply with the reviewed Test Matrix

26.04.2007 2.2.2 Ari Kosonen Delivered for review (14.05.2007)

30.05.2007 2.2.3 Ari Kosonen Updated after review 14.05.2007

14.06.2007 3.0.0 Ari Kosonen Approved (TMö 6.6.2007)

Issue 3.0.0 © Nokia Corporation 3 (75)

NSN/RA/EDGE SFP


Contents

1 Introduction 61.1 Schedule 7

2 Testing Environments82.1 Test Servers 82.2 Testing Equipment and Software 102.3 Network Hardware Requirements 112.3.1 BSC hardware 112.3.2 BTS hardware 112.3.3 BSS requirements for different test phases 112.4 Network Software Requirements 132.5 (E)GPRS Default Parameters 142.6 BSC configuration suggestions 172.7 Transmission configurations for Abis over IP tests 182.7.1 Abis over IP 182.7.2 Abis optimization 192.8 Subscriber terminal configuration 192.8.1 Connection between MS and laptop PC 212.8.2 Recommended OS for the laptop PC 222.8.3 Windows XP TCP parameters 23

3 Traffic considerations 243.1 Applications and Services 243.2 Traffic Mix 25

4 EDGE Key Performance Indicators 26

5 Test Items 275.1 Quality : Throughput 275.1.1 (E)GPRS UL/DL throughput per TSL in optimal radio

conditions 275.1.2 (E)GPRS UL/DL throughput per TSL in varying radio

conditions 305.1.3 MCS/CS Distribution vs. C/I (or BLER) 335.1.4 MCS/CS distribution vs. RX Level 345.1.5 MCS/CS distribution vs. fading 365.2 Quality : RTT and latency 375.2.1 Active RTT 375.2.2 Idle RTT 395.3 Quality : User Experience 425.3.1 HTTP User Experience (4 seconds) 425.3.2 FTP file download and upload duration435.3.3 HTTP web page download duration 435.3.4 PDP Context Activation Duration 44


NSN/RA/EDGE SFP


5.3.5 GPRS Attach Duration....................................................455.4 Quality : EGPRS Cell changes and reselections............475.4.1 EGPRS Cell Change Duration (NCCR and NACC)........475.5 EDGE-WCDMA Interworking..........................................505.5.1 EDGE-WCDMA Intersystem Cell Change Duration.......505.6 EQoS..............................................................................525.6.1 EGPRS Cell Change Duration with EQoS......................525.6.2 HTTP Download Duration with EQoS.............................535.6.3 RTT with EQoS...............................................................535.7 DTM................................................................................545.7.1 DTM Setup.....................................................................545.7.2 DTM Transfer Break.......................................................555.7.3 DTM non-real time throughput........................................565.7.4 DTM RTT........................................................................575.7.5 DTM Cell Change Duration............................................575.7.6 DTM Coverage...............................................................585.7.7 DTM GPRS Attach Duration...........................................595.7.8 DTM PDP Context Activation Duration...........................605.8 EDA/HMC.......................................................................615.8.1 EDA/HMC Throughput....................................................615.8.2 EDA/HMC RTT...............................................................635.9 Other test items..............................................................635.9.1 PCU LT...........................................................................635.9.2 BSC FT...........................................................................64

6 Test matrix....................................................................66

7 Test reporting...............................................................677.1 Test result storage and format........................................67

8 Reference Information.................................................688.1 Nominal data rates.........................................................688.2 Multislot Classes (incl. High Multislot Classes)..............69


NSN/RA/EDGE SFP


1 Introduction

The purpose of this document is to enable fact based EDGE performance understanding and to describe the validation and benchmarking of the key EDGE performance issues on BSS. The results of the validation and benchmarking will be used as a input for EDGE Performance Roadmap and EDGE Key Benefits documents for customers. The focus on these tests is in BSS level performance of the BSS release (e.g. BSS13) compared to the previous releases, using the latest network element releases compatible on this BSS level (e.g. S13, CX6, EP2)

Figure 1. Focus of the EDGE BSS Performance Testing

Testing is carried out by different testing areas, at least the following:

BSC Functional Testing (Tampere),

BSC Compatibility Testing (Tampere)

BTS Functional Testing (CX(M): Southwood, EP: Gurgaon),

BSS Transport Release Testing (Düsseldorf)


NSN/RA/EDGE SFP


BSC System Testing (Espoo),

BSS System Verification (Espoo),

The focus of this document is on the performance issues related to EDGE. The main objectives are to define the key measurable items related to EDGE on BSS level, to define the test cases to validate these and to define the environments and configurations in which the tests shall be performed.

The aim is to create set of test descriptions that can be used as basis while defining the test cases for different test phased in several releases and thus enable benchmarking between the releases.

The test case descriptions and test reports will be stored to TestMan, under the title “BSS EDGE Performance Testing”.

The summaries of the test results will be collected to Test Matrix, which is stored to PI in the directory of EDGE SFP: Projects/GSM EDGE BSS R&D/System Features/EDGE SFP/01 Nokia EDGE Feature Management/Deliverables/EDGE Performance Test Specification

The parts of this document that have been changed or added since the last approved version (2.0.0 for S12) are highlighted with yellow.

1.1 Schedule

The tests shall be started as soon as the SW is mature enough to provide reliable performance results. Anyhow, the basic performance test results (such as RTT, Throughput) shall be available before the E4 milestone of each release. Results of more complex tests shall be available before E5.

The E4 vs. E5 priority for test items is given in the EDGE BSS Performance Test Matrix.

For each release, the E4 and E5 reports of EDGE Performance shall made based on these test results.


http://esdoc04nok.ntc.nokia.com/urn.htm?id=0b006c37801f13e8&DMW_DOCBASE=espoo11



NSN/RA/EDGE SFP


2 Testing Environments

2.1 Test Servers

Dedicated test servers are provided to be used in the EDGE BSS Performance testing in order to provide controlled environment resulting comparable test results. The set of test files and test pages shall be identical in all of the listed servers.

The addresses for the test servers are given in Table 1.

Table 1. Test servers

Usage Location and address Contact Person

HTTP Test Servers

Common EDGE Performance Test Server: 10.16.62.243

Jukka-Pekka Siberg

FTP Test Server

Common EDGE Performance Test Server: 10.16.62.243 or 10.16.62.244

ftp-username: sopro / password: solution1

Jukka-Pekka Siberg

Multimedia Streaming Server

Common EDGE Performance Test Server: http://10.16.62.243

Jukka-Pekka Siberg

HTTP Test Server (SoPro) reachable from Internet

http://www.soprote.net


http://www.soprote.net/

NSN/RA/EDGE SFP


The test server has a dedicated folder for each test phase. Each of these folder has a similar structure and contains the same test files.

Links to all HTTP test web pages for each test phase are on the following page:http://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /http/testpages.html.

All the test files to be used in FTP tests are in their dedicated folders:

Downlink test files: ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/dl

Uplink test files: ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/ul

The test file structure and the respective folders for each test phase are shown in Figure 2.

Note: the directory paths given above should be shortened to be compatible with the App. Tester. This document will be updated accordingly later.

Figure 2 EDGE Performance Test File Structure


ftp://10.16.62.243/~sopro/EDGE_Test/


http://10.16.62.243/~sopro/EDGE_Test/

NSN/RA/EDGE SFP


2.2 Testing Equipment and Software

The following testing equipments are suggested to be used:

NetHawk GSM Analyzer (Abis, Gb, A) Ethereal / CommView or similar IP protocol analyzer TEMS (5.1 or later with GPRS/EDGE support) NEMO Outdoor (as an alternative to TEMS) Multipath Fading Simulator (commercially available) Delay generator (for interference generation) Attenuators e.g. Step Attenuator Modules (SAM) Wideband noise generators TestHawk (Application Tester)

Note: to ensure that same settings are used in each test phase using the AppTester, the configuration files per each test case shall be stored to a common storage in PI, in to the following folder:

http://esdoc04nok.ntc.nokia.com/urn.htm?id=0b006c37807351c9&dmw_docbase=espoo11

NetAct (e.g. to collect the counters) PCBSC or similar tool to get the counter data from BSC (if NetAct

is not available) Agilent E6474A Wireless Network Optimization Platform (for

DTM testing) IP simulator, e.g. NistNet or PacketStorm (for Abis over IP testing)


http://www.packetstorm.com/

http://snad.ncsl.nist.gov/itg/nistnet/

http://cp.literature.agilent.com/litweb/pdf/5988-5904EN.pdf



http://www.nemotechnologies.com/index.php?249

http://www.ericsson.com/products/tems/index.shtml

http://www.tamos.com/products/commview/

http://www.ethereal.com/

https://www.nethawk.fi/products/nethawk_analyser/gsm_analyser/

NSN/RA/EDGE SFP


2.3 Network Hardware Requirements

2.3.1 BSC hardware

The tests can be performed either with BSC2i or BSC3i hardware, depending on the hardware available in each participating test laboratory.

At least the PCU variants given in Table 2 shall be tested, other PCU variants may also be benchmarked. In S12 and beyond the emphasis is on the PCU2 units, but benchmarking between the PCU generations shall be made.

The BSC and PCU hardware configuration shall be described with the test report.

In the tests performed in PCU LT, all the PCU variants shall be benchmarked.

Table 2. BSC and PCU hardware combinations to be used.

BSC HW BSC2i BSC3i

PCU1 variants PCU-T PCU-B

PCU2 variants PCU2-U PCU2-D

2.3.2 BTS hardware

MetroSite, UltraSite and FlexiBTS, shall be benchmarked in EDGE Performance tests done in BTS test phases.

In other test phases the EDGE-capable BTS hardware currently available shall be used.

2.3.3 BSS requirements for different test phases

2.3.3.1 BSS System Verification Fully operational Network and Switching Subsystem (NSS) , Packet

Core and EDGE capable BSS are required for testing

Streaming Server for EQoS


NSN/RA/EDGE SFP


2.3.3.2 BSC System Testing Fully operational Network and Switching Subsystem (NSS), Packet

Core and EDGE capable BSS are required for testing.


2.3.3.3 BSC Functional Testing Fully operational Network and Switching Subsystem (NSS) , Packet

Core and EDGE capable BSS are required for testing.


2.3.3.4 BTS Functional Testing / Integration Testing Fully operational Network and Switching Subsystem (NSS) , Packet

Core and EDGE capable BSS are required for testing

Signal generator is required to generate interference to verify link adaptation (LA) and incremental redundancy (IR) functionality.

Multipath Fading Simulator is required to generate different fading patterns.

2.3.3.5 Compatibility Testing

Fully operational Network and Switching Subsystem (NSS) , Packet Core and WCDMA and EDGE BSSs are required for testing

2.3.3.6 BSS Transport Release Testing

Fully operational Network and Switching Subsystem (NSS) , Packet Core and EDGE capable BSS are required for testing

IP based transport equipment for Abis over IP


NSN/RA/EDGE SFP


2.4 Network Software Requirements

The required software versions are given in Table 3.

As an entry criterion, the tests may be started after E3, if stabile software builds are available and not any major changes are expected later, i.e. the software must be very close to E4 standard with no Pronto fixes planned, which would impact the EDGE performance.

The final post-E4 tests shall be done with E4 level of both BSC and BTS software.

The main focus in the tests is on the BSS release currently under development (currently S13).

Table 3: Required SW Versions

BSS Release

BSS11.5 BSS12 BSS13 BSS14

BSC S11.5 S12 S13 S14

BTS CX4.1 for UltraSite

CXM4.1 for MetroSite

CX5 for UltraSite

CXM5 for MetroSite

EP1.0 for FlexiBTS

CX6 for UltraSite

CXM6 for MetroSite

EP2.0 for FlexiBTS

CX7 for UltraSite

CXM7 for MetroSite

EP3.0 for FlexiBTS

NetAct OSS4 OSS4.1 OSS4.2 OSS5.1

SGSN SG4 SG5 SG6 ?

GGSN GN4.1 GN5.0 ? ?


NSN/RA/EDGE SFP


2.5 (E)GPRS Default Parameters

The following default settings shall be used during testing:

General GPRS/EGPRS parameters

(E)GPRS is enabled in BSC

EGENA = Y

IMEI checking is turned on in SGSN

Ciphering is used in SGSN. Anyhow, if e.g. NetHawk tracing is needed to analyse the results, the ciphering is not required.

Payload and IP Header Compression is not used in SGSN. This is because the gain of the compressions depends on data content, i.e. without compression it is easier to get comparable results. The gain of compressions is validated in dedicated test cases.

Authentication, Identity Request, and P-TMSI re-allocation are used

BS CV MAX = 9 (Optimized for EDGE).

The initial CS/MCS is defined per each test case, depending on the application used.

LLC PDU size = 1500 bytes (has an impact when RC2 is used).

Suggested GPRS territory configuration:CDED = 4 TSLs / CDEF = 4 TSLs / CMAX = 100%If different territory sizes are used, these shall be reported with the test results.

Link Adaptation parameters

Default values shall be used for the link adaptation parameters:

Initial MCS for acknowledged mode (MCA) = 6 (See TN820, [23])

Initial MCS for unacknowledged mode (MCU) = 5 (See TN820, [23])

Maximum BLER in acknowledged mode (BLA) = 90

Maximum BLER in unacknowledged mode (BLU) = 10

Mean BEP offset GMSK (MBG) = 0

Mean BEP offset 8PSK (MBP) = 0

GPRS Adaptive LA algorithm enabled (ALA) = YES


NSN/RA/EDGE SFP


GPRS DL initial CS with LA in ack mode (DCSA) = 5 (CS2)

GPRS DL initial CS with LA in unack mode (DCSU) = 5 (CS2)

GPRS UL initial CS with LA in ack mode (UCSA) = 5 (CS2)

GPRS UL initial CS with LA in unack mode (UCSU) = 5 (CS2)

RX diversity

RX diversity (RDIV) shall be enabled, if applicable in the test environment

Frequency hopping parameters

RF hopping (HOP=RF) is used on all sectors with 2 TRXs or more

Random hopping mode (HSN > 0) is used

Note: Frequency hopping is not used in the tests, where the performance is measured in different interference situations.

The parameters given in Table 4 parameters have an impact on RTT. The current default values shall be used in testing. See also [17 ] for other suggestions.

Table 4 Parameter values having impact on RTT (EUTM and Flow control parameters)

Parameter (PRFILE) name

(parameter class and ID)

Range Suggested values

EXT_UTBF_USAGE (002:0899) TRUE/FALSE TRUE = EUTM in use

Note: as a default EUTM is not in use. Anyhow, especially for testing the RTT performance, with phones supporting EUTM (Rel4 or later), EUTM shall be enabled.

UL_TBF_REL_DELAY_EXT (046:0071) 300..3000 ms 2000 ms

See TN816, [21]

UL_TBF_RELEASE_DELAY (046:0068) 100..3000 ms 500 ms

UL_TBF_SCHED_RATE_EXT (046:0072) 2…50(40…1000 ms)

4 [ = 80 ms]See TN816, [21]

DL_TBF_RELEASE_DELAY (046:0067) 100..5000 ms 1000 ms


NSN/RA/EDGE SFP


BS CV MAX 9See TN816, [21]

POLLING_INTERVAL_BG (046:0087) PCU2 only 4 See TN845, [24]

CHA_CONC_UL_TBF_FAVOR_DIR (046:0100) Favoured direction in concurrent UL TBF establishment:

0 = UL Favoured

1 = DL Favoured

1

(Note: favoring the UL direction may in some cases provide better RTT results)

CHA_CONC_DL_TBF_FAVOR_DIR (046:0101) Favoured direction in concurrent DL TBF establishment:

0 = UL Favoured

1 = DL Favoured

1

(Note: favoring the UL direction may in some cases provide better RTT results)

FC_R_DIF_TRG_LIMIT (049:0020) 0…1000(0…100 %)

100 [ = 10%]See TN819, [22]

FC_MS_R_DEF (049:0021) 1000…65280B 4500 [ = 36 kbit/s]See TN819, [22]

FC_MS_R_DEF_EGPRS (049:0022) 1000…65280 B/s 15000 [ = 120 kbit/s]See TN819, [22]

FC_MS_R_MIN (049:0023) 20…200 B/s 40 [ = 320 kbit/s]See TN819, [22]

FC_MS_B_MAX_DEF (049:0024) 2000…65280 B 10000 [ = 10 kB]See TN819, [22]

FC_MS_B_MAX_DEF_EGPRS (049:0025) 2000…65280 B 25000 [ = 25 kB]See TN819, [22]

FC_R_TSL (049:0026) 1000…65280 B/s 1500 [ = 12 kbit/s]

FC_R_TSL_EGPRS (049:0027) 1000…65280 B/s 4500 [ = 36 kbit/s]See TN819, [22]

FC_B_MAX_TSL (049:0028) 5000…65280 B 25000 [ = 25 kB]

FC_B_MAX_TSL_EGPRS (049:0029) 5000…75000 B 25000 [ = 25 kB]

EGPRS_UPLINK_PENALTY (046:0047) 1…255 1

EGPRS_UPLINK_THRESHOLD (046:0048) 1…255 25

EGPRS_DOWNLINK_PENALTY (046:0049) 1…255 1

EGPRS_DWNLINK_THRESHOLD (046:0050) 1…255 25


NSN/RA/EDGE SFP


2.6 BSC configuration suggestions

Different lab environments are having different kind of configurations e.g. for radio network and BSS interfaces. These may have an impact on EDGE performance results, e.g. on achieved RTT or Throughput.

Therefore, the radio network and Abis configuration used in test runs should be described with the provided results. Following items should be described:

Number of BTSs and TRXs connected to one logical PCU

Number of GPRS and EGPRS capable TRX

GPRS territory sizes (e.g. CDED = 5 TSLs / CDEF = 5 TSLs / CMAX =100% )

Number of GPRS, EGPRS and EDAP channels connected to one logical PCU

Number of EDAP pools per PCU

Size (number of 64 kbit/s TSLs) of the EDAPs

Number of SMCHs and SSCHs (getting this information may be difficult. They may be calculated if the RNW/EDAP/DSP allocation is known)

Gb interface (FR) configuration

GP slot shall be configured to the same carrier as BCCH

RX diversity shall be used, if applicable in the test environment

Usage of combined BCCH (MBCCHC) is not recommended. Anyhow, usage of MBCCH or MBCCHC shall be mentioned in the test results as it has impact on RTT and throughput results

The Gb interface over Frame Relay shall be used in these tests. Use of a full 31*64 kbit/s Gb interface per each logical PCU is suggested in ETSI environment, or alternatively a 24 * 64 kbit/s Gb interface in ANSI environment.

The Gb over IP (S11.5 ->) is used for benchmarking in some of the cases. In these cases both IPv4 and IPv6 is tested separately.

The test configuration should be dimensioned so, that it does not place bottlenecks to EDGE performance, but on the other hand it should reflect a typical configuration used currently (or in the near future) in the live networks. See [17]for other suggestions.


NSN/RA/EDGE SFP


For RTT and throughput tests, especially with Rel4 phones, the Extended UL TBF Mode (EUTM) shall be enabled. EUTM is activated by setting the PRFILE parameter EXT_UTFB_USAGE (002:0899) value to TRUE. As a default EUTM is not in use. For the recommended parameter settings for EUTM, see [21].

Example of a configuration used in live networks currently:

EDAP size : 6 TSLs (64 kbit/s), shared by 3 sectors (all TRXs are EGPRS capable)

Default EGPRS territory: 3 TSLs

Dedicated EGPRS territory: 1 TSL

Further information on EDGE dimensioning principles is available on the S11.5 EDGE Dimensioning documents [12]. [13], [14] [15] and [16] The documents are available from NED.

2.7 Transmission configurations for Abis over IP tests

2.7.1 Abis over IP

Note: the Abis over IP configuration details and hardware to be used are still partially open, this chapter will be updated later.

Parameters for Device Under Test (DUT):

Jitter buffer = xxx

Number of TDM frames mapped to a packet = xxxx

Parameters of IP network; two scenarios:

"Best case"

o IP network delay 2ms

o delay variaton 0,5 ms


http://clayface.ntc.nokia.com/ned/NED?action=retrieve&identifier=dn056622&edition=1&language=en&coverage=global&encoding=xhtml_1_0&component=toc&item=data&fragment=1&product_release=bss11_5

NSN/RA/EDGE SFP


o packet loss 0,002%

"Worst case"

o IP network delay 10ms

o delay variation 5 ms

o packet loss 0,01%

Test equipment

IP simulator; e.g. NistNet or PacketStorm

Abis over IP equipment (currently 3 potential vendors: RAD, Seabridge, Tellabs)

2.7.2 Abis optimization

Test equipment:

Abis optimization equipment(currently 2 potential vendors: NMS, Seabridge)

2.8 Subscriber terminal configuration

A typical mobile handset commonly available shall be used in these tests.

Currently, (mid-2007) the handsets suggested to be used in these tests are recent Rel4 phones, such as Nokia N80 and Nokia N93. Other handsets may be used in tests requiring big amounts of MSs, e.g. in BSS Load Tests.

GSM mobile handsets used in the test phases shall support the following features

GPRS (5+1, 4+1, 3+2)

EGPRS (5+1, 4+1, 3+2, 3+1)

Extended UL TBF (S11)


NSN/RA/EDGE SFP


Improved Extended UL TBF (S13)

NACC (S11.5)

NCCR (S11.5 )

SAIC (S12)

EDA/HMC (S12)

DTM (S12 ED)

EQoS (S13)

A mobile handset supporting EUTM, and with the latest RTT enhancements (e.g. N80) shall be used e.g. for the throughput and RTT measurements, while other handsets may be used to generate additional traffic. The capabilities of Nokia MSs are described in [2]. Some examples of the capabilities of the handsets suggested to be used in these tests are given in Table 5.

Note: the Mobile Call Generator used in KPI testing supports only certain MSs which may differ from the recommedations given above. Currently only N6200 and 6230 are supported for PING generation in the MCG environment.


NSN/RA/EDGE SFP


Table 5. Capabilities of some handsets

MS Model

GPRS EGPRS

Freq (MHz)Down Up Down Up

3 2 3 2

Nokia N80(Rel4)

? ? 4 1 GSM: 850 / 900 / 1800 / 1900WCDMA 21002 3

Nokia N93(Rel4)EDA/HMC, DTM support

5 1 5 1 GSM: 900 / 1800 / 1900WCDMA 21003 2 4 2

3 3 3 3

Nokia 6230

(Rel99)

4 1 4 1 900 / 1800 / 1900

3 2 3 2

3 2 3 2

Other handsets (also from other manufacturers) may be used in some test phases, these include, for example, the following:

EQoS non-aware MSs. See [2] for details.

Sony Ericsson TEMS (1800)

NEMO

MSs used in IOT

Other suitable MSs from other vendors

2.8.1 Connection between MS and laptop PC

In the test cases where a (laptop) PC computer is connected to the MS, the cable connection shall be used (i.e. not IrDA or Bluetooth as these may add a bottleneck due to limited speed).

There are performance differences between these cables, e.g. DKU-2 gives better throughput compared to DKU-5. DKU-2 is usable only with handsets having an internal modem

The recommended cable type is Nokia Connectivity Cable DKU-2 or CA-53


NSN/RA/EDGE SFP


2.8.2 Recommended OS for the laptop PC

For these tests a Basic Windows XP installation without any extras (e.g. not the Nokia WinXP image) shall be used as the operating system of the client PC.

Check that the PC does not contain e.g. the following applications potentially reducing performance:

PointSec or other laptop/mobile device security applications

Hard disk encryption software

Any applications having automatic update or similar features producing spurious traffic

Any unnecessary applications reducing the performance

Other issues to be checked:

Turn off the “Client for Microsoft Networking”

Turn off the “File and Print Sharing”

If an USB-cable is used between the mobile handset and the PC, check that there is no other devices connected to the USB.

The PC used in testing should be as fast as possible

Check that the TCP parameters (see Chapter 2.8.3) are not altered e.g. when installing the drivers for MS connection.


NSN/RA/EDGE SFP


2.8.3 Windows XP TCP parameters

The default set of Windows XP TCP parameters shall be used. The optimised parameter values [4] are given for reference in Table 6. See also [17 ] for other suggestions.

Table 6 Windows XP TCP parameters

Parameter Registry key Default (optimised values)

TCP Receive Window Size

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters:GlobalMaxTcpWindowSize

16384 32768

(16384 for EUTM)

TCP Receive Window Size

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters:TcpWindowSize

16384 32768

(16384 for EUTM)

SACK HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters:SackOpts

0 1

Time Stamp

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters:Tcp1323Opts

0 2

MTU HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\ {Interface ID}:MTU

576

(or automatic detection)

1460


NSN/RA/EDGE SFP


3 Traffic considerations

3.1 Applications and Services

The test items currently include the following applications and services:

HTTP Browsing (interactive)

FTP download (background)

FTP upload (background)

To verify the performance of EQoS, also other applications shall be included:

Streaming services:

Audio streaming (downstream)

Video + audio streaming (downstream and upstream)

Interactive services:

WAP Browsing

PoC

Background services

Email download, mail box synchronizing

Introduction of DTM brings new services to be considered for testing:

SWIS

Push-email


NSN/RA/EDGE SFP


3.2 Traffic Mix

The test items may in the future include cases modelling the traffic mix in the live networks The following traffic mixes is proposed [5] for PCU PET testing:

1.Streaming (20%), web services (20%), messages (60%)2.Streaming (20%), web services (40%), messages (40%)3.

Streaming (50%), web services (20%), messages (30%)


NSN/RA/EDGE SFP


4 EDGE Key Performance Indicators

The recommended set of EDGE and GPRS KPIs are described in the document “EDGE and GPRS Key Performance Indicators” [3], which is available in NOLS.

The KPI validation is not within the scope of this test specification , i.e. the EDGE KPI validation process shall be defined separately. Anyhow, some of the test results may be used for KPI validation, and therefore the values collected from KPIs when performing the test items shall be included in the test reports.


https://www.online.nokia.com/nols/doccenter/dcDownload/download?fileidlist=0900cc92800a6cd0&library=a

NSN/RA/EDGE SFP


5 Test Items

5.1 Quality : Throughput

The following test cases are related to Quality issues from the throughput point of view. The features that may have an impact on throughput shall be taken into account. Critical features are, for example: CS/3/CS4 (enabled), EQoS (disabled), EUTM (enabled), NACC (disabled) and NCCR (disabled), EDA/HMC (enabled).

5.1.1 (E)GPRS UL/DL throughput per TSL in optimal radio conditions

Description: Purpose of this test item is to evaluate the maximum (E)GPRS UL and DL throughput , while the radio conditions (signal strength, C/I ratio) are kept as good as possible.

In additional cases the maximum throughput can be measured with with different values of the parameters and settings having an impact on (E)GPRS throughput.

The throughput values are given for one TSL. If multislot transfer is used in the testing the throughput is given per one TSL and the number of TSLs used in the test shall be mentioned in the test report. For GPRS tests CS3/CS4 support shall be activated.

The changes in the number of timeslots and CS/MCSs used shall be monitored during the tests and reported in test reports.


NSN/RA/EDGE SFP


In the test cases for Extended Cell, the targeted RX Levels and C/I are assumed to be good, considering the path loss due to extended distance from the cell, but without fading. See Table 7.

Table 7. Radio path for throughput measurements in optimal radio conditions and for Extended Cell cases.

Normal Area (0-35 km)“optimal RF”

Extended Area (35-70 km)

Target RX level (downlink, measured with the terminal Field Test Display

- 50 dBm - 70 dBm

Target C/I ratio 30 dB 15 dB

Propagation model No fading, line-of-sight

The optimised values shown in Table 4 for the parameters related extended UL TBF to shall be used.

Note: As the optimal radio conditions may be difficult to obtain in certain test environments (e.g. SyVe, drive tests) , the figures above may not be reached. In these situations, “as good as possible” radio environment may be used. This means as good C/I ratio and RX level as is reasonably feasible with a stationary mobile terminal.

The test is performed with a FTP application, i.e. a test file is downloaded/uploaded from the test server and throughput is measured. Suggested test files to be used are a 5 Mbyte random binary file for downlink tests and a 2 Mbyte random binary file for uplink tests.

These are available from the test server as:

Downlink : ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/dl/5Mbyte.bin

Uplink : ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/ul/2Mbyte.bin

If other test files or file sizes are used these shall be mentioned in the test report.




NSN/RA/EDGE SFP


Basic test cases (Case ID):

EGPRS Cases:

EGPRS UL/DL Throughput (FTP) per unshared, dedicated TSL in optimal radio conditions; without frequency hopping; using Gb on FR. The throughput is measured from data transfer using FTP application and the optimised set of RTT parameters. The test shall be performed both for PCU1 and PCU2 units (depending on the test phase, in some environments only PCU2 may be available. See the Test Matrix for reference) (EPT5111)

EGPRS UL/DL Throughput (FTP) in optimal radio conditions, 2 MSs sharing the same TSLs (EPT5112)

EGPRS UL/DL Throughput (FTP) in optimal radio conditions using Gb over IP (IPv4) (EPT5116)

EGPRS UL/DL throughput (FTP) per TSL in Extended Cell (Ext.Area) (EPT5190)

EGPRS UL/DL throughput (FTP) per TSL with Abis over IP (EPT5191)

EGPRS UL/DL throughput (FTP) per TSL with Abis optimization(EPT5192)

GPRS Cases:

GPRS UL and DL Throughput per TSL (FTP) in optimal radio conditions on dedicated TSL (EPT5114)

GPRS Cases:

GPRS UL/DL throughput (FTP) per TSL in Extended Cell (Ext.Area) (EPT5193)

GPRS UL/DL throughput (FTP) per TSL with Abis over IP (EPT5194)

GPRS UL/DL throughput (FTP) per TSL with Abis optimization(EPT5195)

Related KPIs:

Average Effective UL Throughput per used TSLs in kbit/s (trf_72h)

Average Effective DL Throughput per used TSLs in kbit/s (trf_73g)


http://nea.ntc.nokia.com/formula.php?ne=BSC&kpi=trf_73g

http://nea.ntc.nokia.com/formula.php?ne=BSC&formula_id=trf_72h

NSN/RA/EDGE SFP


Result Format: throughput per TSL in kbit/s, possible varying parameter values and used applications to be described in the results.

5.1.2 (E)GPRS UL/DL throughput per TSL in varying radio conditions

Description: Purpose of this item is to assess the impact of different radio conditions to the (E)GPRS throughput.

Link adaptation is enabled in these tests, i.e. CS/MCS varies according to the C/I. For GPRS test CS3/CS4 support shall be activated. RX diversity shall be enabled. STIRC shall be disabled.

Signal Strength (RX Level)

Target range: -50 … -110 dBm, measured in DL using the MS Field Test Display

While the throughput remains in good level the measurements may be taken e.g. with steps of 5 dBm and with smaller steps (1 dBm) within the range where the throughput starts to deteriorate.

Throughput shall be measured at least with the following signal levels:

Good: RX Level = -70 dBm

Typical: RX Level = -90 dBm (indoors), -80 dBm (outdoors)

Cell Border: RX Level = -100 dBm (level typical for cell border for voice traffic)

Interference (C/I):

Target range: 0 ... 40 db

Depending on the test phase the interference (C/I) situation is either generated using the signal generator or collected from the MS test log or statistics. Because C/I depends on the RX Level it may not be possible to keep the C/I stabile in the real network environments. In the test phases where the C/I can be controlled, the target level for C/I is ~30 dB for measurements where RX Level is varied. Note: Because C/I values may not be available in all test phases, the values may be given as EGPRS UL/DL Throughput per TSL vs. BLER instead.

Throughput shall be measured at least with the following C/I levels:

Optimal: C/I = 30 dB (or at level above which the throughput does not increase)

“turning point” C/I = 20 dB

Typical: C/I = 15 dB


NSN/RA/EDGE SFP


Cell Border: C/I = 10 dB (level typical for cell border for voice traffic)

Low extreme; “signalling threshold” C/I = 5 dB

Note: Because the C/I values may not be available in all test phases, the results may be given as (E)GPRS Throughput per TSL vs. BLER instead.

Note: the recommended power level calibration procedure is described in Appendix E.

Fading:

Throughput is measured with the following fading profiles:

Stationary (no fading)

TU3, RX Level within -50…-110 dBm

TU50, RX Level within -50 … -110 dBm

The throughput results are given for one TSL. Anyhow, multislot transfer with the maximum number of TSLs supported by the MSs is used in the tests (e.g. 4 UL TSLs and 2 DL TSLs with Nokia 7270). The number of TSLs used in the tests shall be mentioned in the test report.

The test is performed with a FTP application , i.e. a test file is downloaded/uploaded from the test server and throughput is measured.

For EGPRS the suggested test files to be used are a 5 Mbyte random binary file for downlink tests and a 2 Mbyte random binary file for uplink tests. These test files are available from the test server as follows:

EGPRS Downlink : ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/dl/5Mbyte.bin

EGPRS Uplink : ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/ul/2Mbyte.bin

For GPRS the suggested test files to be used are a 1 Mbyte random binary file for downlink tests and a 500 kbyte random binary file for uplink tests. These test files are available from the test server as follows:

GPRS Downlink : ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/dl/1Mbyte.bin

GPRS Uplink : ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/ul/500kbyte.bin








NSN/RA/EDGE SFP


EGPRS Cases:

EGPRS UL and DL Throughput (FTP), when RX Level = -50 … -110 dBm with 5 db steps. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5126)

o EGPRS UL and DL Throughput (FTP), when RX Level = -70 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5126a)

o EGPRS UL and DL Throughput (FTP), when RX Level = -90 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5126b)

o EGPRS UL and DL Throughput (FTP), when RX Level = -100 dBm. No fading (stationary MS), No frequency hopping. FTP transfer (EPT5126c).

EGPRS UL and DL Throughput (FTP), when C/I = 0 … 40 dB with 5 db steps. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5127)

EGPRS UL and DL Throughput (FTP) in TU3. RX Level varies within range -50 … -100 dBm (measured) and C/I varies within range 0 … 40 dB (measured). FTP transfer, no frequency hopping. (EPT5121)

EGPRS UL and DL Throughput (FTP) in TU50. RX Level varies within range -50 … -100 dBm (measured) and C/I varies within range 0 … 40 dB (measured). FTP transfer, no frequency hopping. (EPT5122)

EGPRS UL and DL Throughput (FTP) with Gb/IP (IPv4) when RX Level = -50 … -110 dBm with 5 db steps. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5120)

o EGPRS UL and DL throughput (FTP) with Gb/IP (IPv4) when RX Level = -70 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5120a)

o EGPRS UL and DL throughput (FTP) with Gb/IP (IPv4) when RX Level = -90 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5120b)

o EGPRS UL and DL throughput with Gb/IP (IPv4) when RX Level = -100 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5120c)

GPRS Cases:


NSN/RA/EDGE SFP


GPRS UL and DL Throughput, when RX Level = -50 … -110 dBm with 5 db steps. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5128)

o GPRS UL and DL Throughput, when RX Level = -70 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5128a)

o GPRS UL and DL Throughput, when RX Level = -90 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5128b)

o GPRS UL and DL Throughput, when RX Level = -100 dBm. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5128c)

GPRS UL and DL Throughput, when C/I = 0 … 40 dB with 5 db steps. No fading (stationary MS), No frequency hopping. FTP transfer. (EPT5129)

GPRS UL and DL Throughput in TU3. RX Level varies within range -50 … -100 dBm (measured) and C/I varies within range 0 … 40 dB (measured). FTP transfer, no frequency hopping. (EPT5123)

GPRS UL and DL Throughput in TU50. RX Level varies within range -50 … -100 dBm (measured) and C/I varies within range 0 … 40 dB (measured). FTP transfer, no frequency hopping. (EPT5124)

Related KPIs:



Result format: throughput per TSL in kbit/s separately for different fading patterns. Applications and varying parameter values shall be described in the results. Changes in the number of timeslots and MCSs used to be described if the information is available.

5.1.3 MCS/CS Distribution vs. C/I (or BLER)

Description: The purpose of this item is to assess the impact of the interference to the MCS/CS used, i.e. the performance of the link adaptation algorithm. The distribution of MCSs/CSs is collected in several different C/I ratios, at least in the following: 10dB, 20dB, 30dB. The signal strength of the carrier is kept on a relatively good level, e.g. -




NSN/RA/EDGE SFP


70 dBm (RX level measured from the receiving end) while the interfering signal strength is varied, e.g. starting from 30 dB.

In lab environment the C/I situation is caused using the signal generator. Adaptive Link Adaptation (PCU2 only) or “normal” Link Adaptation (PCU1) are enabled in these tests, i.e. CS/MCS varies according to the C/I. RX diversity shall be enabled. STIRC shall be disabled.

The distribution is measured both in uplink and in downlink, using a FTP application.

Suggested test files to be used are the same as used in the throughput cases. These are available from the test server as follows:

EGPRS DL: ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/dl/5Mbyte.bin

EGPRS UL: ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/ul/2Mbyte.bin

GPRS DL: ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/dl/1Mbyte.bin

GPRS UL: ftp://10.16.62.243/~sopro/EDGE_Test/ (Test Phase) /ftp/ul/500kbyte.bin


For GPRS also CS-3 and CS-4 shall be enabled.

Because C/I values may not be available in all test phases, the values may be given as MCS/CS distribution vs. BLER instead. Note: the distribution results may be collected within the test cases for (E)GPRS Throughput., i.e. the C/I range shall be similar as in those tests.

Note: the recommended power level calibration procedure is described in Appendix E.


EGPRS: MCS distribution vs. C/I, with LA (EPT5131)

GPRS: CS distribution vs. C/I, with (Adaptive) LA (EPT5132)

Related KPIs: RLC Block counters in Coding Scheme Measurement (79) can be used to collect the distribution information.

Result format: Percentage share of each MCS class for each of the measured C/I points. Alternatively, a scatter graph showing the samples on MCS vs. C/I scale.


http://nea.ntc.nokia.com/measurement.php?ne=BSC&table_name=p_nbsc_coding_scheme






NSN/RA/EDGE SFP


5.1.4 MCS/CS distribution vs. RX Level

Description: The purpose of this item is to assess the impact of the signal strength to the MCS/CS used, i.e. the performance of the link adaptation algorithm. The distribution of MCSs/CSs is collected with several different RX Levels, at least in the following: -110dBm, -80dBm, -50dBm. The signals are adjusted to these levels using attenuators in the lab environment. Adaptive Link Adaptation (PCU2 only) or “normal” Link Adaptation (PCU1) are enabled in these tests, i.e. CS/MCS varies according to the C/I. C/I shall be kept in relatively good (~30 dB) level, i.e. no generated interference is injected to the scenario. RX diversity shall be enabled. STIRC shall be disabled.









Note: the distribution results may be collected within the test cases for (E)GPRS Throughput., i.e. the RX Level range shall be similar as in those tests. RX Level is measured in DL using the MS Field Test Display.


EGPRS: MCS distribution vs. RX Level, with LA (EPT5141)

GPRS: CS distribution vs. RX Level, with (Adaptive) LA (EPT5142)


Result format: Percentage share of each MCS class for each of the measured signal strength point. Alternatively, a scatter graph showing the samples on MCS vs. signal strength may be given.








NSN/RA/EDGE SFP


5.1.5 MCS/CS distribution vs. fading

Description: The purpose of this item is to assess the impact of fading in radio path to the MCS/CS used, i.e. the performance of the link adaptation algorithm.

The average signal strength is kept in these cases on “good” level, e.g. ~80 dBm, if possible or at least within -70…-90 dBm (variation of RX Level to be measured and included in the test results).

Adaptive Link Adaptation (PCU2 only) or “normal” Link Adaptation (PCU1) are enabled in these tests. RX diversity shall be enabled. STIRC shall be disabled.









Note: the distribution results may be collected within the test cases for (E)GPRS Throughput.


EGPRS: MCS distribution in TU3, with LA (EPT5133)

EGPRS: MCS distribution in TU50, with LA (EPT5134)

GPRS: CS distribution in TU3, with (Adaptive) LA (EPT5135)

GPRS: CS distribution in TU50, with (Adaptive) LA (EPT5136)









NSN/RA/EDGE SFP


Result format: Percentage share of each MCS/Cs class for each of the measured fading model.

5.2 Quality : RTT and latency

The following test cases are related to Quality issues from the round-trip-time (RTT) and latency point of view. The features that may have an impact on RTT shall be taken into account. Critical features are, for example: EQoS (disabled), EUTM (enabled), NACC (disabled) and NCCR (disabled), EDA/HMC (disabled).

5.2.1 Active RTT

Description: Active RTT means end-to-end roundtrip time measured from repeated ICMP messages excluding the first result and using short period (<1s) to ensure that delayed DL TBF and Extended UL TBF mechanisms are operating. [6]

The test shall be performed using the PING application with different packet sizes (32, 256, 1460 bytes). Test is repeated at least 50 times for each packet size, the final value is the average over these. The first ping (not actually representing Active RTT) shall be left out from the calculated average result. The interval between the pings is according to the default values of the used test application.



NSN/RA/EDGE SFP


Table 8. Radio path for RTT measurements in optimal radio conditons and for the Extended Cell cases.

Normal Area (0-35 km)“optimal RF”

Extended Area (35-70 km)

Target RX level (downlink, measured with the terminal Field Test Display

- 50 dBm - 70 dBm

Target C/I ratio 30 dB 15 dB

Propagation model No fading, line-of-sight

The default values shown in Table 4 for the parameters related to extended UL TBF shall be used.

Note: if combined BCCH (MBCCHC) is used in the test scenario, this shall be mentioned with the results.

The test shall be performed both for PCU1 and PCU2 units.


EGPRS Cases:

EGPRS Active RTT on dedicated TSL in optimal radio conditions. (EPT5211)

EGPRS Active RTT while C/I = ~0 … 40 dB on dedicated TSL (EPT5218)

EGPRS Active RTT while RX Level = -50 … -110 dBm on dedicated TSL (EPT5217)

o EGPRS Active RTT while RX Level = -70 dBm on dedicated TSL (EPT5217a)

o EGPRS Active RTT while RX Level = -90 dBm on dedicated TSL (EPT5217b)

o EGPRS Active RTT while RX Level = -100 dBm on dedicated TSL (EPT5217c)

EGPRS Active RTT while MS is on Extended Area (Ext.Cell)(EPT5291)

EGPRS Active RTT with Abis over IP(EPT5292)

EGPRS Active RTT with Abis optimization(EPT5293)


NSN/RA/EDGE SFP


GPRS Cases:

GPRS Active RTT on dedicated TSL in optimal radio conditions (EPT5210)

GPRS Active RTT on dedicated TSL , while C/I = ~0 … 40 dB, 5 dB steps (EPT5214)

GPRS Active RTT on dedicated TSL , while RX Level = -50 … -110 dBm, 5 dB steps (EPT5219)

GPRS Active RTT while MS is on Extended Area (Ext.Cell)(EPT5294)

GPRS Active RTT with Abis over IP(EPT5295)

GPRS Active RTT with Abis optimization(EPT5296)

Related KPIs: none

Result format: The Active RTT measurement is repeated at least 50 times for each different packet size. The following figures (in milliseconds) are calculated from the results:

Average Active RTT

95% percentile of Active RTT

Minimum and maximum values for Active RTT

Standard deviation of Active RTT

Success Rate

5.2.2 Idle RTT

Description: Idle RTT means individual end-to-end roundtrip measured by single ICMP message or repeated with long period (>5s) to ensure that UL and DL TBF need to be established every time. [6]

The test shall be performed using the PING application with different packet sizes (32, 256, 1460 bytes). Test is repeated at least 50 times for each packet size, the final value is the average over these. The interval between the pings is according to the default values of the used test application.



NSN/RA/EDGE SFP


Note: if combined BCCH (MBCCHC) is used in the test scenario, this shall be mentioned with the results.

The test shall be performed both for PCU1 and PCU2 units.


EGPRS Cases:

EGPRS Idle RTT on dedicated TSL in optimal radio conditions (EPT5221)

EGPRS Idle RTT while C/I = ~0 … 40 dB on dedicated TSL (EPT5227)

EGPRS Idle RTT while RX Level = -50 … -110 dBm on dedicated TSL (EPT5228)

o EGPRS Idle RTT while RX Level = -70 dBm on dedicated TSL (EPT5228a)

o EGPRS Idle RTT while RX Level = -90 dBm on dedicated TSL (EPT5228b)

o EGPRS Idle RTT while RX Level = -100 dBm on dedicated TSL (EPT5228c)

EGPRS Idle RTT while MS is on Extended Area (Ext.Cell) (EPT5297)

EGPRS Idle RTT with Abis over IP(EPT5298)

EGPRS Idle RTT with Abis optimization(EPT5299)

GPRS Cases:

GPRS Idle RTT on dedicated TSL in optimal radio conditions (EPT5220)

GPRS Idle RTT on dedicated TSL , while RX Level = -50 … -110 dBm, 5 dB steps (EPT5229)

GPRS Idle RTT on dedicated TSL , while C/I = ~0 … 40 dB, 5 dB steps (EPT5224)

GPRS Idle RTT while MS is on Extended Area (Ext.Cell)(EPT5290

GPRS Idle RTT with Abis over IP(EPT5289)


NSN/RA/EDGE SFP


GPRS Idle RTT with Abis optimization(EPT5288)

Related KPIs: none

Result format: The Idle RTT measurement is repeated at least 50 times for each different packet size. The following figures (in milliseconds) are calculated from the results:

Average Idle RTT

95% percentile of Idle RTT

Minimum and maximum values for Idle RTT

Standard deviation of Idle RTT

Success Rate


NSN/RA/EDGE SFP


5.3 Quality : User Experience

5.3.1 HTTP User Experience (4 seconds)

Description: According to ITU G.1010, a web page should be downloadable within 2 seconds. 3GPP suggests 4 seconds, as acceptance level for interactive services, i.e. majority of the users expect for example the web page to download within 4 seconds.

Purpose of this test is to verify the impact of BSS delays to the amount of DL user data that can be download within 4 seconds from the start of the HTTP download This can be accomplished by measuring the amount of data transferred over HTTP in 4 seconds, or by trying to download files of different sizes (0..200kB) and see what is the maximum file size that is completed in the 4 second window. Test is performed with default values of the parameters having impact on RTT. See Chapter 2.5

The number of timeslots used in test should be typical supported by current terminals (currently; 4+1 is suggested)

Test is performed with both the default values as well as different other values within the allowed range. The test is repeated in 12 seconds period so that 12 seconds is waited after earlier transfers is stopped. This models the average reading time of http user. If the test is performed in controlled lab environment, the optimal radio conditions are used.

Target RX Level for the test : -50 dBm. RX Level is measured in DL using the MS Field Test Display

Target C/I ratio for the test: 30 dB

PDP context shall be activated before the download (i.e. it is not included in the 4 s.)

The measurement is done with the Application Tester.

Test page to be used: http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/200kB_1.html

Note: there is a specific directory for each test phase, so “EDGE_SFP” in the URL given above shall be replaced with the test phase. See Figure 2 for details.

Basic test case (Case ID):

HTTP User Experience (4 seconds) test in optimal radio conditions. (EPT5311)

Related KPIs: none


http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/200kB_1.html

NSN/RA/EDGE SFP


Result format: A graph showing the cumulative amount of downloaded data during 0..4 seconds from start. Values measured at 2, 4 and 6 seconds from the start to be reported. The used parameter values shall be included in the test results.

5.3.2 FTP file download and upload duration

Description: Files are downloaded and uploaded using FTP. Duration of download/upload is measured. The downloading/uploading is repeated 20 times, the final value is the average over these. If the test is performed in controlled lab environment, the optimal radio conditions are used.



The following test files shall be used from the dedicated test server:

EGPRS DL (5 MB random binary file):ftp://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/ftp/dl/5MByte.bin

EGPRS UL (2 MB random binary file):ftp://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/ftp/ul/2MByte.bin

Note: there is a specific directory for each test phase, so “EDGE_SFP” in the URL given above shall be replaced with the test phase. See Figure 2 for details.


FTP file download duration with 5 Mbyte file (EPT5321)

FTP file download duration with 5 Mbyte file with Gb/IP (EPT5323)

FTP file upload duration with 2 Mbyte file (EPT5322)

FTP file upload duration with 2 Mbyte file with Gb/IP (EPT5324)

Related KPIs: none

Result format: Download duration in milliseconds, calculated as average over 20 repetitions. Size and name of the test file to be included in the report.

5.3.3 HTTP web page download duration

Description: Web pages of different sizes are downloaded using a browser. The following web page sizes are used:




NSN/RA/EDGE SFP


- 10 kByte page with only html-text, - 100 kByte page with html text and one 50 kB picture, - 1000 kByte page consisting of ten 100 kB pictures and some html text.Each page is downloaded ten times and the duration of download for each page size is calculated as an average over these downloads. If the test is performed in controlled lab environment, the optimal radio conditions are used.



Test pages are available in the following addresses (only from NOKLAB network):

Main page of test pages, links to different test pages:http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/testpages.html

1 Mbyte test page:http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/1MB.html

Note: there is a specific directory for each test phase, so “EDGE_SFP” in the URL given above shall be replaced with the test phase. See Figure2 for details


HTTP download duration with 1 megabyte webpage in optimal radio conditions (EPT5331)

Related KPIs: none

Result format: Download duration in milliseconds for each different page size, calculated as average over 10 repetitions.

5.3.4 PDP Context Activation Duration

Description: The purpose of this test is to validate the duration of the process when subscriber activates a new PDP Context. The duration is measured between the time when BSC receives the PDP context activation request and the time when BSC replies on the Abis. Note that the duration experienced by the MS is at least 180 ms longer due to delays in Abis and Air interfaces in both directions (2 * 90 ms).

In these tests the duration is measured from the Abis interface using e.g. NetHawk as the time difference between the messages described in Table9.


http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/1MB.html

http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/testpages.html

http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/testpages.html

NSN/RA/EDGE SFP


Table 9. Messages related to PDP context activation

Measurement begins (from MS) Measurement ends (to MS)

ACT. PDP CONTEXT REQUEST (GPRS SM)

ACT. PDP CONTEXT ACCEPT (GPRS SM)

As this comes after the BSN1, the Packet DL Ack/Nack acknowledging the BSN1 shall be found. Then a BSN1 transmission which has occurred at least 180 ms before the acknowledgement shall be found. Timestamp of this message containing BSN1 is used as the ending point of the PDP context activation duration.

The end-to-end duration is measured from the MS.


Duration of PDP context activation performed by subscriber (EPT5341)

Related KPIs: Not currently available, counters coming in SG6 to SGSN.

Result format: PDP Context activation duration in milliseconds, two separate results:

o BSC share of the duration, measured from the Abis traces

o End-to-end duration measured from the MS

5.3.5 GPRS Attach Duration

Description: The purpose of this test is to validate the duration of the process when subscriber attaches to GPRS service. Two different aspects shall be measured, i.e. end-to-end duration and BSC (PCU) share of the duration.

The BSC share of the duration is measured from the Abis interface using e.g. NetHawk as the time difference between the messages described in ..

Table 10. Messages related to GPRS Attach


NSN/RA/EDGE SFP



ATTACH REQUEST (GPRS MM) ATTACH ACCEPT (GPRS MM)

Reception of the ACCEPT message shall be confirmed by the following Packet DL Ack/Nack message



Duration of GPRS Attach performed by subscriber (EPT5361)


Result format: GPRS Attach duration in milliseconds, two separate results:



.


NSN/RA/EDGE SFP


5.4 Quality : EGPRS Cell changes and reselections

5.4.1 EGPRS Cell Change Duration (NCCR and NACC)

Description: Purpose of this test is to verify the duration of discontinuity of UL and DL data transfer when the MS performs a cell change (either autonomous or network controlled). Each test case is repeated 10 times and the duration is calculated as average of each repetition. According to 3GPP TR 44.901 [11], the Service Outage Time in cell change is the time between the last received uplink RLC block from the mobile station in the old cell and the mobile station's first uplink RLC block received in the new cell.

Different ways to measure the service outage are shown in Figure 3.

Figure 3. Signalling with Cell Changes


Application Outage

EGPRS Packet Downlink Ack/Nack (source cell)

MS GERAN

First IP packets

Last IP packets

DataOutage

CellOutage

EGPRS Packet Downlink Ack/Nack (target cell)

Packet Uplink Ack/Nack (target cell)Packet control acknowledgement

Immediate assignment & Packet serving cell data

Application Outage

EGPRS Packet Downlink Ack/Nack (source cell)

MS GERAN

First IP packets

Last IP packets

DataOutage

CellOutage

EGPRS Packet Downlink Ack/Nack (target cell)

Packet Uplink Ack/Nack (target cell)Packet control acknowledgement

Immediate assignment & Packet serving cell data

NSN/RA/EDGE SFP


Terminology:

Cell outage:In one-phase access: the time between the last EGPRS Packet Downlink Ack/Nack message and the first Packet Uplink Ack/Nack.

Data outage: the time between the last and the first EGPRS Packet Downlink Ack/Nack message.

Application outage: the time between the last and the first successfully received FTP-packet

See [18] for details.

In these tests the, application level outage shall be measured, if possible. This is done by monitoring the Abis interface e.g. with the NetHawk tool and the outage time is calculated from the occurrence of the last user data frame on the old cell and the first user data frame in the new cell.

If this is not feasible, them the data outage can be measured instead, measuring the signalling messages representing the beginning or end of the service outage.

For downlink data transfer, the cell change durations are measured between the occurrences of the messages given in the following in Table 11, depending on the set of features activated. As some of these messages are used also in other context than in cell changes, the occurrence of cell change may be detected e.g. by finding the Flush-II messages from the Gb-interface monitoring. The respective messages related to cell change in UL data transfer are given in Table 12.

Table 11 Messages related to cell changes in DL data transfer.

NACC NCCR Measurement begins (old cell)

Measurement ends (new cell)

No No EGPRS Packet Downlink ACK/NACK

EGPRS Packet Downlink ACK/NACK (after the downlink assignment)

No Yes EGPRS Packet Downlink ACK/NACK (after Packet Cell Change Order)


Yes No EGPRS Packet Downlink ACK/NACK


Yes Yes EGPRS Packet Downlink ACK/NACK (after Packet Cell Change Order)

EGPRS Packet Downlink ACK/NACK (after the downlink


NSN/RA/EDGE SFP


assignment)

Table 12 Messages related to cell changes in UL data transfer.

NACC NCCR Measurement begins (old cell)

Measurement ends (new cell)

No No Packet Uplink ACK/NACK Packet Uplink ACK/NACK (after Packet Channel Request and Immediate Assignment)

No Yes Packet Uplink ACK/NACK (preceding a Packet Cell Change Order)

Packet Uplink ACK/NACK (after Packet Immediate Assignment)

Yes No Packet Uplink ACK/NACK Packet Uplink ACK/NACK (after Packet Channel Request and Immediate Assignment)

Yes Yes Packet Uplink ACK/NACK (after a Packet Cell Change Order)

Packet Uplink ACK/NACK (after Packet Immediate Assignment)

Note: The default set of NCCR and NACC parameters shall be used, unless otherwise stated in some test cases.


EGPRS Cell Change Duration within one SGSN, different RAs (RA update); intra-PCU, without NACC and NCCR (EPT5410)

EGPRS Cell Change Duration between SGSNs (inter-SGSN, RA update) without NACC and NCCR (Not possible in BSS SyVe) (EPT5412)

EGPRS Cell Change Duration; intra-PCU; without NACC and NCCR (EPT5413)

EGPRS Cell Change Duration; inter-PCU; without NACC and NCCR (EPT5414)

EGPRS Cell Change Duration; intra-PCU; within one Routing Area (cell update); with NACC (EPT5415)


NSN/RA/EDGE SFP


EGPRS Cell Change Duration; intra-PCU; within one Routing Area (cell update); with NCCR (EPT5416)

EGPRS Cell Change Duration; intra-PCU within one Routing Area (cell update), with both NACC and NCCR (EPT5401)

EGPRS Cell Change Duration; intra-PCU; combining both RA and LA update with NMOI, without NACC and without NCCR (EPT5418)

EGPRS Cell Change Duration; intra-PCU combining both RA and LA update with NMOII, without NACC and without NCCR (EPT5419)

EGPRS Cell Change Duration, inter-PCU RAU, without NACC and without NCCR (EPT5411)

Related KPIs: none

Result format: Cell change duration in milliseconds calculated as average over 10 repetitions. The measurement method (application outage or data outage) shall be described with the results. The information on activated features (NACC, NCCR) to be included in the test results.

5.5 EDGE-WCDMA Interworking

The following test items are related to interworking between EDGE and WCDMA networks. These tests may be done in the Nokia Test Network.

5.5.1 EDGE-WCDMA Intersystem Cell Change Duration

Description: Purpose of this test is to verify the duration of discontinuity of UL and DL data transfer when the MS performs a cell change between EDGE and WCDMA systems. The duration is measured from the time difference between the last data packet sent on the source BTS and the first data packet sent on the target BTS. Intersystem cell change duration for both directions shall be measured.


NSN/RA/EDGE SFP


Table 13. Measurement points for EDGE/WCDMA cell changes

Measurement starts Measurement ends

EDGE to WCDMA 2G Abis last Packet DL Ack/Nack Iub first DCH DL DATA FRAME

WCDMA to EDGE Iub last DCH DL DATA FRAME 2G Abis first Packet DL Ack/Nack

Open issue: the measurement points in Table 13 to be verified!


EDGE to WCDMA Cell Change Duration, without NACC and without NCCR (EPT5511)

WCDMA to EDGE Cell Change Duration, without NACC and without NCCR (EPT5512)

Related KPIs: none

Result format: Cell change duration in milliseconds. The information on activated features to be included in the test results.


NSN/RA/EDGE SFP


5.6 EQoS

5.6.1 EGPRS Cell Change Duration with EQoS

Description: Purpose of these test items is to validate the impact of EQoS to Cell Change Duration, i.d. duration of cell changes due to QC initiated by EQoS. These cases are similar to Cell Change Duration tests described in Chapter 5.4.1, but instead of FTP transfer the video streaming application is used (exept for one case where FTP is used to benchmark the impact of EQoS to similar case without EQoS).

NCCR shall be activated in these tests. The impact of NACC with EQoS may be tested as an additional test item.

Cell change duration shall be measured separately both for Uplink and Downlink

See Chapter 5.4.1 for descriptions of methods (Application Outage and Data Outage) to measure Cell Change Duration. See [18] for details.


EGPRS Cell Change (QC) Duration with EQoS, intra-PCU2, QoS aware MS, FTP (Background) Transfer. (EPT5611)

EGPRS Cell Change (QC) Duration with EQoS, intra-PCU2, non-QoS aware MS, Streaming video. (EPT5612)

EGPRS Cell Change (QC) Duration with EQoS, intra-PCU2, QoS aware MS, Streaming video. (EPT5613)

EGPRS Cell Change (QC) Duration with EQoS, inter-PCU2, QoS aware MS, Streaming video. (EPT5614)

Related KPIs: none

Result format: Cell change duration in milliseconds calculated as average over 10 repetitions. The measurement method (application outage or data outage) shall be described with the results.


NSN/RA/EDGE SFP


5.6.2 HTTP Download Duration with EQoS

Description: Purpose of this test is to verify the impact of EQoS to HTTP download (Interactive class).

Test page: http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/1MB.html (Espoo)


HTTP Download Duration with EQoS witg different distances from cell (or different C/I). (EPT5621)

Related KPIs: none

Result format: Download duration in milliseconds.

5.6.3 RTT with EQoS

Description: Purpose of this test item is to verify the impact of EQoS to Round Trip Times, using a EQoS-aware MS.

The test shall be performed using the PING application with different packet sizes (32, 256, 1460 bytes). PING application (or the MS) shall be mapped to the Streaming class. Test is repeated at least 50 times for each packet size, the final value is the average over these.

The measurements are done in good C/I (30 dB) on dedicated TSLs

These test cases are similar to the basic RTT cased described in the Chapters 5.2.1 (Active RTT) and 5.2.2 (Idle RTT).


EGPRS Active RTT with EQoS in optimal radio conditions on dedicated TSL (EPT5631)

EGPRS Idle RTT with EQoS in optimal radio conditions on dedicated TSL (EPT5632)

Related KPIs:

Result format: The RTT measurements are repeated at least 50 times for each different packet size (32, 256 and 1460 bytes). The following figures (in milliseconds) are calculated from the results:


http://10.16.62.243/~sopro/EDGE_Test/EDGE_SFP/http/1MB.html

NSN/RA/EDGE SFP


Average RTT

95% percentile of RTT

Minimum and maximum values for RTT

Standard deviation of RTT

Success Rate

5.7 DTM

5.7.1 DTM Setup

Description: Purpose of this test is to verify the duration of DTM setup process while a CS call is ongoing and user initiates a simultaneous PS connection (DTM MO) or a simultaneous PS connection is initiated by external service such as push email (DTM MT).

The measurement is done by sending a ping request during an ongoing CS call (causing a move to DTM mode) and measuring the resulting delay. The setup time is measured as time difference between the signalling points given in Table 14.

Case Measurement starts Measurement ends

CS call ongoing, DTM MO (ping request sent from MS)

DTM REQUEST PACKET UL ACK/NACK(or preceding UL MAC EGPRS header, both timestamps shall be collected)

CS call ongoing, DTM MT (ping request sent from nw to MS)

ECHO PACKET DL ACK NACK(or preceding DL MAC EGPRS header, both timestamps shall be collected)

Table 14. Signalling point between which the DTM setup time is measured



NSN/RA/EDGE SFP


CS call ongoing, DTM MO (ping request sent from MS) (EPT5711)

CS call ongoing, DTM MT (ping request sent from nw to MS) (EPT5712)

Related KPIs: none

Result format: Setup duration in milliseconds (average value over 20 setups)

5.7.2 DTM Transfer Break

Description: Purpose of this test is to verify the duration of a break in data transfer while the PS connection is moved to DTM mode and back to PS mode. The transfer break is measured as time difference between the signalling points given in Table 15

Case Measurement starts Measurement ends

PS DL transfer ongoing, MO CS call started, transfer continues on DTM mode.

EGPRS Packet Downlink Ack/Nack (before DTM REQUEST)

EGPRS Packet Downlink Ack/Nack (after ASSINGMENT COMPLETE)

PS UL Transfer ongoing, MO CS call started, transfer continues on DTM mode

Packet Uplink Ack/Nack (before DTM REQUEST)

Packet Uplink Ack/Nack (after ASSINGMENT COMPLETE)

DL Transfer ongoing on DTM mode, CS call ends, DL transfer continues on PS mode

EGPRS Packet Downlink Ack/Nack (before PACKET TBF RELEASE)

EGPRS Packet Downlink Ack/Nack (after PACKET DOWNLINK ASSIGNMENT)

UL Transfer ongoing on DTM mode, CS call ends, UL transfer continues on PS mode.

Packet Uplink Ack/Nack(before PACKET TBF RELEASE)

Packet Uplink Ack/Nack (after PACKET TBF RELEASE)

Table 15. Signalling point between which the DTM transfer break is measured


PS DL Transfer ongoing, MO CS call started, transfer continues on DTM mode. (EPT5721)


NSN/RA/EDGE SFP


PS UL Transfer ongoing, MO CS call started, transfer continues on DTM mode. (EPT5722)

DL Transfer ongoing on DTM mode, CS call ends, DL transfer continues on PS mode. (EPT5723)

UL Transfer ongoing on DTM mode, CS call ends, UL transfer continues on PS mode. (EPT5724)

Related KPIs: none

Result format: Break duration in milliseconds (average value over 20 setups)

5.7.3 DTM non-real time throughput

Description: Purpose of this test is to measure impact of DTM mode to the EGPRS nRT throughput. The throughput is measured separately in the following phases:

Before moving to the DTM mode

During DTM mode

The test is performed with a FTP application, i.e. a test file is downloaded/uploaded from the test server and throughput is measured.

Suggested test files to be used are a 5 Mbyte random binary file for downlink tests and a 2 Mbyte random binary file for uplink tests.






Non-real time UL/DL throughput before and during DTM mode. (EPT5731)

Related KPIs:








NSN/RA/EDGE SFP


Result Format: throughput per TSL in kbit/s, possible varying parameter values and used applications to be described in the results.

5.7.4 DTM RTT

Description: Purpose of this test item is to verify the impact of DTM (simultaneous CS call) to Round Trip Times of PS service.

The test shall be performed using the PING application with different packet sizes (32, 256, 1460 bytes). Test is repeated at least 50 times for each packet size, the final value is the average over these.




DTM Active RTT with simultaneous CS call in good C/I (30 dB) on dedicated TSL (EPT5741)

DTM Idle RTT with simultaneous CS call in good C/I (30 dB) on dedicated TSL (EPT5742)

Related KPIs: none


Average Active/Idle RTT

95% percentile of Active/Idle RTT

Minimum and maximum values for Active/Idle RTT

Standard deviation of Active/Idle RTT

Success Rate

5.7.5 DTM Cell Change Duration

Description: Purpose of these test items is to validate the impact of DTM to Cell Change Duration, i.d. duration of cell changes in DTM mode. These cases are similar to Cell Change Duration tests described in Chapter 5.4.1.


NSN/RA/EDGE SFP


As a default other feature such as NACC or NCCR are not activated in these tests. Anyhow, their impact can be validated with EQoS as additional test cases.

See Chapter 5.4.1 for descriptions of methods (Application Outage and Data Outage) to measure Cell Change Duration. See [18] for details.

In the Inter-PCU cases both PCUs reside in the same BSCU.


DTM Intra-BTS (reallocation) cell change duration, with CS call and PS DL data transfer. (EPT5751)

DTM Intra-cell (inter-BTS) cell change duration, with CS call and PS DL data transfer. (EPT5752)

DTM Intra-PCU, inter-RA cell change duration, with CS call and PS DL data transfer. (EPT5753)

DTM Intra-PCU, inter-RA cell change duration, with CS call and PS UL data transfer. (EPT5754)

DTM Inter-PCU, inter-RA cell change duration, with CS call and PS DL data transfer. (EPT5755)

DTM Intersystem (EDGE to WCDMA) cell change duration, with CS call and PS DL Transfer. (EPT5756)

DTM Intersystem (WCDMA to EDGE) cell change duration, with CS call and PS DL Transfer. (EPT5757)

Related KPIs: none

Result format: Cell change duration in milliseconds calculated as average over 10 repetitions. The measurement method (application outage or data outage) shall be described with the results.

5.7.6 DTM Coverage

Description: Purpose of this test case is to verify coverage offered by DTM service, i.e. ability to maintain CS voice quality and EGPRS throughput in different radio conditiona (signal strength).

The PS transfer in the test is performed with a FTP application, i.e. a test file is downloaded/uploaded from the test server and throughput is measured.


NSN/RA/EDGE SFP


The Voice Quality is measured as MOS or alternatively as BLER. The quality and throughput figures are collected both from the application and from the statistics.

Suggested test files to be used are a 5 Mbyte random binary file for downlink tests and a 2 Mbyte random binary file for uplink tests.





The test is performed using EGPRS, without fading, without frequency hopping. Target range for RX Level: -50 … -110 dBm. RX Level is measured in DL using the MS Field Test Display


PS UL Throughput vs. RX Level during simultaneous CS Voice Call. (EPT5783)

PS DL Throughput vs. During simultaneous CS Voice Call. (EPT5784)

Related KPIs: none

Result format: Voice Quality (MOS or BLER) and UL /DL throughput (kbit/s) as a function of signal strength (-50 … -110 dBm)

5.7.7 DTM GPRS Attach Duration

Description: The purpose of this test is to validate the impact of DTM to the duration of the process when subscriber attaches to GPRS service, i.e. the GPRS Attach is performed while a simultaneous CS Voice call is ongoing. The case is similar to the basic case decribed in Chapter 5.3.5

Two different aspects shall be measured, i.e. end-to-end duration and BSC (PCU) share of the duration.

The BSC share of the duration is measured from the Abis interface using e.g. NetHawk as the time difference between the messages described in Table 16.




NSN/RA/EDGE SFP


Table 16. Messages related to GPRS Attach


ATTACH REQUEST (GPRS MM) ATTACH ACCEPT (GPRS MM)

Reception of the ACCEPT message shall be confirmed by the following Packet DL Ack/Nack message



GPRS Attach Duration with DTM. (EPT5791)


Result format: GPRS Attach duration in milliseconds, two separate results:



5.7.8 DTM PDP Context Activation Duration

Description: The purpose of this test is to validate the impact of DTM to the duration of the process when subscriber activates a new PDP Context. The case is similar to the basic case described in Chapter 5.3.4

The duration is measured from the time difference between the time when BSC receives the PDP context activation request and the time when BSC replies on the Abis. Note that the duration experienced by the MS is at least 180 ms longer due to delays in Abis and Air interfaces in both directions (2 * 90 ms).

In these tests the duration is measured from the Abis interface using e.g. NetHawk as the time difference between the messages described in Table17


NSN/RA/EDGE SFP


Table 17. Messages related to PDP context activation


ACT. PDP CONTEXT REQUEST (GPRS SM)

ACT. PDP CONTEXT ACCEPT (GPRS SM)

As this comes after the BSN1, the Packet DL Ack/Nack acknowledging the BSN1 shall be found. Then a BSN1 transmission which has occurred at least 180 ms before the acknowledgement shall be found. Timestamp of this message containing BSN1 is used as the ending point of the PDP context activation duration.



PDP Context Activation Duration with DTM. (EPT5701)


Result format: PDP Context activation duration in milliseconds.

5.8 EDA/HMC

5.8.1 EDA/HMC Throughput

Description: The purpose of this test is to validate the impact of EDA/HMC to the EDGE Throughput.

The throughput values are given for one TSL. If multislot transfer is used in the testing the throughput is given per one TSL and the number of TSLs used in the test shall be mentioned in the test report.


NSN/RA/EDGE SFP


The changes in the number of timeslots and MCSs used shall be monitored during the tests and reported in test reports.



Propagation model: no fading, line-of-sight

The optimised values shown in Table 4 for the parameters related extended UL TBF to shall be used.

Note: As the optimal radio conditions may be difficult to obtain in certain test environments (e.g. SyVe, drive tests) , the figures above may not be reached. In these situations, “as good as possible” radio environment may be used. This means as good C/I ratio and RX level as is reasonably feasible with a stationary mobile terminal.

The test is performed with a FTP application, i.e. a test file is downloaded/uploaded from the test server and throughput is measured. Suggested test files to be used are a 5 Mbyte random binary file for downlink tests and a 2 Mbyte random binary file for uplink tests.





The case is similar to the basic case described in Chapter 5.1.1.


EDA/HMC UL/DL Throughput (FTP) per TSL in optimal radio conditions on dedicated TSL. (EPT5811)

Related KPIs:



Result format: throughput per TSL in kbit/s separately for different fading patterns. Applications and varying parameter values shall be described in the results. Changes in the number of timeslots and MCSs used to be described if the information is available.






NSN/RA/EDGE SFP


5.8.2 EDA/HMC RTT

Description: The purpose of this test case is to verify the impact of EDA/HMC to the RTT as experienced by the subscriber.

The test shall be performed using the PING application with different packet sizes (32, 256, 1460 bytes). Test is repeated at least 50 times for each packet size, the final value is the average over these.




EDA/HMC Active RTT. (EPT5821)

EDA/HMC Idle RTT. (EPT5822)

Related KPIs: none


Average Active/Idle RTT

95% percentile of Active/Idle RTT

Minimum and maximum values for Active/Idle RTT

Standard deviation of Active/Idle RTT

Success Rate

5.9 Other test items

Results from the following test cases in current test plans of different test phases shall be collected and included in the EDGE Performance Test Report

5.9.1 PCU LT

Following PCU2 Load Tests shall be collected and included in the EDGE Performance Test Report in E4 and in E5:


NSN/RA/EDGE SFP


EGPRS DSP load: single DSP load with 32 active channels (PCU2)

EGPRS DSP load: single DSP load with Max TSL sharing (PCU2)

GPRS DSP load: single DSP load with 32 active channels (PCU2)

GPRS DSP load: single DSP load with Max TSL sharing (PCU2)

EGPRS PQII load with Gb/FR 2M+2M throughput; MCS-5, 40 class-6 (2+2) MSs (PCU2, PCU1)

GPRS PQII load with Gb/FR 2M+2M throughput; CS-2, 80 class-6 (2+2) MSs (PCU2, PCU1)

EGPRS PQII load with Gb/IP, max. symmetric throughput; MCS-5, >60 class-6 (2+2) MSs(PCU2, PCU1)

Maximum PCU2 throughput with Gb/FR; MCS-9(PCU2, PCU1)

Maximum PCU2 throughput with Gb/IP; MCS-9(PCU2, PCU1)

5.9.2 BSC FT

IT0281051113 Scheduling step size modification, modify the downlink high priority scheduling step size

IT0281051114 Scheduling step size modification, modify the downlink normal priority scheduling step size and downlink low priority scheduling step size

IT0281056900 GPRS & EGPRS multiplexing: 1 tsl, 1 egprs MS DL and 1 gprs MS DL

IT0281056901 GPRS & EGPRS multiplexing: 1tsl, 1 egprs MS DL and 1 gprs MS UL

IT0281056911 GPRS & EGPRS multiplexing: 1 tsl, 1 egprs MS UL and 1 gprs MS UL, mobiles has different allocation class.

IT0281056912 GPRS & EGPRS multiplexing: 1 tsl, 1 egprs MS UL and 1 gprs MS UL, mobiles has same allocation class


NSN/RA/EDGE SFP


IT0281056904 GPRS & EGPRS multiplexing: 2 tsl's, 1 egprs MS DL and 1 gprs MS UL

IT0281056903 GPRS & EGPRS multiplexing: 2 tsl's, 1 egprs MS DL and 2 gprs MS's DL.

IT0281056913 GPRS & EGPRS multiplexing: 2 tsl's, 1 egprs MS UL and 1 gprs MS UL, mobiles has same allocation class.

IT0281056902 GPRS & EGPRS multiplexing: 2 tsl´s, 1 egprs MS DL and 1 gprs MS DL.

IT0281056905 GPRS & EGPRS multiplexing: 3 tsl's, 1 egprs MS DL and 1 gprs MS DL

IT0281056908 GPRS & EGPRS multiplexing: 3 tsl's, 1 egprs MS DL and 1 gprs MS UL

IT0281056906 GPRS & EGPRS multiplexing: 3 tsl's, 1 egprs MS DL and 2 gprs MS's DL




IT281053063 Simultaneous 8-PSK and GMSK TBFs allocation in same Abis TSL (RTSL)

IT281053062 Simultaneous 8-PSK and GMSK TBFs allocation in same PCU


NSN/RA/EDGE SFP


6 Test matrix

The EDGE Performance Test Matrix contains mapping between the test items and the test phases where each items should be implemented as a test case

The EDGE Performance Test Matrix is stored to PI, to the subdirectory of EDGE SFP: Projects/GSM EDGE BSS R&D/System Features/EDGE SFP/01 Nokia EDGE Feature Management/Deliverables/EDGE Performance Test Specification





NSN/RA/EDGE SFP


7 Test reporting

7.1 Test result storage and format

The test reports (for test coverage) are stored to BSC TestMan ,

The test results and EDGE Performance Test Report are stored to PI directory for EDGE SFP.


NSN/RA/EDGE SFP


8 Reference Information

8.1 Nominal data rates

The following tables provide information on GPRS/EGPRS data rates.

Table 18: GPRS User Data Rates

Timeslots

User Data Rate (Kbps)

CS-1 CS-2 CS-3 CS-4

1 9.05 13.4 15.6 21.4

2 18.1 26.8 31.2 42.8

3 27.15 40.2 46.8 64.2

4 36.2 53.6 62.4 85.6

5 45.25 67.0 78.0 107

Table 19: EGPRS User Data Rates

Timeslots

User Data Rate (Kbps)

MCS-1 MCS-2 MCS-3 MCS-4 MCS-5 MCS-6 MCS-7 MCS-8 MCS-9

1 8.8 11.2 14.8 17.6 22.4 29.6 44.8 54.4 59.2

2 17.6 22.4 29.6 35.2 44.8 59.2 89.6 108.8 118.4

3 26.4 33.6 44.4 52.8 67.2 88.8 134.4 163.2 177.6

4 35.2 44.8 59.2 70.4 89.6 118.4 179.2 217.6 236.8

5 44.0 56.0 74.0 88.0 112.0 148.0 224.0 272.0 296.0


NSN/RA/EDGE SFP


8.2 Multislot Classes (incl. High Multislot Classes)

Table 20. Multislot classes

Multislot Class

Maximum No of Slots

Rx Tx Sum1 1 1 22 2 1 33 2 2 34 3 1 45 2 2 46 3 2 47 3 3 48 4 1 59 3 2 5

10 4 2 511 4 3 512 4 4 530 5 1 631 5 2 632 5 3 633 5 4 634 5 4 635 4 1 536 4 2 537 4 3 538 4 4 539 4 4 540 5 1 641 5 2 642 5 3 643 5 4 644 5 4 645 5 4 6


NSN/RA/EDGE SFP


Appendice

Appendix A: Additional and removed test items

Appendix B: NACC Delay Breakdown, v 1.0 (Marko Pirinen)

Appendix C: Cell Change Throughput slideset (Volker Wille)

Appendix D: C/I DL & UL calibration guide (Veli-Pekka Ketonen)

(Note: not valid with RX diversity)


NSN/RA/EDGE SFP


References[1] BSS Load Testing, Test specification for validating BTS SW release

CX(M)4.0 & BSC SW11, version 0.0.5 (DRAFT), Anthony Aird et. al., 12.07.2004

[2] Nokia GPRS/EGPRS Bearer SW Platform Statement, v.1.3

[3] EDGE Key Performance Indicators, BSS11.5 EDGE/GSM, Issue 3-2, dn7032527

[4] TCP Throughput Tweaking Guide for EGPRS, Issue 1.2, Kari Niemelä

[5] Traffic Mix in PET, Proposal, Tapio Muikku, 07.10.2004

[6] Round Trip Time Reduction, BSS Feasibility Study, version 1.0.0, Kari Niemelä et al., 31.03.2004

[7] EDGE Performance Roadmap

[8] EDGE System Performance Tests

[9] PCU Reference Configuration, defined in S11.5 PCU Requirement Specificiation for the feature BSS115167 “PCU Memory, Overload Protection and Capacity Enhancements”, v.1.2.0, Ari Kosonen, 28.05.2004

[10] Nokia 2G SGSN Counters, Issue 7-1, dn99567812, (2004).

[11] TR 44.901, External Network Assisted Cell Change, v.5.1.0, 3GPP, 2002-05

[12] BSC EDGE Dimensioning, BSS11.5 EDGE/GSM, Issue 1-0, dn056767

[13] Abis EDGE Dimensioning, BSS11.5 EDGE/GSM, Issue 1-0, dn056782

[14] Gb EDGE Dimensioning, BSS11.5 EDGE/GSM, Issue 1-0, dn056794

[15] BTS EDGE Dimensioning, BSS11.5 EDGE/GSM, Issue 1-0, dn056755

[16] BTS EDGE Dimensioning, BSS11.5 EDGE/GSM, Issue 1-0, dn056779

[17] Laboratory settings for Maximum Throughput for EDGE, v.0.7 Leo Bhebhe et al.

[18] NACC Delay Breakdown, v.1.0., Marko Pirinen. Included as Appendix B.

[19] Cell Change Throughput, slide set, Volker Wille, Included as Appendix C

[20] C/I DL and UL Measurement Setups, v.0.0.1, slide set, Kimmo Myllymäki 17.06.2003, Included as Appendix D

[21] TN816: EUTM and BS_CV_MAX Parameter Value Recommendations for (E)GPRS, Technical Note No. 816, Veli-Pekka Ketonen and Mika Anttonen, 18.04.2005

[22] TN819: BSSGP Flow Control Parameter Value Recommendation for (E)GPRS, Technical Note No. 819, Mika Anttonen, 12.05.2005

[23] TN820: Initial MCS Parameter Value Recommendation for EGPRS, Technical Note No. 820, Mika Anttonen, 12.05.2005


https://aec.online.nokia.com/nols/nols/doccenter/dcDownload/download?fileidlist=0900cc9280056ba8&library=a



https://aec.online.nokia.com/nols/nols/doccenter/dcDownload/download?fileidlist=0900cc9280051d68&library=a

https://aec.online.nokia.com/nols/nols/doccenter/dcDownload/download?fileidlist=0900cc9280051d68&library=a

https://www.online.nokia.com/nols/doccenter/dcDownload/download?fileidlist=0900cc92800a6cd0&library=a

NSN/RA/EDGE SFP


[24] TN845: POLLING_INTERVAL_BG Parameter Value Recommendation for PCU2, Technical Note No. 845, Antti Palojärvi, 21.11.2006


https://www.online.nokia.com/nols/doccenter/dcDownload/download?fileidlist=0900cc92800eca1c&library=a

https://www.online.nokia.com/nols/doccenter/dcDownload/download?fileidlist=0900cc92800eca1c&library=a

GlossaryAMR Adaptive Multi-Rate

APS Air Performance System

BLER Block Error Rate

BSC Base Station Controller

BSCU Base Station Controller Signalling Unit

BSS Base Stations Subsystem

BTS Base Transceiver Station

CIR Committed Information Rate

CS Circuit Switched

CS1-2 Coding Scheme 1-2

DL Downlink

DTM Dual Transfer Mode

DUT Device Under Test

EDA Enhanced Dynamic Allocation

EDAP EGPRS Dynamic Abis Pool

EFR Enhanced Full Rate

EGPRS Enhanced General Packet Radio System

EQoS Enhanced Quality of Service

FH Frequency Hopping

FR Full Rate

FTP File Transfer Protocol

GGSN Gateway GPRS Support Node

GPRS General Packet Radio System

GSM Global System for Mobile Communication

HMC High Multislot Classes

HR Half Rate Speech

HTTP Hypertext Transfer Protocol

HT100 Hilly Terrain, 100 km/h (fading profile)

Ideal FH Ideal FH (Frequency Hopping) assumes perfect decorrelation between bursts.

IP Internet Protocol

IR Incremental Redundancy

LA Link Adaptation

LLC Logical Link Control

MCMU Marker and Cellular Management Unit

Glossary

MCS1-9 Modulation and Coding Scheme 1-9

MMS Multimedia Messaging Service

MMSC Multimedia Messaging Service Center

MOC Mobile Originated Call

MOS Mean Opinion Score (a measure e.g. for voice quality)

MS Mobile Station

MTC Mobile Terminated Call

NEMO Portable engineering tool (by Nemo Technologies, a Electrobit company) for measuring and monitoring the air interface of wireless networks.

NSS Network and Switching Subsystem

PING Packet Internet Groper

PS Packet Switched

QoS Quality of Service

RA Routing Area

RA250 Rural Area, 250 km/h (fading profile)

RC2 Reliability Class 2

SAIC Single Antenna Interference Cancellation

SAM Step Attenuator Module

SGSN Serving GPRS Support Node

SMS Short Message Service

SMCH Synchronization Master Channel

SRC Smart Radio Concept

SSCH Synchronization Slave Channel

SW Software

SWIS See-What-I-See

TCP Transmission Control Protocol

TEMS Engineering tool (by Ericsson) for measuring and monitoring the air interface of wireless networks.

TFTP Trivial File Transfer Protocol (over UDP)

TU3 Typical Urban, 3 km/h (fading profile)

TU50 Typical Urban , 50 km/h (fading profile)

UDP User Datagram Protocol

UL Uplink

VLR Visitor Location Register

WAP Wireless Application Protocol


Open issues:

C/I measurement calibration method to be described (the method proposed in [20] may give too optimistic results).

Trigggering points for intersystem (EDGE<-->WCDMA) cell change duration measurements, Chapter 5.5

IP configuration details to be added, when available, Chapter 2.7

Documents

Specification+ +Latest+Version