Keynote SIGOS Test Concept for Roaming

Keynote SIGOS Test Concept for Roaming

Document Version: 3.1.0

Date: 2013-07-26

Classification: confidential

Author: Keynote SIGOS GmbH

Keynote SIGOS GmbH

Klingenhofstr. 50d

D-90411 Nuremberg

Fon +49 / 911 / 9 51 68-0

Fax +49 / 911 / 9 51 68-355

www.keynote-sigos.com

Status: confidential Test Concept for Roaming

UC_Roaming_Testing.doc Page 2 of 37

Table of Contents

1 Management Summary ...................................................................................... 3

2 Roaming Tests: inbound and outbound .......................................................... 4

2.1 Inbound roaming tests with test system SITE ...................................................... 4

2.2 Outbound roaming tests ....................................................................................... 5

2.2.1 Outbound roaming tests – SITE Test System ................................................ 5

2.2.2 Outbound roaming tests – GlobalRoamer ...................................................... 6

3 All available roaming tests with test system SITE ......................................... 7

3.1 Roaming tests according to IREG 24 .................................................................. 7

3.2 Roaming tests according to IREG 26 ................................................................ 10



3.5 Roaming tests of IN services with CAMEL via IR.32 ......................................... 18

3.5.1 Requests on test environment ...................................................................... 19

3.5.2 IR.32 Camel Phase 1 test cases adapted and implemented by Keynote

SIGOS (based on PRD IR.32 version 3.3.0) ................................................ 21

3.5.3 IR.32 Camel Phase 2 test cases adapted and implemented by Keynote

SIGOS (based on PRD IR.32 version 3.3.0) ................................................ 24

3.6 Roaming tests of GPRS services via IR.35 ....................................................... 28

3.7 IR.42 - Quality of Service parameters and their computation ............................ 30

3.8 2G/2.5G/3G Roaming IR.50 ............................................................................... 32

3.9 Roaming tests for prepaid services via IR.60 .................................................... 34

3.10 Service Specific Test Cases .............................................................................. 36

4 History ............................................................................................................... 37



1 Management Summary

The continuous availability of services inside and outside the home network is

becoming increasingly important in the battle for customers. Whilst in the own

network the testing of these services is feasible at any time, the testing of International

Roaming is proving to be considerably more difficult.

The International Roaming Tests (IREG 24, 26, 27, 29 for GSM basic calls and

supplementary services and IR.32 (for CAMEL), IR.35 (for GPRS), IR 50 (for

2G/2.5G/3G roaming), IR.60 (for prepaid)) ensure that own subscribers roaming in

other networks can telephone in this networks.

The performance of these tests is accomplished with great manual effort with the help

of monitor systems.

As this procedure is both expensive and time-consuming, the tests are not repeated

regularly.

The intention of this document is to outline a test concept to minimise the effort for

roaming testing. Keynote SIGOS offers a test concept for decreasing of the time effort

for roaming tests covering all IR-test specifications.

The main benefits are the following:

Time advantage: Possibility of automatic testing in other networks

Periodic advantage: Regular tests possible (daily, weekly or monthly)

Documentation advantage: Test reports and statistics are delivered

(optionally)

Process advantage: No coordination with the roaming partner is necessary

Bandwidth advantage: Differing customer profiles can be tested

Innovation advantage: Roaming tests for IN services possible via CAMEL,

GPRS and all various prepaid solutions



2 Roaming Tests: inbound and outbound

2.1 Inbound roaming tests with test system SITE

The tests with inbound roaming subscribers from other roaming partners are executed

manually in most of the cases. The first time execution of IREG tests takes a couple of

weeks for each roaming partner; e.g. for IREG 24 usually more than one week. The

re-executions of the same tests are time consuming too.

The intention of Keynote SIGOS is to reduce the time effort for the first execution and

for re-execution of these tests. For this reason Keynote SIGOS has developed all test

cases of IREG 24, 26, 27, 29 and the core executable part of IR.32, the GPRS

roaming tests out of IR.35 and the prepaid roaming tests out of IR.60 on the SIGOS

test system SITE. The connected SIM Multiplexer allows testing of thousands SIM

cards in a very short time. E.g. the execution of the IREG 24 tests should not be

longer than 10 minutes. Analysing of failures takes additional time of course.

Figure 1: Automated test system with SIM Multiplexer for End-to-End Tests

PLMN

ISDN

PSTN

Um

DSS1 So

a/b A

DSS1 S2m

Test System with SIM Multiplexer

Test Probes



2.2 Outbound roaming tests

2.2.1 Outbound roaming tests – SITE Test System

Tests for outbound roaming can be executed by the test engineers of the roaming

partner within their network with the End-To-End-Test-Method. Before the test

execution the own SIM cards have to be sent to the roaming side. In most of the cases

the roaming partner has no automated test system. The first execution normally takes

a long time because of some configuration problems in the switches and because of

the manually execution of the tests. After the first test execution the roaming partner

does not re-execute the tests periodically. Therefore the network provider does not

know if the own services are available in the roaming partner network. This leads to

the undesired effect that roaming problems are first discovered by the end user.

Besides that testing of new profiles of the subscribers the SIM cards have to be sent

periodically to the roaming partner. This handling of this „SIM tourism“ is not easy.

For outbound roaming tests it is necessary to use a distributed test system with

connection to a SIM Multiplexer located in the own network:

Figure 2: Distributed test system with central SIM Multiplexer

LAN/WAN

Local Unit for

Outbound Roaming

Central

Unit

SIM Multiplexer

SIM Card

Selection

1.

Virtual

SIM Card

Transfer 2.

UMTS Uu

GSM/GPRS Um

Local Unit for

Outbound Roaming

GSM A

ISDN

Local Unit for

Outbound Roaming

Local Unit for

Outbound Roaming



Keynote SIGOS has developed such a test platform on which end-to-end and service

tests are performed in various GSM and UMTS networks. This system is usable on a

distributed basis and consists of a central control unit and various locally test units all

linked via an IP network. These can be installed in different European countries (cf.

Figure 2:) and all fitted with test heads for the GSM/GPRS Um interface.

The system has a central SIM Multiplexer, which transmits the SIM card information to

the respective test unit in the selected country. This avoids the „SIM card tourism“, all

SIMs are installed in the SIM Multiplexer located in own network.

If a network provider prefers to buy this distributed test system SITE the provider will

manage the remote local units distributed in other countries himself and will execute

the tests himself too.

If the network provider prefers to leave the outbound roaming test to Keynote SIGOS

the test system of Keynote SIGOS can be used and all tests will be executed by

Keynote SIGOS as described in the following chapter.

2.2.2 Outbound roaming tests – GlobalRoamer

The SIGOS GlobalRoamer is an internationally installed test system for 2, 2.5 and 3G

mobile services. It has a powerful web-based graphical user interface to enable the

user to construct the precise test required in the minimum amount of time. The test

can then be run (repeatedly) in real-time. The user can view the test as it progresses

and has immediate access to detailed trace information as soon as the test completes

in order to troubleshoot rapidly.

With a large library of service test templates and the ability to customize unique tests

across a broad range of services, the GlobalRoamer enables rapid and focused

troubleshooting of roaming service problems.

All tests can be executed on an ad-hoc or permanent basis using the powerful

scheduling capabilities. The test results can be compared to user definable thresholds

triggering alarms once these are crossed. Additionally all test results can be exported

for further off-line analysis. These powerful reporting techniques allow benchmarking

of service quality and performance against historical values or even against those of

competitors.

In addition GlobalRoamer provides reporting capability to view reports by service,

roaming partner or country. Because of the underlying service model technology it is

also possible to drill-down to identify particular service problems, view individual test

results or even view detailed test trace information to help identify the root cause of a

service problem.

Finally, GlobalRoamer provides regular (daily) email summary results in comma

separated variable (csv) format. These results can be imported into spreadsheets,

OLAP or other tools for even more sophisticated analysis and reporting.

Of course it is also possible to outsource the whole outbound testing to Keynote

SIGOS. We define and execute all tests according to the customer requirements and

provide periodic test reports.



3 All available roaming tests with test system SITE

3.1 Roaming tests according to IREG 24

The roaming requirements for the GSM basic call scenarious and a part of the GSM

supplementary services are described in the specification IREG 24 by the GSM

Association. Following tests out of IREG 24 can be executed with the test system

SITE automatically (based on PRD IR.24 version 5.0):

IREG24 Test Cases covered by SITE Comment

yes no

1.1 Roaming Scenario to be tested

Administration task

1.2 Network Operator Information

Administration task

1.3 Toll Ticket Configuration Administration task

2.1 Basic Service

2.1.1 Location Update by MS(a) in VPLMN(b)

X

2.1.2 A Operator Control of Service

X Manually administration needed for delete of subscription from HLR

2.1.2 B Operator Control of Service

X Manually administration needed for ODB

2.1.3 MS1(a) calls MS2(a) both roamed in VPLMN(b)

X No echo detection

2.1.4 PSTN telephone in Country containing PLMN(b) calls MS1(a), which has roamed to VPLMN(b), but the VLR has lost the record for MS1(a).

X

Realized as 2 separate test cases: 2.1.4.a with loss of VLR record (manually administration is needed!); 2.1.4.b normal processing after location update (should be executed if 2.1.4.a fails)

2.1.5 PSTN telephone in Country containing PLMN(a) calls MS(a), which has roamed to VPLMN(b), but has performed IMSI detach.

(X) HPLMN specific announcement is recorded for offline-analysis

2.1.5 PSTN telephone in Country containing PLMN(a) calls MS(a), which has roamed to VPLMN(b), but is not able to respond to paging.

(X)

VPLMN specific announcement is recorded for offline-analysis




yes no

2.2 Supplementary Services

2.2.1.i Barring of All Outgoing Calls, Emergency Call

(X)

Emergency calls have not been automated, but a test case is provided for a call to any external number. The user is responsible for choosing emergency number.

2.2.1.ii Barring of All Outgoing Calls, PSTN

X

2.2.2.i Barring of Outgoing International Calls, PSTN

X

2.2.2.ii Barring of Outgoing International Calls, HPLMN

X

2.2.3.A.i Barring of Outgoing International Calls except to Home PLMN country. Call to HPLMN

X

2.2.3.A.ii Barring of Outgoing International Calls except to Home PLMN country. Call to VPSTN

X

2.2.3.Aiii Barring of Outgoing International Calls except to Home PLMN country. To other country

X

2.2.3.B.i Barring of Outgoing International Calls except to Home PLMN country. To local PSTN

X

2.2.3.B.ii Barring of Outgoing International Calls except to Home PLMN country. To Home PSTN

X

2.2.4.A Barring of all Incoming Calls(BAIC/BAICroaming)

X

2.2.4.B Barring of all Incoming Calls(BAIC/BAICroaming)

X

2.2.5 Call Forwarding on Not Reachable (before IMSI detach)

X

2.2.6 Call Forwarding on Not Reachable (after IMSI detach)

X

2.2.7 Call Forwarding on Busy X

2.2.8 Call Forwarding on No Reply X




yes no

2.3 SMS Test Cases

2.3.1 Mobile Originated and Terminated SMS

X

2.4 SMS Interworking Test Cases

2.4.1 Location Update by MS(a) in VPLMN(b)

X

2.4.2.1 SMS Tests in case (WITH ROAMING AGREEMENT)

2.4.2.1 Mobile(a) in PLMN(b) to Mobile(b) in PLMN(b)

X

2.4.2.2 Mobile(b) in PLMN(b) to Mobile(a) in PLMN(b)

X

2.4.3.1 SMS Tests in case of NO ROAMING AGREEMENT

2.4.2.1 Mobile(b) in PLMN(b) to Mobile(a) in PLMN(a)

X Can be offered if requested

2.4.2.2 Mobile(a) in PLMN(a) to Mobile(b) in PLMN(b)

X Can be offered if requested




The roaming requirements for the GSM supplementary services are described in the

specification IREG 26 by the GSM Association. Following tests out of IREG 26 can be

executed with the test system SITE automatically (based on PRD IR.26 version

3.7.0):


yes no

2.1 Call Waiting and Call Hold

2.1.1 Call Hold X not for UMTS tests

2.1.2 Call Waiting X

2.2 Multiparty

2.2.1 Test case for Multiparty X

2.3 Advice of Charge

2.3.1 Advice of Charge (Charging) for a MOC

X

2.3.2 Advice of Charge (Information) for a MOC

X

2.3.3 Advice of Charge (Charging) for a MTC

X

2.3.3.1 Test of correct functionality X

2.3.3.2 Test of non-support of AoCC X

2.4 Calling Line Identity Service

Option: 1

2.4.1 HPLMN supports CLI / VPLMN supports CLI

2.4.1.1 MS interrogation of the Calling Line Identification Restriction - Permanent Mode

X

2.4.1.2 i Calling Line Identification Presentation/Calling Line Identification Restriction

X

2.4.1.2 ii Calling Line Identification Line Identification Restriction

X

2.4.1.2 iii Calling Line Identification Presentation/Calling Line Identification Restriction

X

2.4.1.3 i Calling Line Identification Presentation/Restriction in Interworking Situation with Conditional Call Forwarding

X

2.4.1.3 ii Calling Line Identification Presentation/Restriction in Interworking Situation with Conditional Call Forwarding

X




yes no


X

2.4.1.4 ii Calling Line Identification Presentation/Calling Line Identification Restriction

X

Option: 2

2.4.2 HPLMN supports CLI/VPLMN does not support CLI

The option 2 (CLI supported by HPLMN but is not supported by VPLMN) is not required by network operators. On request of a network provider the test cases of option 2 can be offered.

Option: 3

2.4.3 HPLMN does not support CLI/VPLMN supports CLI

2.4.3.1 MS interrogation of the Calling Line Identification Restriction - Permanent Mode

X


X

2.4.3.2 ii Calling Line Identification Presentation/Calling Line Identification Restriction

X

2.4.3.3 Calling Line Identification Presentation/Permanent Mode in Interworking Situation with Conditional Call Forwarding

X

2.4.3.4 Calling Line Identification Presentation/Calling Line Identification Restriction - General

X

2.5 Connected Line Identity Services

no test cases required by IREG

2.6 Closed User Group

Test case Closed User Group

X

2.7 Call Completion to a Busy Subscriber

no test cases required by IREG

2.8 Call Transfer no test cases required by IREG

2.9 Operator Determined Barring

Look for Initial conditions:




yes no

2.9.1 Operator Determined Barring of All Outgoing Calls

X

2.9.1.i Emergency Call X Emergency calls have not been automated!

2.9.1.ii Call to PSTN X can be offered, if requested

2.9.2.i Call to local PSTN X can be offered, if requested

2.9.2.ii Call to an information/entertainment service

X can be offered, if requested

2.9.3 Operator Determined Barring of Supplement Services Management

2.9.3.i Activation of SS:BOAC X can be offered, if requested

2.9.3.ii Call to local PSTN X can be offered, if requested

2.9.3.iii Interrogation on SS:BOAC X can be offered, if requested

2.10 USSD

2.10.1 USSD Phase 1 General Test X

2.10.2 USSD Phase 2 General Test X




The roaming requirements for the GSM data services are described in the

specification IREG 27 by the GSM Association. Following tests out of IREG 27 can be

executed with the test system SITE automatically (based on PRD IR.27 version

3.3.0).

Note: These tests are NOT applicable for A-interface and UMTS.


yes no

E 5.1 Mobile Originated Data Call

E 5.1.1 MS(a) roamed to PLMN(b) calls PSTN modem(a) in the country containing PLMN(a) (3.1 kHz audio)

5.1.1.a.1 NT mode of MSISDN of calling MS, 9,6 kb/s

X Note: PSTN modem in country(b), not country(a)

5.1.1.a.2 T mode of MSISDN of calling MS, 9,6 kb/s

X


X


X

5.1.1.a.5 - 5.1.1.a.14 X

E 5.1.2 MS(a) roamed to PLMN(b) calls ISDN TA (a) in the country containing PLMN(a) (UDI)


X Note: ISDN TA in country(b), not country(a)


X


X


X

5.1.2.a.5 - 5.1.2.a.14 X

E 5.1.3 MS(a) roamed to PLMN(b) calls PSTN modem(a) in the country containing PLMN(a) (3.1 kHz audio), Synchronous transmission

X

E 5.1.4 MS(a) roamed to PLMN(b) calls Visited ISP(b) Access Number (3.1 kHz audio)


X Can be offered, if requested


X






yes no

5.1.4.a.4 - 5.1.4.a.14 X

E 5.1.5 MS(a) roamed to PLMN(b) calls Home ISP(a) Access Number (3.1 kHz audio)




X



5.1.5.a.4 - 5.1.5.a.14 X

E 5.1.6 MS(a) roamed to PLMN(b) calls VPLMN ISP(b) Access Number (UDI)




X



5.1.6.a.4 - 5.1.6.a.14 X

E 5.1.7 MS(a) roamed to PLMN(b) calls HPLMN ISP(a) Access Number (UDI)




X



5.1.7.a.4 - 5.1.7.a.14 X

E 5.2 Mobile Terminated Data Call

E 5.2.1 PSTN modem(a) in the country containing PLMN(a) calls MS(a) roamed to PLMN(b). (3.1 kHz audio)

5.2.1.a.1 NT mode of MSISDN of called MS, 9,6 kb/s


5.2.1.a.2 T mode of MSISDN of called MS, 9,6 kb/s

X

5.2.1.a.3 NT mode of MSISDN of called MS, 14,4 kb/s

X

5.2.1.a.4 T mode of MSISDN of called MS, 14,4 kb/s

X

5.2.1.a.5 - 5.2.1.a.14 X

E 5.4 Mobile-to-Mobile call

E 5.4.1 MS(b) calls MS(a) roamed to PLMN(b), (3.1 kHz audio)




yes no


X


X


X


X

5.4.1.a.5 - 5.4.1.a.14 X

E 5.5 MS(a) roamed to VPLMN(b) attaches HPLMN WAP services

5.5.1 MS(a) roamed to VPLMN(b) obtains HPLMN WAP services with digital connection


5.5.2 MS(a) roamed to VPLMN(b) obtains HPLMN WAP services with analogue connection


E 5.6 Mobile Originated Fax call

E 5.6.1 MS(a) roamed to PLMN(b) calls PSTN fax(a) in the country containing PLMN(a).

5.6.1.a.1 T mode of MSISDN of calling MS(a), TS=61 (alt. speech/fax) 9600 bit/s

X

5.6.1.a.2 T mode of MSISDN of calling MS(a), TS=62 (aut.fax) 9600 bit/s


E 5.7 Mobile Terminated Fax call

E 5.7.1 MS(a) roamed to PLMN(b) calls PSTN fax(a) in the country containing PLMN(a).

5.7.1 T mode of MSISDN of called MS, 9600 bit/s


E 5.8 Mobile to Mobile Fax call

E 5.8.1 MS(a2) in Home PLMN calls MS(a1) roamed to PLMN(b)

5.8.1 T mode of MSISDN of called MS, 9600 bit/s

X Note: both MS roaming in visited PLMN

E 5.9 Mobile To Mobile Video call (UDI)

5.9.1 MS(a2) in Home PLMN calls MS(a1) roamed to PLMN(b) (UDI).





yes no

5.9.2 MS(a1) roamed to PLMN(b) calls MS(a2) in Home PLMN (UDI).


5.9.3 MS(a) in Home PLMN calls MS(b) in Home PLMN (UDI).

X Note: MS(a) roaming in visited PLMN

5.9.4 MS(b) in Home PLMN calls MS(a) in Home PLMN (UDI).

X Note: MS(a) roaming in visited PLMN




The roaming requirements for the automatic testing are described in the specification

IREG 29 by the GSM Association. Following tests out of IREG 29 can be executed

with the test system SITE automatically (based on PRD IR.29 version 3.0.0).


yes no

5.1 Basic Service Test Cases

5.1.1 Location update by MS/SIMa1 and MS/SIMa2 in VPLMN and MS/SIMa1 calls MS/SIMa2

X

5.2 Supplementary Service Test Cases

5.2.1 BARRING OF OUTGOING

INTERNATIONAL CALLS X

5.2.2 CALL FORWARDING ON NOT

REACHABLE X

5.3 Data Transmission Services

5.3.1 Data Transmission Indication


5.3.2 Data Transmission, Bearer Service 26


5.4 Short Message Service

5.4.1 SMS Mobile Originating – Mobile Terminating

X

5.5 Telefax Services

5.5.1 MS/SIMA1 CALLS TELEFAX

IN HPLMN(A) (MOBILE

ORIGINATING FAX)

X not for UMTS tests



3.5 Roaming tests of IN services with CAMEL via IR.32

Figure 3: Test configuration for testing according to IR.32

The tests for CAMEL based roaming are specified in the roaming specification IR.32.

Some of the IR.32-test cases are written for the normal behaviour of the GSM

subscribers with IN/CAMEL service profiles. Keynote SIGOS has implemented such

kind of test cases which have direct relationship to the used CAMEL services like

“roaming with prepaid” for the End2End-test configuration. Normal GSM/GPRS test

mobiles are used for testing.

All the selected tests can be run for CAMEL 1 or 2. There is no different behaviour

between CAMEL Phase 1 and 2 out of the view of End2End-Testing. The different

behaviour of phase 1 and 2 shall be regarded if the CAP interface is tested with e.g.

SS7 monitoring equipment. The IR.32 specification describes the CAP behaviour for

phase 1 and 2.

The IR.32 specification contains also test cases written for pre-tests of CAMEL for

checking of the general interoperability between networks without any relevance for

the normal usage. All these test cases which are not relevant for normal CAMEL

subscribers and which requires very special SIM cards for testing issues have been

skipped. However this kind of tests is not used by various network providers testing

CAMEL roaming and there don’t exist of any special SIM cards anywhere.

SIM Multiplexer

SITE

SCP

CAP

SS7 Monitoring

MAP

HLR

MSC/VLR/SSP

Roaming partner

(HPLMN)

Own network (VPLMN)



The intention of the IR.32 tests is testing of the general CAMEL functionality. With the

selected test cases out of IR.32 specification the general CAMEL functionality can be

tested with focus on some CAMEL services like “prepaid”. These tests can not cover

the testing of the complete service functionality. For service tests like “prepaid

roaming” there are other specifications like IR.60 which shall be covered for roaming

tests too.

In chapter two all the requests on the test equipment environment are described.

Chapter 3 contains the IR.32 test cases adapted and implemented by Keynote

SIGOS. All these test cases have been implemented on the test system SITE and are

available.

3.5.1 Requests on test environment

The selected IR.32 test cases ensure inbound CAMEL roaming with a certain roaming

partner’s network. These tests have to be executed with each roaming partner.

Test system

The test system SITE with its graphical user interface contains various test mobiles

and SIM Multiplexer containing the SIM cards from all existing roaming partner. The

huge number of test cases like GSM basic call, supplementary services, SMS, WAP,

MMS, roaming IREG.24, 26, 27, IR.32, IR.35 and IR.60 are implemented in SITE and

can be executed automatically and in parallel with different roaming partners at the

same time.

Test monitoring equipment for CAP and MAP

With using of the test system SITE all implemented IR.32 can be tested in End-To-End

test method. In case of errors analysing of the CAP interface between the own

network SSP and roaming partners SCP is needed.

A SS7 monitoring system can be used for analysing of the CAP interface according to

IR.32 test specification in case of errors between two networks. The SS7 monitoring

equipment is also needed for checking of the MAP interface in some of the CAMEL

related operations. These kinds of checks are noted in all test cases.

Required SIM cards per roaming partner

The IR.32 test cases offered by SITE are implemented in such a way that they can be

executed without requiring “special SIM cards” for “special IR.32 services”.

Therefore “normal SIM cards” (usually prepaid SIM cards) from the roaming partners

can be used for these tests.

The tests require two SIM cards (named MS1(a) and MS2(a)) from the roaming

partner (HPLMN(a)) with different profiles and one SIM card (MS3(b)) of the PLMN

serving as visited network (VPLMN(b)).



Profiles of the used SIM cards for Camel Phase 1 (minimal set):

MS1(a): MS2(a): MS3(b):

Teleservice: TS11 O-CSI with defaultCallHandling:

continueCall SS: BOICexHC provisioned SS: CFNRy provisioned


releaseCall T-CSI with defaultCallHandling:

releaseCall ODB-BOICexHC active

No special profile needed for MS3(b)

Profiles of the used SIM cards for Camel Phase 2 (minimal set):

MS4(a): MS5(a): MS3(b):


releaseCall SS: BOICexHC provisioned SS: CFNRy provisioned


continueCall T-CSI with defaultCallHandling:

releaseCall ODB-BOICexHC active

No special profile needed for MS3(b)



3.5.2 IR.32 Camel Phase 1 test cases adapted and implemented by Keynote SIGOS (based on PRD IR.32 version 3.3.0)

Test Case Imple-mented

TC Name Purpose

2.1.1.1 Location Update by MS1(a) in VPLMN(b)

YES T_IR32_2111 Correct location update and insertion of subscriber data (e.g. O-CSI) in VLR is verified for MS1(a). This test confirms the MAP V3 interoperability between the two networks.

The MAP interface between HLR(a) and VLR(b) should be monitored for correct setting of CAMEL specific parameter (O-CSI).


YES T_IR32_2112 Correct location update and insertion of subscriber data (e.g. O-CSI, T-CSI) in VLR is verified for MS2(a). This test confirms the MAP V3 interoperability between the two networks.

The MAP interface between HLR(a) and VLR(b) should be monitored for correct setting of CAMEL specific parameter (O-CSI and T-CSI).

2.1.2 MOC by MS1(a) for CAMEL-Ph1 rerouting

YES T_IR32_212 Purpose: To verify the correct usage of CAP operations IDP and CON in case of the service realisation via CAMEL Phase 1 rerouting. This test is performed by establishing a mobile originating (MOC) call for MS1(a).

The CAP interface between SCP(a) and SSP(b) should be monitored for correct setting of CAMEL specific operations (IDP and CON).

2.1.3 Check operation of PSI via MTC to MS2(a)

YES T_IR32_213 To verify the correct usage of the MAP operation PSI. Therefore a mobile terminated call (MTC) is established towards MS2(a). The test is executed for the idle case.

Note: It is required to monitor the MAP interface between HLR(a) and VLR(b) for correct usage of PSI because in end-to-end method it is not possible to verify the usage of PSI!

For this test T-CSI for MS2(a) is necessary!




TC Name Purpose

2.1.4 MOC by MS1(a) with RRB, ERB and CON at CAP

YES T_IR32_214 Purpose: To verify the correct usage of CAP operations RRB and ERB and CON in case of the service realisation directly at the CAP interface. This test is performed by establishing a mobile originating (MOC) call for MS1(a).

This test can be seen as an alternative to S_IR32_212 and is to be used if the service is not realised via CAMEL-Phase 1 rerouting.

The CAP interface between SCP(a) and SSP(b) should be monitored for correct setting of CAMEL specific operations (IDP, RRB, ERB and CON).

2.1.5 MOC by MS1(a) with RRB, ERB and CUE at CAP

YES T_IR32_215 Purpose: To verify the correct usage of CAP operations RRB and ERB and CUE in case of the service realisation directly at the CAP interface. This test is performed by establishing a mobile originating (MOC) call for MS1(a).

This test equals S_IR32_212 except that CUE is used instead of CON.

The use of CUE is realised in choosing a specific B-party.


2.1.6.1 Check defaultCallHandling continueCall

YES T_IR32_2161 To verify the correct usage of the defaultCallHandling continueCall.

This test is performed by establishing a successful mobile originating (MOC) call for MS1(a).

Note: for this test it is necessary to suppress the response by the SCP to the IDP. This can be achieved by removing the global title translation from the MSC/SSP.

2.1.6.2 Check defaultCallHandling releaseCall

YES T_IR32_2162 To verify the correct usage of the defaultCallHandling releaseCall.

This test is performed by establishing an unsuccessful mobile originating (MOC) call for MS2(a).





TC Name Purpose

2.1.7 Check Suppression of Announcement

NO T_IR32_2162

Not applicable for Prepaid

2.1.8.1 Call Forwarding on no reply (CFNRy) for MS1(a)

YES T_IR32_2181 To verify that in case of a mobile forwarded call (CFNRy) originating CAMEL service is invoked for a subscriber with O-CSI.


not for UMTS tests

2.1.8.2 SS Barring of outgoing international calls except to HPLMN country (BOICexHC), barred call

YES T_IR32_2182a To verify that in case of SS barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to an international number will be barred.

Note: This test requires SS-BOICexHC provisioned for MS1(a).

2.1.8.2 SS Barring of outgoing international calls except to HPLMN country (BOICexHC), successful call

YES T_IR32_2182b To verify that in case of SS barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to a number belonging to the HPLMN-country(a) will be successful.


2.1.8.3 ODB Barring of outgoing international calls except to HPLMN country (BOICexHC), barred call

YES T_IR32_2183a To verify that in case of ODB barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to an international number will be barred.

Note: This test requires ODB-BOICexHC provisioned for MS2(a).

2.1.8.3 ODB Barring of outgoing international calls except to HPLMN country (BOICexHC), successful call

YES T_IR32_2183b To verify that in case of ODB barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to a number belonging to the HPLMN-country(a) will be successful.




3.5.3 IR.32 Camel Phase 2 test cases adapted and implemented by Keynote SIGOS (based on PRD IR.32 version 3.3.0)


TC Name Purpose


YES T_IR32_2211 Correct location update and insertion of subscriber data (e.g. O-CSI) in VLR is verified for MS4(a). This test confirms the MAP V3 interoperability between the two networks.

The MAP interface between HLR(a) and VLR(b) should be monitored for correct setting of CAMEL specific parameter (O-CSI).


YES T_IR32_2212 Correct location update and insertion of subscriber data (e.g. O-CSI, T-CSI) in VLR is verified for MS5(a). This test confirms the MAP V3 interoperability between the two networks.

The MAP interface between HLR(a) and VLR(b) should be monitored for correct setting of CAMEL specific parameter (O-CSI and T-CSI).

2.2.2.1 MOC by MS4(a), Check operation of IDP and CON

YES T_IR32_2221 Purpose: To verify the correct usage of CAP operations IDP and CON. This test is performed by establishing a mobile originating (MOC) call for MS1(a).


2.2.2.2 Called Number Length not Defined by Destination Number Length List

NO


2.2.3 Check operation of PSI via MTC to MS2(a)

YES T_IR32_223 To verify the correct usage of the MAP operation PSI. Therefore a mobile terminated call (MTC) is established towards MS4(a). The test is executed for the idle case.

Note: It is required to monitor the MAP interface between HLR(a) and VLR(b) for correct usage of PSI because in end-to-end method it is not possible to verify the usage of PSI!

For this test T-CSI for MS2(a) is necessary!




TC Name Purpose

2.2.4 MOC by MS4(a) with RRB, ERB and CON at CAP

YES T_IR32_224 Purpose: To verify the correct usage of CAP operations RRB and ERB and CON in case of the service realisation via these operations. This test is performed by establishing a mobile originating (MOC) call for MS4(a).


2.2.5.1 Follow-On Call due to “Called Party does not Reply”

NO


2.2.5.2 Follow-On Call due to “Called Party is busy”

NO


2.2.5.3 Follow-On Call due to “Called Party is not Reachable”

NO


2.2.5.4 Follow-On Call due to “Route Select Failure” by “Release before Answer”

NO


2.2.6.1 MOC by MS4(a) with SCI and AC at CAP

YES T_IR32_2261 Purpose: To verify the correct usage of CAP operations SCI and AC in case of the service realisation via these operations. This test is performed by establishing a mobile originating (MOC) call for MS4(a).

This test can be seen as an alternative to S_IR32_224


2.2.6.2 Check reporting of call periods (2)

NO Not applicable for automatic end-to-end testing

2.2.6.3 Check warning tone and automatic call release

NO Not applicable for automatic end-to-end testing

2.2.7 Check operation of ETC and DFC

NO Not applicable for prepaid




TC Name Purpose

2.2.8.1 Check defaultCallHandling continueCall

YES T_IR32_2281 To verify the correct usage of the defaultCallHandling continueCall.

This test is performed by establishing a successful mobile originating (MOC) call for MS5(a).


2.2.8.2 Check defaultCallHandling releaseCall

YES T_IR32_2282 To verify the correct usage of the defaultCallHandling releaseCall.

This test is performed by establishing an unsuccessful mobile originating (MOC) call for MS4(a).


2.2.9 Check Suppression of Announcement

NO


2.2.10.1 Check operation of SS-CSI, MPTY

NO


2.2.10.2 Call Forwarding on no reply (CFNRy) for MS1(a)

YES T_IR32_22102 To verify that in case of a mobile forwarded call (CFNRy) originating CAMEL service is invoked for a subscriber with O-CSI.


2.2.10.3 SS Barring of outgoing international calls except to HPLMN country (BOICexHC), barred call

YES T_IR32_22103a To verify that in case of SS barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to an international number will be barred.





TC Name Purpose

2.2.10.3 SS Barring of outgoing international calls except to HPLMN country (BOICexHC), successful call

YES T_IR32_22103b To verify that in case of SS barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to a number belonging to the HPLMN-country(a) will be successful.


2.2.10.4 ODB Barring of outgoing international calls except to HPLMN country (BOICexHC), barred call

YES T_IR32_22104a To verify that in case of ODB barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to an international number will be barred.


2.2.10.4 ODB Barring of outgoing international calls except to HPLMN country (BOICexHC), successful call

YES T_IR32_22104b To verify that in case of ODB barring of outgoing international calls except to HPLMN country (BOICexHC) a mobile originated call to a number belonging to the HPLMN-country(a) will be successful.




3.6 Roaming tests of GPRS services via IR.35

Keynote SIGOS has developed GPRS test cases with roaming requirements on the

own test system which can cover IR.35 test cases (Version 5.0).

GSM MoU IR35 Test Cases covered by SITE Comment

yes no

2.1 Mobile registration

2.1.1 GPRS-Attach of MS1(a) in VPLMN(b)(GPRS attach only)

X

2.2 GPRS PDP context activation by MS1(a) in VPLMN(b)

2.2.1 Intranet-/ISP-Access of MS1(a) in VPLMN(b) using home GGSN (user provides Network-ID and no Operator-ID and is not allowed to use visited GGSN), HTTP-WEB-Browsing

X

2.2.2 WAP access of MS1(a) in VPLMN(b) using home GGSN (user provides Network-ID and no Operator-ID and is not allowed to use visited GGSN), WAP-Browsing

X WAP1.x and WAP2.0

2.2.3 MMS with MS1(a), MS2(a) in VPLMN(b) using home GGSN and MMSC

X MMS1.x and MMS2.0

2.2.4 Intranet-/ISP-Access of UE1(a) in VPLMN(b) using home PGW over Gp-interface (user provides Network-ID and no Operator-ID and is not allowed to use visited GGSN), HTTP-WEB-Browsing

X

2.2.5 Intranet-/ISP-Access of UE1(a) in VPLMN(b) using visited SGW and home PGW over S8-interface (user provides Network-ID and no Operator-ID and is not allowed to use visited GGSN), HTTP-WEB-Browsing

X



GSM MoU IR35 Test Cases covered by SITE Comment

yes no

2.3 MAP-Tests (SMS, Location Cancellation, GPRS-Barring)

2.3.1 Mobile originated and terminated SMS(MS1(a) GPRS-attached only / MS2(a) GPRS-attached only)

X not for UMTS at A Side

2.3.2 Location cancellation for MS1(a)-GPRS Subscription held in VPLMN(b)

X

2.3.3 Operator Determined Barring for all MS initiated PDP context activatons of MS1(a)

X

19.1.1 LTE Attach and Detach X

19.2.1 LTE Cancel Location X

19.3.1 LTE Operator Determined Barring

X



3.7 IR.42 - Quality of Service parameters and their computation

SITE provides IR.42 KPIs based on: GSM Association, IR.42 Version 3.2.1, February

2003. Therein they say “General considerations:

All the defined quality of service parameters and their computations are based on field

measurements. That indicates that the measurements were made from customers

point of view (full End-to-End perspective, taking into account the needs of testing).

It is assumed that the end customer can handle his mobile and the services he wants

to use (operability is not evaluated at this time). For the purpose of measurement it is

assumed

1. that the service is available and not barred for any reason

2. routing is defined correctly without errors and

3. the target subscriber equipment is ready to answer the call.

Voice quality values measured should only be employed by calls ended successfully

for statistical analysis.

However, measured values from calls ended unsuccessfully (e.g. dropped) should be

available for additional evaluations and therefore, must be stored.”

IR.42 test cases adapted and implemented by Keynote SIGOS:

IR.42 Test Cases covered by SITE Comment

yes no

2.2 Service Independent

2.2.1 Network Accessibility Circuit Switched

X NA - CS

2.1.2 Network Accessibility Packet Switched

X NA - PS

2.3 Telephony

2.3.1 Service Accessibility Telephony

(X) SA-T ; in development / modifications needed

2.3.2 Setup Time Telephony X ST-T

2.3.3 Speech Quality X SpQ

2.3.4 Call Completion Rate Circuit Switched Telephony

(X) CCR-CS-T; in development / modifications needed

2.4 Short Message Service

2.4.1 Service Accessibility SMS MO

X SA-SMS-MO

2.4.2 Access Delay SMS MO X AD-SMS-MO

2.4.3 End-to-End Delivery Time SMS

X DT-SMS

2.4.4 Completion Rate SMS Circuit Switched

X CR-SMS-CS; We can not recognize duplicate received SMS



IR.42 Test Cases covered by SITE Comment

yes no

2.5 Circuit Switched Data Service

2.5.1 Service Accessibility, Circuit Switched Data

(X) SA-CSD ; in development / modifications needed

2.5.2 Set-up Time X ST-CSD

2.5.3 Data Quality (X) DQ-CSD; in development / modifications needed

2.5.4 Call Completion Rate Circuit Switched Data

(X) CR-CSD; in development / modifications needed

2.6 Packet Switched Data Service (General Packet Ratio Service)

2.6.1 Service Accessibility Rate – Packet Switched Data

X SA-PSD

2.6.2 Set-up Time – Packet Switched Data

X ST-PSD

2.6.3 Data Quality X DQ-PSD

2.6.4 Completed Session Ratio (X) CoSeR-PSD; in development / modifications needed



3.8 2G/2.5G/3G Roaming IR.50

Keynote SIGOS has developed test cases with roaming requirements on the own test

system which can cover IR.50 test cases based on: GSM Association, IR.50 Version

4.0.

Roaming-aspects of roaming between 2G-, 2.5G- and 3G-networks is meant as an

reference to operators and their roaming partners to define the international roaming

service in the 2/2.5/3G environment.

The IR.50 test cases are referenced on test cases of IR.24/32/35/60. The real IR.50

content is generated of IR.24/32/35/60 results in IR.50 reports (PDF-Files, not Excel).

That also means the IR.50 test application requires test applications IR.24, IR.32,

IR.35 and IR.60. Following test cases are relevant:

Test case name Description

IR24_2_1_1 Location Update by MS(a) in VPLMN(b)

IR24_2_1_2a A Operator Control of Service

IR24_2_1_2b Check Operator Determined Barring (ODB)

IR24_2_1_3 MS1(a) calls MS2(a) both roamed in VPLMN(b)

IR24_2_1_4b

PSTN telephone in Country containing PLMN(b) calls MS1(a),

which has roamed to VPLMN(b), but the VLR has lost the record

for MS1(a). Normal processing after location update

IR24_2_2_1i Barring of all outgoing calls - BAOC - successful call to external number (e.g. emergency number).

IR24_2_2_1ii Barring of all outgoing calls, call to PSTN

IR24_2_2_2i Barring of all outgoing international calls, call to national number.

IR24_2_2_8 Call Forwarding on No Reply (CFNRY)

IR24_2_3_1 Mobile Originated and Terminated SMS

IR27_5_9_1 MOBILE TO MOBILE VIDEO CALL (UDI)

IR35_2_2_1

Intranet-/ISP-Access of MS1(a) in VPLMN(b) using home GGSN

(user provides Network-ID and no Operator-ID and is not allowed

to use visited GGSN), HTTP-WEB-Browsing

IR35_2_2_2

WAP access of MS1(a) in VPLMN(b) using home GGSN (user

provides Network-ID and no Operator-ID and is not allowed to

use visited GGSN), WAP-Browsing

IR35_2_2_3 MMS with MS1(a), MS2(a) in VPLMN(b) using home GGSN and

MMSC

IR35_2_3_2

Location cancellation for MS1(a)-GPRS Subscription held in

VPLMN(b)

Note: This test requires operator interaction!!!

IR35_2_3_3

Operator Determined Barring for all MS initiated PDP context

activatons of MS1(a)

Note: This test requires operator interaction!!!

IR60_6_7_1 SMS MO supported

IR60_6_15 Call Forwarding No Reply (CFNRy)

IR60_6_17 MOC: Account depleted

IR60_8_1 Video Telephony



Glossary

2G Network PLMN with all cells of GSM radio bearer type The network supports classic GSM services, i.e. GSM Services except GPRS. (see e.g. GSM TS 02.02 and 02.03)

2.5G Network PLMN with all cells of GSM radio bearer type extended to support GPRS.

3G Network PLMN with all cells of UMTS radio bearer type

2.5G/3G Network PLMN with cells of type GSM and cells of type UMTS



3.9 Roaming tests for prepaid services via IR.60

The test specification IREG 60 provides service specific tests for prepaid international

roaming for CAMEL (direct dialling) as well as for Call Back solutions.

Following tests out of IREG 60 can be executed with the test system SITE

automatically (GSM Association IREG Version 4.0).

Note: With SITE 2.3 optional automated account balance checks for each test

are offered via USSD, IVR or SMS. The extraction of the account balance is only

possible for USSD. For IVR the received Audio message is stored for offline

analyisis in the test case trace. For SMS the text of the received response SMS

is stored for offline analyisis in the test case trace and is delivered as KPI for

offline analysis.


yes no

6 GSM Test Cases

6.0.1.1 Location Update for MS1 X

6.0.1.2 Location Update for MS2 X

6.1 Account Top-Up via IVR X Announcement is recorded for offline-analysis

6.2 Account Top-Up via USSD X

6.3 Account Top-Up via SMS X Received SMS stored for offline-analysis

6.4 Account Enquiry via IVR X Announcement is recorded for offline-analysis

6.5 Account Enquiry via USSD X

6.6 Account Enquiry via SMS X Received SMS stored for offline-analysis

6.7.1 SMS MO X

6.7.2 SMS MO barred by HPLMN X

6.8 SMS MT X

6.9.1.1 MOC supported (VPLMN Number)

X

6.9.1.2 MOC supported (HPLMN Number)

X

6.9.1.3 MOC supported (Voice Mail) X

6.9.1.4 MOC supported (special important short code)

X

6.9.1.5 MOC supported (free phone number)

X

6.9.1.6 MOC supported (xxx) X

6.9.2 MOC barred by HPLMN X




yes no

6.10.1.1 Call setup using Call-back solution (VPLMN Number)

X

6.10.1.2 Call setup using Call-back solution (HPLMN Number)

X

6.10.1.3 Call setup using Call-back solution (Voice Mail)

X

6.10.1.4 Call setup using Call-back solution (special important short code)

X

6.10.1.5 Call setup using Call-back solution (free phone number)

X

6.10.1.6 Call setup using Call-back solution (xxx)

X

6.11 CS-WAP Access X

6.12 MTC X

6.13 Call Forwarding on Busy (CFB)

X

6.14 CFNRc (no paging response)

X

6.15 Call Forwarding on No Reply (CFNRy)

X

6.16 EMC (VPLMN) (X)

Emergency calls have not been automated, but a test case is provided for a call to any external number. The user is responsible for choosing emergency number.

6.17 MOC: Account depleted X

6.18 Insufficient Balance MOC X

6.19 Insufficient Balance SMS MO

X

7 GPRS Test Cases

7.1 GPRS WAP Access X

7.2 GPRS Access to a specific application

X ftp and http

7.3 GPRS: Account deleted X Not yet covered; it is under investigation whether this test can be automated.

7.4 GPRS: Insufficient balance X

7.5 GPRS: SMS MO X A Side not for UMTS




yes no

7.6 GPRS: Insufficient Balance SMS MO

X A Side not for UMTS

3.10 Service Specific Test Cases

Test cases which shall cover specific features of other services than basic call,

supplementary services and data services (IREG 24, 26, 27), CAMEL tests out of

IR.32, GPRS tests out of IR.35 and prepaid service tests out of IR.60 are to be defined

and can be executed similar to the test concept written in chapter 2.1 and 2.5.



4 History

Version Date Author(s) Note

1.9 27.06.07 Adil Kaya, Matthias Menke, Joachim Heinrich

Initial version and 1.1 – 1.9 - no details – no history

2.0 16.07.08 Gesine Woellert IR50 test cases

2.1 06.02.09 Matthias Menke Adaption to SITE 2.1.9 Implementation, e.g.: New IR.27 Tests (VT); Limitations (USSD for GSM_A) removed.

3.0 02.03.10 Matthias Menke Adaption to SITE 2.3 Implementation, e.g.: Adaptions of IR24, IR35, IR60 to newest versions.

3.1 02.03.10 Matthias Menke Adaptions to SITE 2.7_02 of IR35 to newest version 5.0.

Documents

Keynote SIGOS Test Concept for Roaming