Cck Report on the Quality of Service 2012-2013

Page | 1

REPORT ON THE QUALITY OF SERVICE (QOS) PERFORMANCE ASSESSMENT FOR CELLULAR

MOBILE NETWORKS

PERIOD 2012/2013

Page | 2

1. BACKGROUND

Section 23 of the Kenya Information and Communication Act, Cap 411A, mandates the Commission to ensure provision of

good quality services by licensed telecommunication operators and service providers. The obligation to provide quality

services is further reinforced by the respective licences under the quality of service obligation.

The QoS assessment framework is based on eight (8) Key Performance Indicators (KPIs) and a well-defined assessment

methodology that were adopted in 2008/09 after an elaborate and exhaustive consultative process. The current assessment

framework uses the enhanced KPIs that were to be applicable three years after the adoption of the framework. Five out of

the eight parameters were enhanced in the current period as indicated in Table 1.

Table 1: QoS Performance Targets

No. KPI TARGETS

Immediate

(From 2009)

After 3 years

(i.e. from 2012)

1. Completed Calls 90% 95%

2. Call Drop Rate 2% 2%

3. Call Block Rate 10% 5%

4. Call Set Up Time 13.5 seconds 13.5 seconds

5. Call Set Up success

Rate (CSSR)

90%

95%

Page | 3

No. KPI TARGETS

Immediate

(From 2009)

After 3 years

(i.e. from 2012)

6. Handover Success Rate 85% 90%

7. Speech Quality MOS for 95% of samples > 2.7 95% of samples >3.1

8. Rx Lev Outdoor = - 102 dBm Outdoor = - 102 dBm

Indoor = -95 dBm Indoor = -95 dBm

In car = - 100 dBm In car = - 100 dBm

The Commission continued to carry out mobile QoS monitoring and assessment during the period 2012/2013 in line with

the above stated mandate in order to ensure provision of good quality mobile services. The results are used to not only

assess the level of compliance in the current period, but also to determine if adequate remedial measures based on the

previous period assessment results have been undertaken by operators.

1.1. Assessment Criteria

As done in the previous years, the measurements for 2012/2013 are also based on intra-network mobile to mobile

configuration for all the operators. This means that the assessment is based exclusively on the network components of the

particular operator (i.e. end-to-end assessment).

1.2. Summary of overall performance for 2012/13

The overall performance of the respective mobile operators in the 2012/13 financial year is summarized in Table 2 below.

Page | 4

Table 2: All Operators Performance Rating 2012-2013

Name of

Network

Operator

No of

Assessed QoS

parameters

No. of

Parameters

Complied with

Minimum %

required to

attain overall

compliance

Achieved %

Compliance level

Overall

Compliance Status

Safaricom (K)

Ltd

8 4 80% 50% Non- Compliant

Airtel Kenya

Ltd


Essar Telecom

(K) Ltd


Telkom Kenya

Ltd (TKL)

8 5 80% 62.5% Non- Compliant

1.3. Summary of overall performance for all the operators from 2009 to 2013

The graphs below show performance trends of each operator over the four-year period starting in 2009.

As depicted by the graphs, Safaricom started off with poor performance but later on peaked up in the second assessment

period (2010-2011) before declining. In the 2011/2012 financial year, Safaricoms performance declined significantly

and has since remained at 50% in the last two assessment periods.

Page | 5

Airtel has depicted a steady decline from the onset as shown by the graph above.

37.5

75

50 50

0

10

20

30

40

50

60

70

80

2009-2010 2010-2011 2011-2012 2012-2013

Safaricom

safaricom

87.5

75

62.5

50

0

10

20

30

40

50

60

70

80

90

100

2009-2010 2010-2011 2011-2012 2012-2013

Airtel

Airtel

Page | 6

Essar and TKL have depicted a kinked trend from the last assessment. The first two assessments showed steady

improvement, with the two mobile operators meeting the minimum 80% compliance target in the third assessment.

However, in the current assessment period, the performance of the two companies has significantly dropped below target.

Despite being below target, TKL recorded a better performance compared to the rest of the networks in the current

assessment period.

50

75

87.5

50

0

20

40

60

80

100

2009-2010 2010-2011 2011-2012 2012-2013

Essar

Essar37.5

50

87.5

62.5

0

20

40

60

80

100

2009-2010 2010-2011 2011-2012 2012-2013

TKL

TKL

Page | 7

2. INDIVIDUAL OPERATORS COUNTRYWIDE PERFORMANCE ASSESSMENT RESULTS

The summary of the operators overall performance arising from the specific quality of service parameters and targets

are summarized in the tables below.

2.1. Average Overall Performance and Summary of parametric performance

The overall performance in the period under review shows a general downward trend with none of the operators

achieving the minimum KPI compliance target of 80%. It is also important to note that five (5) of the eight (8) KPIs

were enhanced during the period under review and this could have contributed to the outcome.

2.1.1. Safaricom Kenya Limited

Safaricom Kenya Limited registered a QoS performance of 50.0% as was the case in the previous assessment period.

The mobile network operator (i.e. Safaricom) did not meet the target in any of the regions assessed. The best

performance by Safaricom was in Nairobi at 75% which showed significant improvement from the previous period at

50%. In Central, Rift Valley, Western and Nyanza regions, Safaricom showed significant improvement in the current

assessment period moving from 37.5% in the previous period to 50%. The registered QoS performance in Eastern,

North Rift and North Eastern regions was at 37.5%.

Page | 8

Table 3: Safaricoms Overall Compliance Result

No. PARAMETER TARGET PERFORMANCE COMPLIANCE

STATUS

1 Completed Calls 95% 87 Not Compliant

2 Call Drop Rate 2% 1.9 Compliant

3 Call Block Rate 5% 11 Not Compliant

4 CSSR 95% 89 Not Compliant

5 Handover Success Rate 90% 98 Compliant

6

Rx Level (dBm) Outdoor -105 dBm -75

Compliant

7 Speech Quality 95% of samples > 3.1 90 Not Compliant

8 Call Set Up Time < 13.5 Sec 9.2 Compliant

Compliance Level 80% 50% Non-Compliant

Page | 9

2.1.2. Airtel Kenya Limited

Airtel Kenya Limited registered an overall performance of 50% out of the minimum KPI target of 80%. The

networks performance has over the four-year period declined from a compliance rate of 87.5% in the first assessment

period (2009/10) to the current 50% compliance rate.

Airtel recorded good performance in Nairobi at 100% and in Rift Valley at 87.5%. The network had a 75%

compliance rate in the Coast, and 50% in Western, Nyanza and North Eastern regions. A compliance rate of 37.5%

was recorded in Central and North Rift regions.

Table 4: Airtels overall compliance result


STATUS

1 Completed Calls 95%

91 Not Compliant

2 Call Drop rate 2% 1.6 Compliant



5 Handover success rate 90% 95 Compliant

6 Rx Level (dBm) Outdoor -105 dBm -83 Compliant

Page | 10


STATUS




2.1.3. Essar Telecom Kenya Limited

Essar Telecom registered an overall performance of 50% in the assessment period down from the previous periods performance of 87.5%.

Essar Telecom achieved the best QoS performance in the Rift Valley region at 87.5%. This was a significant

improvement considering it was its weakest performance area in the last assessment period recorded at 37.5%. Essar

managed to score 75% in Nairobi and recorded an average performance of 50% in Central, Western, Nyanza and

North Eastern regions.

Its poorest performance was in Eastern and North Rift regions where it scored 37.5% against the set target of 80%.

Page | 11

Table 5: Essars overall compliance result


STATUS










Page | 12

2.1.4. Telkom Kenya Limited

Telkom Kenya Limited met five out of the eight KPIs, recording a performance of 62.50% which is below the 80%

threshold. This marks a decline in performance compared to 87.5% recorded in the previous assessment period. This

notwithstanding, Telkom Kenya was the best performing network in the current assessment period.

Telkom Kenya displayed an outstanding performance in the Rift Valley and Nairobi region scoring 100%.

Table 6: Telkoms overall compliance result


STATUS







7 Speech Quality 95% of samples > 3.1 96 Compliant


Compliance Level 80% 62.5% Non-Compliant

Page | 13

3. COMPARISON OF OPERATORS PER REGION PER PARAMETER (KPI)

3.1. COMPLETED CALLS - 95%

This parameter measures the number of calls that are completed/ connected on a network satisfactorily compared to the

total number of call attempts made by callers. The requirement for completed calls is such that operators should ensure

that at least 95% of all calls are completed. In the previous three years, the threshold was 90%.

Performance on completed calls in most parts of the country was poor, with none of the operators meeting the threshold

in North Rift, North Eastern, Eastern and Central. In Rift Valley, three operators met the threshold of 95% except

Safaricom. In Nairobi region, Airtel and TKL met the set threshold of 95%. In the Coast, Airtel was the only operator

that met the threshold, while Essar was the only operator that met the completed calls threshold in Western and Nyanza

regions. The Graphs below indicate performance in terms of call completion in different regions.

Page | 14

95 96

91

95 95

88899091929394959697

Target Airtel Safaricom TKL Essar

Rift Valley

Target

Airtel

Safaricom

TKL

Essar

95 95.1

91.6

92.9 93.5

89

90

91

92

93

94

95

96


Coast

Target

Airtel

Safaricom

TKL

Essar

95

84.9

87.8

92.9 91.7

78

80

82

84

86

88

90

92

94

96


North Rift

Target

Airtel

Safaricom

TKL

Essar

95

92.8

88.9

92.9 92.7

84

86

88

90

92

94

96


Western & Nyanza

Target

Airtel

Safaricom

TKL

Essar

Page | 15

95

82 80

92.9

77

0

20

40

60

80

100


North Eastern

Target

Airtel

Safaricom

TKL

Essar

95 87

69

90

38

0

20

40

60

80

100


Central

Target

Airtel

Safaricom

TKL

Essar

95

90

93

90

88

84

86

88

90

92

94

96


Eastern

Target

Airtel

Safaricom

TKL

Essar

95

97

94

96

92

89

90

91

92

93

94

95

96

97

98


Nairobi

Target

Airtel

Safaricom

TKL

Essar

Page | 16

3.2. CALL DROP RATE - < 2%

This indicator describes calls that are terminated by the network unexpectedly as a result of technical reasons, including

entry into a dead zone. Call drops may be as a result of poor handoff or lack of network coverage. All the four mobile

operators met the less than 2% requirement for this KPI in Rift Valley, Western, Nyanza and Nairobi. This performance

clearly shows that handoff and coverage are good in these regions.

North Rift and Eastern had the poorest call drop rate on the Essar network at 3.5% and 4%, respectively.

2

0.9

2

0.6

1.6

0

0.5

1

1.5

2

2.5


Rift Valley

Target

Airtel

Safaricom

TKL

Essar

2

2.5

1.7

2.6

1.1

0

0.5

1

1.5

2

2.5

3


Coast

Target

Airtel

Safaricom

TKL

Essar

Page | 17

2

1.6 1.6

1.2

3.5

0

0.5

1

1.5

2

2.5

3

3.5

4


North Rift

Target

Airtel

Safaricom

TKL

Essar

2

1.1

1.4 1.6

1.5

0

0.5

1

1.5

2

2.5


Western & Nyanza

Target

Airtel

Safaricom

TKL

Essar

2

1.3

2.5

1.9

0.7

0

0.5

1

1.5

2

2.5

3


North Eastern

Target

Airtel

Safaricom

TKL

Essar

2

2.4

2 2

0.9

0

0.5

1

1.5

2

2.5

3


Central

Target

Airtel

Safaricom

TKL

Essar

Page | 18

3.3. CALL BLOCK RATE - < 5%

When a call is not completed, it is reported as a blocked call. The requirement is that less than 5% of calls on the

network should fail to have a conversation. Nairobi and the North Rift are the only regions that showed a low block rate.

In this assessment period, the overall performance with respect to this parameter was poor with none of the operators

attaining the minimum threshold. TKL showed the highest blocked call rate at 25% in North Eastern region compared to

the 5% requirement. In Central, Safaricom and Essars call block rate were recorded at 20% and 61%, respectively.

Essars performance in Central was the poorest in all the regions in terms of blocked calls. In the Rift Valley, three out of

the four operators met the threshold. Nairobi was the only region where the blocked call rate was compliant across the

four networks.

2

2.5 2.7

1.7

4

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5


Eastern

Target

Airtel

Safaricom

TKL

Essar

2

0.8

1.6

1.3

1.9

0

0.5

1

1.5

2

2.5


Nairobi

Target

Airtel

Safaricom

TKL

Essar

Page | 19

3

6

4

3

5

0

1

2

3

4

5

6

7

Airtel Safaricom TKL Essar target

Rift Valley

Airtel

Safaricom

TKL

Essar

target

3

6.7 6.3

7.1

5

0

1

2

3

4

5

6

7

8

Airtel Safaricom TKL Essar Target

Coast

Airtel

Safaricom

TKL

Essar

Target

13.5 11.4

8.3

17

5

0

5

10

15

20

North Rift

Airtel

Safaricom

TKL

Essar

Target

5.1

9.2

5.4 6.3

5

0

2

4

6

8

10


Western & Nyanza

Airtel

Safaricom

TKL

Essar

Target

Page | 20

15 17

25

10

5

0

5

10

15

20

25

30


North Eastern

Airtel

Safaricom

TKL

Essar

Target9

29

8

61

5

0

10

20

30

40

50

60

70


Central

Airtel

Safaricom

TKL

Essar

Target

4.9

10 9

8

5

0

2

4

6

8

10

12


Eastern

Airtel

Safaricom

TKL

Essar

Target

2.3

4

2.8

1.4

5

0

1

2

3

4

5

6


Nairobi

Airtel

Safaricom

TKL

Essar

Target

Page | 21

3.4. CALL SET UP SUCCESS RATE (CSSR) 95%

This KPI measures the number of times a consumer tries to make a call that results in a connection to the dialed number.

It is considered one of the most important KPIs because it reflects the consumers Quality of Experience (QoE) when

making a call. The threshold for this KPI is 95%. In the North Rift, North Eastern and Central regions, it is apparent

from the tables below that consumers in these areas had to make several attempts to get a call through, as none of the

operators met the 95% threshold in the current assessment period.

As discernible from the graphs below, Airtel is the only network that had good call set up success rates as shown in the

regions of Rift Valley, Coast Western and Nyanza, Nairobi and Eastern. TKL and Safaricom met the threshold in

Nairobi, and Essar managed to provide a good call success rate in the Rift Valley.

95

97

93.3

96

97

91 92 93 94 95 96 97 98

target

Airtel

Safaricom

TKL

Essar

Rift Valley

target

Airtel

Safaricom

TKL

Essar95

97

93.2

93.7

93

91 92 93 94 95 96 97 98

target

Airtel

Safaricom

TKL

Essar

Coast

target

Airtel

Safaricom

TKL

Essar

Page | 22

95

86.6

88.6

91.7

83.4

75 80 85 90 95 100

target

Airtel

Safaricom

TKL

Essar

North Rift

target

Airtel

Safaricom

TKL

Essar

95

95

90.9

94.7

93.8

88 89 90 91 92 93 94 95 96

target

Airtel

Safaricom

TKL

Essar

Western & Nyanza

target

Airtel

Safaricom

TKL

Essar

95

85

83

76

90

0 20 40 60 80 100

target

Airtel

Safaricom

TKL

Essar

North Eastern

target

Airtel

Safaricom

TKL

Essar

95

91

71

92

39

0 20 40 60 80 100

target

Airtel

Safaricom

TKL

Essar

Central

target

Airtel

Safaricom

TKL

Essar

Page | 23

3.5. CALL SET UP TIME - 13.5 Sec

Call set up time is the period of time between the end of dialing of a telephone call and the start of voice or data

transmission. All the operators met the target for this KPI in all regions.

95

95

90

92

92

86 88 90 92 94 96

target

Airtel

Safaricom

TKL

Essar

Eastern

target

Airtel

Safaricom

TKL

Essar95

98

96

98

96

93 94 95 96 97 98 99

target

Airtel

Safaricom

TKL

Essar

Nairobi

target

Airtel

Safaricom

TKL

Essar

Page | 24

13.5

8.4

9.2

8.3

9.2

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

Rift Valley

target

Airtel

Safaricom

TKL

Essar

13.5

5.3

8.9

7.6

9

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

Coast

target

Airtel

Safaricom

TKL

Essar

13.5

5.2

9.2

8.1

9.6

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

North Rift

target

Airtel

Safaricom

TKL

Essar

13.5

5.1

9.4

7.2

9.5

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

Western & Nyanza

target

Airtel

Safaricom

TKL

Essar

Page | 25

13.5

8.2

9.7

9.4

9.8

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

North Eastern

target

Airtel

Safaricom

TKL

Essar

13.5

8.9

8.9

8.4

9.7

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

Central

target

Airtel

Safaricom

TKL

Essar

13.5

8.3

8.8

8.7

8.7

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

Eastern

target

Airtel

Safaricom

TKL

Essar13.5

8.2

8.8

7

8.6

0 5 10 15

target

Airtel

Safaricom

TKL

Essar

Nairobi

target

Airtel

Safaricom

TKL

Essar

Page | 26

3.6. HANDOVER SUCCESS RATE 90%

Call handover occurs when a mobile handset moves out of one cell to the next and is handed over automatically from the

base station of the first cell to that of the next with no discernible disruption of the call. 90% of calls are expected to be

successfully handed over.

In this assessment period, save for TKL in North Eastern and Airtel in North Rift regions, all the operators met the target

of 90% in all the other regions.

90

100 98 98

96

8486889092949698

100102

Nairobi

target

Airtel

Safaricom

TKL

Essar

90

96

98 98 99

84

86

88

90

92

94

96

98

100

target Airtel Safaricom TKL Essar

Eastern

target

Airtel

Safaricom

TKL

Essar

Page | 27

90

93

100

98

90

84

86

88

90

92

94

96

98

100

102


Central

target

Airtel

Safaricom

TKL

Essar

90

100

97

87

94

80

85

90

95

100

105


North Eastern

target

Airtel

Safaricom

TKL

Essar

90

95

98 99

97

84

86

88

90

92

94

96

98

100


Western & Nyanza

target

Airtel

Safaricom

TKL

Essar

90 89

97

100 99

82

84

86

88

90

92

94

96

98

100

102


North Rift

target

Airtel

Safaricom

TKL

Essar

Page | 28

3.7. SPEECH QUALITY (MOS) 95% OF SAMPLES >3.1

This KPI can best be described as the clarity of speech delivered during a call (without noise/echo/interference).

In the period under review, only Telkom Kenya Limited met the threshold of 95% in all regions as measured in terms of

Mean Opinion Score (MoS). Airtel was only compliant in Nairobi while Essar and Safaricom were not compliant in any

of the assessment regions in relation to speech quality.

90

94

99 99 99

84

86

88

90

92

94

96

98

100


Coast

target

Airtel

Safaricom

TKL

Essar90

95

100 100

98

84

86

88

90

92

94

96

98

100

102


Rift Valley

target

Airtel

Safaricom

TKL

Essar

Page | 29

90

74

93 97

57

0

20

40

60

80

100

120


Rift Valley

target

Airtel

Safaricom

TKL

Essar

90 81.5

88.9 96.5

48

0

20

40

60

80

100

120


Coast

target

Airtel

Safaricom

TKL

Essar

90 82.2

86.8 95.9

44.2

0

20

40

60

80

100

120


North Rift

target

Airtel

Safaricom

TKL

Essar

90 81.8

89.6 95

48.9

0

10

20

30

40

50

60

70

80

90

100


Western & Nyanza

target

Airtel

Safaricom

TKL

Essar

Page | 30

90 90 89

96

87

82

84

86

88

90

92

94

96

98


North Eastern

target

Airtel

Safaricom

TKL

Essar

90 85 89

96

78

0

20

40

60

80

100

120


Central

target

Airtel

Safaricom

TKL

Essar

90 83 83

95

74

0

20

40

60

80

100


Eastern

target

Airtel

Safaricom

TKL

Essar

90 97 93 95

76

0

20

40

60

80

100

120


Nairobi

target

Airtel

Safaricom

TKL

Essar

Page | 31

3.8. RX Level outdoors - 105dBm

The Rx level is the power in decibels (dBm) of the received signal. The target for this parameter is -102dBm.

. If the Rx level (or signal strength) is too high i.e. -105dBm, signal distortion may occur, leading to erratic connections.

All the operators met the target in this assessment period.

-105

-82 -74 -75 -78

-120

-100

-80

-60

-40

-20

0


Rift Valley


-105

-84 -73

-79 -82

-120

-100

-80

-60

-40

-20

0


Coast


Page | 32

-105

-85 -73 -77 -77

-120

-100

-80

-60

-40

-20

0


North Rift


-105

-83 -77

-70 -72

-120

-100

-80

-60

-40

-20

0


Western & Nyanza


-105

-81 -75

-66

-95

-120

-100

-80

-60

-40

-20

0


North Eastern


-105

-83 -74 -77 -79

-120

-100

-80

-60

-40

-20

0


Central


Page | 33

4. CONCLUSION

In the 2012-2013 assessment period, Telkom Kenya Limited had the highest QoS score of 62.5%, while the three (3) other

operators (i.e. Airtel Kenya Limited, Essar Kenya Limited and Safaricom Kenya Limited) were rated at 50% out of the set

minimum score of80%. Overall, all four (4) mobile operators failed to meet the compliance target of 80%.

Overall, all the operators met the KPI targets in respect to RX Level, Call set up time, Handover Success Rate and Call drop

rate in the current assessment period.

The KPIs that have not been met by any of the operators in the current assessment period included call block rate,

completed calls and call set up success rate (CSSR). The Commission will continue to engage the operators with a view to

ensuring quality services are provided to the consumer. In addition, we are working on a number of regulatory measures to

improve mobile cellular quality of services and it is hoped this will be reflected in the next report.

-105

-84 -75

-68 -78

-120

-100

-80

-60

-40

-20

0


Eastern


-105

-80 -75

-81 -74

-120

-100

-80

-60

-40

-20

0


Nairobi


Documents

Cck Report on the Quality of Service 2012-2013