Download pdf - Government Data Quality Findings Recommendations quality review - Compendium Report .pdf · Compendium of Government Data Quality Findings & Recommendations 9 integrity of the data

Government Data Quality Findings &

Recommendations:

A Compendium

Final Draft

28 November 2014

Nils Riemenschneider & Jesse McConnell

Oxford Policy Management

Commissioned by the Millennium Challenge Account Namibia

with funding from the Millennium Challenge Corporation

Bi‐Annual Ex‐Post Data Quality Review

Table of Contents

List of tables .................................................................................................................. 3

List of figures ................................................................................................................. 4

List of acronyms ............................................................................................................. 5

Preface .......................................................................................................................... 6

1. Introduction ........................................................................................................... 7

1.1. Structure of the Report ......................................................................................... 7

1.2. Data Sources Reviewed ......................................................................................... 8

1.3. Data Quality Review Methods ............................................................................... 8

2. Goal Indicators ..................................................................................................... 10

2.1. Namibia Household Income and Expenditure Survey (Namibia Statistics Agency)

............................................................................................................................ 10

2.2. Namibia Labour Force Survey (Namibia Statistics Agency) ................................. 12

3. Education Indicators ............................................................................................. 15

3.1. Education Management Information System (Ministry of Education) ............... 15

3.2. Directorate of National Examination and Assessment (Ministry of Education) .. 17

3.3. Education Administrative Data (Ministry of Education) ...................................... 19

3.4. Vocational Education Administrative Data & VETMIS (Namibia Training

Authority) ............................................................................................................ 20

4. Tourism Indicators ................................................................................................ 23

4.1. Foreign Arrival Records (Ministry of Environment and Tourism) ........................ 23

4.2. Etosha National Park Administrative Data (Ministry of Environment and

Tourism) .............................................................................................................. 24

4.3. Tourism Jobs Model (Namibia Tourism Board) ................................................... 28

4.4. Google Analytics (Namibia Tourism Board) ......................................................... 29

4.5. Bed Levy Statistics administrative data (Namibia Tourism Board) ..................... 31

5. Agriculture Indicators ........................................................................................... 33

5.1. System for Livestock Health Statistics (Ministry of Agriculture, Water and

Forestry) .............................................................................................................. 33

5.2. Formal Cattle Slaughter Data (Meat Corporation of Namibia) ........................... 35

5.3. Namibia Communal Land Administration System (Ministry of Lands and

Resettlement) ..................................................................................................... 36

Annex 1: Data sources and corresponding reviews ....................................................... 37

Annex 2: List of documents consulted .......................................................................... 38

Annex 3: International Data Quality Standards ............................................................ 39

3.1 IMF Data Quality Assessment Framework .......................................................... 39

3.2 UN Fundamental Principles of Official Statistics ................................................. 40

Annex 4: Supporting documentation ........................................................................... 41

Compendium of Government Data Quality Findings & Recommendations

3

List of tables

Table 1: Data sources reviewed 8

Table 2: MCA‐N related indicators: goal (NHIES) 11

Table 3: MCA‐N related indicators: goal (NLFS) 12

Table 4: MCA‐N related indicators: education (EMIS) 15

Table 5: MCA‐N related indicators: education (DNEA) 18

Table 6: MCA‐N related indicators: education (MoE) 19

Table 7: MCA‐N related indicators: education (NTA) 21

Table 8: MCA‐N related indicators: tourism (Arrivals Statistics – MET) 23

Table 9: MCA‐N related indicators: tourism (ENP administrative data – MET) 25

Table 10: MCA‐N related indicators: tourism (jobs model – NTB) 29

Table 11: MCA‐N related indicators: tourism (Google Analytics – NTB) 30

Table 12: MCA‐N related indicators: tourism (bed levy statistics – NTB) 31

Table 13: MCA‐N related indicators: agriculture (DVS – MAWF) 33

Table 14: MCA‐N related indicators: agriculture (Meatco) 35

Table 15: MCA‐N related indicators: agriculture (NCLAS – MLR) 36

Table 16: Data source and corresponding reviews 37


4

List of figures

Figure 1: Galton Gate – Yellow Book & white forms 27

Figure 2: Visitor data record sheet (Yellow Book) – Galton Gate 42

Figure 3: Visitor data record sheet (white form) – Anderson Gate 43

Figure 4: Covering Advice, 23 May 2014 44

Figure 5: Tourism Levy Return & Statistics Form 45

Figure 6: Google Analytics – visits to NTB website (sample taken from Oct‐Dec 2013) 46

Figure 7: Poster by MLR to communicate the process around land registration 47


5

List of acronyms

AEC Annual Education Census

BAR Bi‐Annual Review

DVS Directorate of Veterinary Services in the Ministry of Agriculture, Water and

Forestry

DNEA Directorate of National Examination and Assessment in the Ministry of Education

DoT Directorate of Tourism and Gaming

DQAF Data Quality Assessment Framework

DQR Data Quality Review

EMIS Education Management Information System

GDQR Government Data Quality Review

GRN Government of the Republic of Namibia

IMF International Monetary Fund

JSC Junior Secondary Certificate

MAWF Ministry of Agriculture, Water and Forestry

MCA‐N Millennium Challenge Account Namibia

M&E Monitoring and Evaluation

MoE Ministry of Education

MLR Ministry of Lands and Resettlement

NCLAS Namibia Communal Land Administration System

NHIES Namibia Household Income and Expenditure Survey

NLFS Namibia Labour Force Survey

NPC National Planning Commission

NSA Namibia Statistics Agency

NSSCHL Namibia Senior Secondary Certificate Higher Level

NSSCOL Namibia Senior Secondary Certificate Ordinary Level

OPM Oxford Policy Management

SSC Senior Secondary Certificate


6

Preface

The Millennium Challenge Account Namibia (MCA‐N) has contracted Oxford Policy

Management (OPM) to provide consultancy services for the Data Quality Review (DQR) of

different MCA‐N monitoring and evaluation (M&E) indicators. The DQRs occurred over the

period 2010 to 2014. As the culmination of this process, this Compendium Report provides a

summation of all findings and recommendations related to government data sources

included in all preceding data quality reviews conducted by OPM for MCA‐N. The report

draws on those previous reviews, and is targeted towards highlighting key considerations in

the future use of the data sources, e.g., in national M&E frameworks like the National

Planning Commission’s monitoring of achievement towards Vision 2030 and National

Development Plan objectives.


7

1. Introduction

This report aims to synthesize the findings from various data quality reviews (DQRs) of

Namibian government data sources. These DQRs were conducted from 2010 to 2014 and

have been documented in various reports. The synthesis targets government data sources

that formed part of MCA‐N’s monitoring and evaluation (M&E) framework during its

implementation of the Compact between the Government of the Republic of Namibia (GRN)

and the Government of the United States (represented by the Millennium Challenge

Corporation (MCC)). The report provides an overview of previous reviews of the data

sources, a snapshot of the current or most recent status on the reported data sources, and

considerations for future use and/or improvement of those data sources.

A related workshop that took place on 18‐19 November 2014 disseminated these

consolidated findings and recommendations to key government stakeholders – particularly

the National Planning Commission’s (NPC’s) M&E unit and the Namibia Statistics Agency

(NSA) – to ensure that GRN is well aware of the issues; the workshop also had a training

component to give the relevant institutions solid grounding for conducting or managing their

own follow‐up or new DQRs.

The DQR period for which OPM was contracted by MCA‐N extended from 2010 to 2014, and

the findings related to government sources have been documented in the following reports,

which also serve as the key source documents to this report:

i. Government Data Quality Review (2011)

ii. Bi‐Annual Review I (2012)

iii. Bi‐Annual Review II (2012)

iv. Bi‐Annual Review III (2014)

v. Data Quality Review Follow‐Up: Education Management Information System (EMIS)

(2014)

vi. Data Quality Review Follow‐Up: Vocational Education and Training Management

Information System (VETMIS) (2014)

vii. Support for High Quality Data Reporting: Quarter 19 (Q19) (2014)

Each of these reports are referenced within the text of this report. They are sources of

further details underlying the processes of data collection that pertain to the relevant

indicators, should the reader wish to delve into those details. This report itself seeks only to

highlight key issues for follow‐up based on the findings and recommendations from the

previous reviews.

1.1. Structure of the Report

The general structure of the report is based around the MCA‐N Compact, beginning with the

overall Goal indicators, followed by indicators falling under its three Projects: Education,

Tourism and Agriculture. Under each Project, the data source(s) included in previous reviews

are discussed, beginning with a brief overview of the data source itself. This is followed by


8

relating it to the specific MCA‐N indicators that the data sources were used to monitor,

followed then with a review history, including key findings and recommendations. It

concludes with considerations for further monitoring of the data source.

Each sub‐section tabulates the MCA‐N indicators related to the data sources. The narrative

in the sub‐section provides a chronological overview of the data source, including the

processes that it underwent in its review history, in which the evolution of the indicator and

data source(s) are briefly described.

1.2. Data Sources Reviewed

The specific data sources that were reviewed and included in this report relate to

government‐specific data sources. They are listed below, according to the MCA‐N Project

that they pertain to, with their institutional affiliation provided in parentheses1:

Table 1: Data sources reviewed Goal Indicators

1. Namibia Household Income and Expenditure Survey (NSA)

2. Namibia Labour Force Survey (NSA)

Education Indicators

3. Education Management Information System (MoE)

4. Examination results from the Directorate of National Examinations and

Assessment (MoE)

5. Administrative data from the NTA

6. Administrative data from MoE

Tourism Indicators

7. Foreign Arrival Records (MET)

8. Etosha National Park administrative data (MET)

9. Tourism Jobs Model (NTB)

10. Google Analytics (NTB)

11. Bed Levy Statistics (NTB)

Agriculture Indicators

12. System for Livestock Health Statistics (MAWF)

13. Formal Cattle Slaughter Data (MEATCO)

14. Namibia Communal Land Administration System (MLR)

1.3. Data Quality Review Methods

The data quality reviews conducted by OPM were, at their premise, aimed at understanding

in detail the processes that underpin data collection for the reviewed data sources. In the

process of understanding how data are collected and reported on, the DQR Team sought to

identify where weaknesses in the process might exist that would challenge the quality and

1 Please see Annex 1 for a tabulation of the data sources together with their related data quality reviews


9

integrity of the data being collected or, in cases where the process was considered

sufficiently robust to result in solid data, to deem the data source ‘fit for purpose’.

The methods used for undertaking the data quality reviews varied according to the data

sources, necessarily adapting to the processes that the data sources themselves entailed.

This was an important feature of the data quality reviews, given the diversity of the data

sources that were reviewed, as well as the broad period of time over which the DQRs were

conducted. In general, the reviews sought to comply with the internationally accepted

standards on statistical and data quality, such as the ‘prerequisites of quality’ outlined in the

International Monetary Fund (IMF) Data Quality Assessment Framework (DQAF)2 and the 10

Fundamental Principles of Official Statistics outlined by the UN.3

The reviews generally included some form of document review at the outset, followed by a

formulation of key questions resulting from the document review, which could then be

taken into field and raised during consultations with the data collectors for each source.

These stakeholder consultations were generally conducted in‐field at the data collectors’

area of work, and included not only discussions about the data collection processes, but –

when possible – observations of the data collection processes themselves.

Often follow‐up consultations were also held by way of clarifying previous findings in order

to ensure accurate understanding of the processes and reasonable premises for further

recommendations. The findings from the review were then written in the data quality

review reports (listed above) which were then submitted to the MCA‐N M&E unit, which

circulated them among the relevant data producers, NPC M&E staff and/or sector teams

responsible for the implementation and management of its projects. The input provided

formed part of an iterative process in the finalisation of the recommendations. The objective

was to identify recommendations that were pragmatic and viable for implementation

which were also aligned with the overall objectives of the MCA‐N Compact.

2 International Monetary Fund, Data Quality Assessment Framework (2012)

3 United Nations, Fundamental Principles of Official Statistics (1994)


10

2. Goal Indicators

The Goal indicators in MCA‐N’s M&E Plan are drawn from Namibia’s Vision 2030 and the

National Development Plan 3, with the view that the MCA‐N program would contribute to

achieving national objectives.4 However, MCA‐N’s intervention is noted as not being of

sufficient scale to independently achieve these goals at the national level. The data sources

that informed the Goal indicators included in this compendium report include the Namibia

Household Income and Expenditure Survey (NHIES) and the Namibia Labour Force Survey

(NLFS), both conducted by the Namibia Statistics Agency (NSA) (which was formerly the

Central Bureau of Statistics in the National Planning Commission).

2.1. Namibia Household Income and Expenditure Survey (Namibia Statistics Agency)

Overview of the data source

The Namibia Household Income and Expenditure Survey (NHIES) was conducted most

recently in 2009/2010. Previous versions of the survey having been conducted in 1993/1994

and 2003/2004.5 As the 2009/2010 NHIES report states, the NHIES is ‘a survey collecting data

on income, consumption and expenditure patterns of households, in accordance with

methodological principles of statistical enquiries, which are linked to demographic and

socio‐economic characteristics of households.’6 The survey is managed and implemented by

the Namibia Statistics Agency (NSA) every 5 years. An initial interval period of 10 years was

seen to be too long. Given these methodological and statistical mandates, the NHIES was

deemed as an appropriate data source for several goal‐level indicators for MCA‐N.

MCA‐N related indicators

For MCA‐N’s critical Goal indicators – those that monitor overall impact of the Compact –

the three indicators were informed by the NHIES. These indicators bear the following high‐

impact goals: reduce the poverty rate, reduce the unemployment rate, and increase the

median household income. These indicators are listed below, together with their definitions,

data sources, and responsible parties:7

4 Oxford Policy Management, Government Data Quality Review (2011), p25 5 Namibia Statistics Agency, Namibia Household Income and Expenditure Survey 2009/2010, piv 6 NSA, NHIES 2009/2010, piv

7 Data sources listed in parentheses indicate original data sources that changed to those currently listed as the main data source in the same column.


11

Table 2: MCA‐N related indicators: goal (NHIES)8

Indicator Definition Data Source

Responsible Party

Poverty Rate

The cost of a food basket enabling households to meet a minimum nutritional requirement plus an allowance for the consumption of basic non‐food items. Households with consumption expenditure in excess of this threshold are considered non‐poor and households with expenditure less than the threshold are considered poor

NHIES NSA

Unemployment Rate The percentage of the economically active population that is currently unemployed

NHIES(NLFS)

NSA

Median Household Income

The sum of total consumption and non‐consumption expenditures. Savings are not included

NHIES NSA

Review history: key findings & recommendations

The first review for these indicators and related data sources was in the Government Data

Quality Review (GDQR). At that point the 2009/2010 version of the NHIES was not yet

available. Thus the 2003/2004 version formed the basis for the review. The NHIES originally

provided data for two MCA‐N indicators – poverty rate and median household income – as

noted in the table above. However, it was later recommended to also serve as the data

source for the unemployment rate indicator, due to problems found with the original source

(see section 2.2 of this report).

The overall findings of the NHIES review were that the survey and report were ‘generally of

high quality and its results...reliable’.9 Minor recommendations were provided for areas

where the survey could be improved. These included points such as reduced number of

households per enumeration area; improving the sample size per region so that the sample

can be disaggregated at a regional level; calculating sampling errors for key variables;

making minor improvements to the questionnaire; using price deflators to standardise costs

in estimating value of total consumption for households; calculating poverty based on the

cost of basic needs (not by food shares); using double data‐entry for certain complex

sections of a question in order to reduce data capture errors where double‐entry is not

possible for the full questionnaire.

Considerations going forward

Future considerations that should be taken into account related to the NHIES should firstly

ensure that the survey maintains comparability over time. While certain recommendations

have been provided in the reviews, these are somewhat minor issues that should not detract

from the replicability of the survey. Being able to replicate the survey is vital to make

8 Millennium Challenge Account Namibia, Monitoring and Evaluation Plan (2014), Annex 1, p1 (Please note that all references to the MCA‐N M&E Plan relate to the prevailing Plan at the time of the referenced data quality review unless currently referenced, in which case the most recent and formally referenced version (2014) applies.) 9 OPM, GDQR, p42


12

comparisons over time. This will help to accurately measure change and monitor Namibia’s

critical socio‐economic development indicators.

Other matters for further improvement include that data should be made more timeously

accessible to users. The 2003/2004 report only became available more than two years after

fieldwork ended, and the poverty report four years after fieldwork ended. More timely

access to the data is vital for user confidence in the data and for informing decision‐making

processes more substantially.10

2.2. Namibia Labour Force Survey (Namibia Statistics Agency)


The 2004 Namibia Labour Force Survey (NLFS) was the third of its kind conducted since

independence, and is a nationally representative household‐based sample survey, with the

objectives ‘to measure the extent of available and unused labour time and human resources’

and ‘to measure the relationship between employment and other socio‐economic

characteristics’.11

The NLFS is specifically targeted towards informing critical policy decisions and programmes

that aim at building the economic base for development in Namibia. In 2004 this was stated

as the survey aiming to provide ‘all the necessary information on employment,

unemployment and underemployment to meet the demands of policy‐makers, analysts and

other institutions for period policy and comprehensive reviews of the employment situation

in the country’.12 In its latest version, the NLFS targets its data towards ‘assessment of labour

market conditions in Namibia.’13 The survey is now conducted annually by the NSA.14

MCA‐N‐related indicators

The NLFS was originally identified as the data source for the goal indicator on

unemployment (though, given the findings of the GDQR, this was recommended to be

changed to the NHIES), tabulated in further detail below:

Table 3: MCA‐N related indicators: goal (NLFS)15


Responsible Party

Unemployment Rate The percentage of the economically active population that is currently unemployed

NHIES(NLFS)

NSA

10 OPM, GDQR, p42‐45

11Namibia Statistics Agency, Namibia Labour Force Survey 2004, p17 12 NSA, NLFS 2004, pi 13 NSA, NLFS 2014, p8

14 NSA, NLFS 2014, p19

15 MCA‐N, M&E Plan, Annex 1, p1


13


The NLFS was first reviewed in the GDQR – along with the NHIES – but without the same

positive outcome.16 The GDQR review (conducted in 2010‐2011) was based on the published

results of the 2004 NLFS – which was compared to the data from the 2003/2004 NHIES

database – as this was the most recent data available at the time of the review.

One of the findings of the review pertaining to this indicator was that significant

discrepancies existed between the two major national surveys – the NLFS and the NHIES –

particularly in the areas of the unemployment rate.17 The review found that there were

‘varying interpretations of the standard concepts and definitions surrounding economic

activity’, particularly those used by the NLFS.18 More specifically, the review noted ‘the

exclusion of subsistence farmers from the economically active population’, reflecting an

over‐inflated level of unemployment (37%) to that of reported in the NHIES (23%).19

Around the time of the GDQR (but not necessarily as the exclusive reason for it), the

Namibian government officials requested the World Bank to further investigate the quality

of the 2008 NLFS data and its estimates of the unemployment rate.

The World Bank document was reviewed by the DQR Team in the Bi‐Annual Review (BAR),

round II report, which formed the second review of the data source (after it was first

reviewed in the GDQR).20 The main question for this review was whether the conclusions of

the World Bank study (looking at the 2008 NLFS) and the GDQR report (looking at the 2004

NLFS) on the reasons for the high estimates of the unemployment rate were consistent. In

addition, the objective was to identify any new findings in the World Bank review that could

lead to additional recommendations.

Although the descriptions of the estimation methodology for the 2004 and 2008 NLFS are

different in the World Bank review and the GDQR, they nevertheless reach similar

conclusions that the over‐estimation of the unemployment rate is partly due to the

treatment of subsistence farming in the survey data. The BAR II submitted a list of

recommendations for the improvement of the NLFS.

MCA‐N recognized these findings by noting in its final M&E Plan (of July 2014) the following

additional information for the unemployment rate indicator: ‘Given the weakness of the

NLFS‐sourced unemployment rate as identified by the Data Quality Review (DQR) consultant

in the Government Data Quality Review and in Round 2 of the bi‐annual ex‐post reviews, the

unemployment rate from the NHIES (which was found to be a reliable source of data) should

be used as a parallel indicator.’21

16 See OPM, GDQR, p46‐59

17 This is of course of particular significant to MCA‐N, given that the NLFS was identified as the data source for the

unemployment indicator. 18 OPM, GDQR, p58 19 OPM, GDQR, p58

20 See Oxford Policy Management, Bi‐Annual Ex‐Post Data Quality Review, Round 2 (2012), p8‐16

21 MCA‐N, M&E Plan, Annex 1, p1


14

The NLFS has since been significantly overhauled, which was recognized by MCA‐N in its

note in the Indicator Tracking Table (in which the organisation reports actual achievement

against indicator targets) that the latest report unemployment rate should be cautiously

interpreted in relation to previous figures given changes in methodology.22


The first concern raised by the reviews is the timely and adequate access to data for users. It

is critical that users be able to have quick access to the data in order to inform relevant

decision‐making processes.

The fact of the previous versions of the NLFS being untimely in their release to the public

appears to have found traction at the highest levels of the NSA, as the latest NLFS report

(2014) indicates: ‘the 2013 Labour Force Survey was conducted with the objective of

generating “timely collection and release of key socio‐economic indicators for assessment of

labour market conditions in Namibia”’.23

Some of the definitional challenges found in the NLFS (i.e. unemployment parameters) also

appear to have been addressed – largely with the revised legal framework under which the

NSA was created and given the mandate of statistics autonomy and national data

production. MCA‐N also reported that the recommendations by the BAR II report had been

implemented.

However, it is still vital that the definitions of key statistics be constantly updated to keep

pace with international standards while also ensuring that the various national surveys

follow similar methodologies in order to ensure comparability and integrity across the

various national data.

22 MCA‐N, Indicator Tracking Tables of July 2014 and September 2014

23 Namibia Statistics Agency, Namibia Labour Force Survey 2013 Report (2014), p8


15

3. Education Indicators

MCA‐N’s Education Project was aimed at contributing to ‘improving the quality of education

and training in Namibia as well as expanding access of underserved groups to education and

vocational training.’24 The data sources included in this compendium report include the

Education Management Information System (EMIS), the Directorate of National Examination

and Assessment (DNEA) and administrative data from the Ministry of Education (MoE), as

well as the Vocational Education Training Management Information System (VETMIS) and

various administrative data from the Namibia Training Authority (NTA).

3.1. Education Management Information System (Ministry of Education)


The Education Management Information System (EMIS), managed by the Ministry of

Education (MoE), was selected as a data source for MCA‐N for the following two main

reasons: i) ‘to capture change in the flow of learners in the school system (by observing

promotion rates and the proportion of new entrants to targets grades); and ii) to measure

change in learners’ access to textbooks (by observing learner to textbook ratios)’.25

MCA‐N related indicators

Within MCA‐N’s Education Project, the Education Management Information System (EMIS)

was identified as the data source for the M&E Plan indicators listed below, together with

their definitions, data sources, and responsible parties:

Table 4: MCA‐N related indicators: education (EMIS)26


Responsible Party

Learner promotion27 Percentage of all learners in grade 5 who were promoted and continued schooling in grade 6 in the year the data are reported

EMIS MoE

Learner promotion Percentage of all learners in grade 7 who were promoted and continued schooling in grade 8 in the year the data are reported

EMIS MoE

New entrant rates28

Percentage of students in grade 5 who are there for the first time (new enrolments or learners who were promoted at the end of the previous year and continued school)

EMIS MoE

New entrant rates

Percentage of students in grade 8 who are there for the first time (new enrolments or learners who were promoted at the end of the previous year and continued school)

EMIS MoE

Learner‐textbook ratio of 1 to 1 disaggregated

Percentage of schools which have a learner‐textbook ratio of 1 to 1 for science,

EMIS MoE

24 OPM, GDQR, p60 25 OPM, GDQR, p60 26 MCA‐N, M&E Plan, Annex I, p3


28 Ibid


16


Responsible Party

by science, maths and English29

mathematics and English books for all grades

Learner‐textbook ratio of 1 to 2 disaggregated by science, maths and English

Percentage of schools which have a learner‐textbook ratio of 1 to 2 for science, mathematics and English books for all grades

EMIS MoE

Learners (any level) participating in the 47 schools sub‐activity

The number of learners enrolled or participating in educational programmes in the 47 schools plus any new schools that split from the original schools and are co‐located.

EMIS MoE

Teacher qualification – 47 schools

% of teachers (in the 47 schools plus any new schools that split from the original schools and are co‐located) who have a teacher qualification of Code 4, 5, or 6 for Professional Qualifications in the Annual Education Census.

EMIS MoE


The EMIS was first reviewed in the GDQR (2011), then the BAR I (2012) and again in the DQR

follow‐up report on EMIS (2014). The first review (GDQR) identified the three indicators that

were currently monitored by EMIS. The indicators aimed to i) ‘capture change in the flow of

learners in the school system (by observing promotion rates and the proportion of new

entrants to targeted grades); ii) change in learners’ access to textbooks (by observing leaner

to textbook ratios); and iii) change in the quality of the school system (by observing the

proportion of learners passing the Junior Secondary Certificate (JSC) and Senior Secondary

Certificate (SSC) exams at a targeted pass rate level)’.30

For the GDQR report, the review included not only a review of the EMIS itself, but also the

data sources that feed into EMIS for the other indicators, including a review of the Annual

Education Census (AEC) and Fifteenth School Day Statistics survey forms (used to collect data

for EMIS), the Education Statistics publications, a report of a survey of textbooks, and the

Manual for Schools for Registers of Orphans and Vulnerable Children.

The EMIS data are based on the AEC, which is conducted in September each year. The EMIS

unit also supports the implementation of the Fifteenth School Day Statistics survey, in which

schools report on enrolment, teachers and other basic information at the beginning of each

academic year.31

As a result of the review, indicators pertaining to learner promotion were noted as facing

challenges to adequately indicate change based on the MCA‐N intervention. This is due to

the fact that AEC forms do not distinguish children that are transferred (without passing) to

the next grade. Moreover, the standards required for children to pass a class can also vary

29 Oxford Policy Management, Bi‐Annual Ex‐Post Data Quality Review, Round 1 (2012), p11

30 OPM, GDQR, p60



17

by school. Hence the indicator was not recommended as a measure for the MCA‐N

intervention.32

The BAR I report reviewed the indicators pertaining to learner‐textbook ratios. It suggested

that the indicators be dropped from the MCA‐N logframe, as the data were unreliable.33 The

review found that the data collected by EMIS did not distinguish between old and new

textbooks, and thus risked including old and outdated textbooks, skewing the ratio.

Similar findings emerged in the DQR Follow‐Up: EMIS report as that of the GDQR. In it, the

capacity constraints noted previously within EMIS were still prevalent. Key roles in statistics

and database management were still vacant, while the data collection forms were also

noted as similarly burdensome and un‐modified in their data requirements.

The GDQR noted significant human resource constraints within EMIS, especially in the area

of statistics and database management. The GDQR recommended that these skills gaps will

be addressed through training and sourcing adequate capacity. The capacity constraints

were further highlighted by the fact that there were plans to decentralise EMIS data entry to

the regional levels, which are even further characterised by inadequate human resources. A

number of the data collection forms for EMIS were also noted as being burdensome to the

regional and local levels where the data are collected. Recommendations were made that

these be streamlined as much as possible, and any superfluous or repeated data

requirements be omitted. Similar findings emerged from the follow‐up DQR exercise.


In spite of the significant capacity constraints that both reviews noted for EMIS, it was

further discovered was that the EMIS management has a well‐developed vision for

developing and implementing a learner tracking system. That system would significantly

streamline the entire data collection process for the Ministry of Education throughout the

country. It would feed more robust, accurate and timely data into EMIS. In order for this

system to proceed towards development, sufficient political will and resources need to be

oriented towards its consideration and development. The potential benefits that such a

system presents make its strong consideration a high recommendation by the DQR Team.

3.2. Directorate of National Examination and Assessment (Ministry of Education)


As a directorate within the Ministry of Education, the main functions of the Directorate of

National Examination and Assessment (DNEA) are ‘to provide a national assessment and

certification service for the school system, to assist in enhancing the quality of education,

and with the monitoring of educational standards’. 34 More specially, the DNEA administers


33 See OPM, BAR I, p11‐16

34 National Examination and Assessment, http://www.dnea.gov.na/en (accessed 15 October 2014)


18

and monitors the standards of the Junior Secondary Certificate (JSC) and the Namibia Senior

Secondary Certificate Ordinary Level (NSSCOL) and the Namibia Senior Secondary Certificate

Higher Level (NSSCHL).

For MCA‐N the DNEA was selected as a data source in order to measure ‘change in the

quality of the school system (by observing the proportion of learners passing the JSC and SSC

exams at a targeted pass rate level)’.35


The DNEA holds data on the national examinations and assessment statistics, which has

been used to report on eight MCA‐N indicators on learner achievement. These are listed

below:

Table 5: MCA‐N related indicators: education (DNEA)36


Responsible Party

Percentage of learners attaining ‘Basic Achievement’ or higher on the Grade 5 NSAT – English – 47 schools

The percentage of learners attaining ‘Basic Achievement’ or higher on the Grade 5 NSAT – English – 47 schools

DNEA MoE

Percentage of learners attaining ‘Basic Achievement’ or higher on the Grade 5 NSAT – Mathematics – 47 schools

The percentage of learners attaining ‘Basic Achievement’ or higher on the Grade 5 NSAT – Mathematics – 47 schools

DNEA MoE

Pass Rate of JSC learners (grade 10) – Math – 47 schools

The percentage of learners achieving D or better in core mathematics (at 45 of the 47 schools that include 10th grade)

DNEA MoE

Pass Rate of JSC learners (grade 10) – Science – 47 schools

The percentage of learners achieving D or better in Physical and Life Science (at 45 of the 47 schools that include 10th grade)

DNEA MoE

Pass Rate of JSC learners (grade 10) – English – 47 schools

The percentage of learners achieving D or better in English as a second language (at 45 of the 47 schools that include 10th grade)

DNEA MoE

Pass Rate of NSSC learners (grade 12) – Math – 47 schools

The percentage of learners achieving D or better in ordinary level mathematics (at 9 of the 47 schools that include 12th grade)

DNEA MoE

Pass Rate of NSSC learners (grade 12) – Science – 47 schools

The percentage of learners achieving D or better in ordinary level Physical Science (at 9 of the 47 schools that include 12th grade)

DNEA MoE

Pass Rate of NSSC learners (grade 12) – English – 47 schools

The percentage of learners achieving D or better in ordinary level English as a second language (at 9 of the 47 schools that include 12th grade)

DNEA MoE

35 OPM, GDQR, p60

36 MCA‐N, M&E Plan, Annex I, p3‐4


19


The DNEA as a data source was reviewed in the GDQR (2011) and in the BAR II (2012). The

findings from the GDQR state simply that the DNEA data were of good quality with no

major concerns.37

In this second review of the DNEA, the key question of the review was whether the DNEA

has ensured that the exams used at grades 10 and 12 were comparable over time to ensure

consistency in the data. The outcome of the review showed that there was indeed an

equating process to ensure comparability of the grades. Thus, similar to the GDQR, no

significant issues were noted with the DNEA as a data source.38


There are no critical issues to follow‐up on with DNEA.

3.3. Education Administrative Data (Ministry of Education)


Administrative data from the Ministry of Education was used for several MCA‐N indicators.

The administrative data are generally routinely collected by processes already in place at the

MoE. The reviews conducted on these data were generally oriented towards understanding

the processes of data collection, rather than being motivated by a specific concern with the

data quality.


Administrative data from the Ministry of Education (MoE) served as a data source on three

MCA‐N indicators that were included in data quality reviews. These indicators are listed

below:

Table 6: MCA‐N related indicators: education (MoE)39

Indicator Definition Data Source Responsible Party

% of schools with positions filled to teach Information, Communications and Technology (ICT) Literacy

Percentage of MCA‐N‐supported schools with temporary or permanent teachers hired to teach ICT Literacy at schools where ICT facilities/equipment are provided by MCA‐N.

Education Administrative Data

MoE

Educators trained to be textbook management trainers40

The total number of educators who have received training to be trainers in textbook management


MoE


38 OPM, BAR II, p16‐19 39 MCA‐N, M&E Plan, Annex I, p3 40 Oxford Policy Management, Bi‐Annual Ex‐Post Data Quality Review, Round 3 (2014), p40


20


Educators trained to be textbook utilisation trainers41

The total number of educators who have received training to be trainers in textbook utilisation


MoE


The first review of MoE as a data source dealt with it primarily in regards to EMIS. It

identified key capacity gaps as their largest constraint, as described above. The second

review – in BAR III – looked at all the indicators above – ICT positions and educators

trained.42

The BAR III report found that the data collected on the ICT positions indicator were poor,

due largely to the fact that it was seen as a once‐off indicator and thus no systematic data

collection processes were established or implemented.43

Findings from the BAR III report in regards to the educators trained indicators did not reveal

any significant challenges to the data quality itself, as these had been resolved

(discrepancies were identified between the organiser and trainer reports); minor issue was

taken with the indicator itself, which was seen to omit issues of quality relevance, and

usefulness of training.44


The data needed for any potential future monitoring of ICT positions filled to teach ICT

literacy should ensure that a more systematic processes be put in place than what had been

previously used. No specific issues require further monitoring for the educators trained in

textbook management or utilisation data.

3.4. Vocational Education Administrative Data & VETMIS (Namibia Training Authority)


The Namibia Training Authority (NTA) is mandated to manage the vocational training sector

in Namibia, endeavouring ‘to ensure a sustainable skills delivery system under which quality

vocational and technical skills are imparted to young Namibians through Vocational

Education and Training programmes which meet the current and emerging needs of

industries’.45 Given its mandate for managing the vocational skills training sector, the NTA

was able to provide routinely collected administrative data towards monitoring various

MCA‐N indicators.

41 OPM, BAR 3, p15 42 Oxford Policy Management, Bi‐Annual Ex‐Post Data Quality Review, Round 3 (2014), p15‐17, 40‐44 43 OPM, BAR III, p17

44 OPM, BAR III, p40‐44

45 http://www.nta.com.na/?page_id=246 (accessed 16 October 2014)


21


The Namibia Training Authority (NTA) provided data for three MCA‐N indicators included in

the DQR process. They form a part of MCA‐N’s Competitive Grants for High Priority

Vocational Skills Training sub‐activity. These indicators are listed below:

Table 7: MCA‐N related indicators: education (NTA)46


Responsible Party

Vocational trainees assisted through the MCA‐N grant facility

47

The number of vocational trainees assisted through the MCA‐N grant facility

MCA‐N (NTA)

MCA‐N

Graduates from MCC‐supported education activities

The number of students graduating from the highest grade (year) for that educational level in MCC‐supported education schooling programs

NTA NTA

NQA‐accredited and/or NTA‐registered vocational training providers

The number of NQA‐accredited and/or NTA‐registered vocational training providers

NTA & NQA

NTA & NQA


The Namibia Training Authority (NTA) was first included in a data quality review at the third

round (2014) of the bi‐annual reviews, which looked at two of the three indicators above,

excluding the vocational trainees. All three indicators were reviewed in the DQR Follow‐Up:

Quarter 19 report. The DQR Follow‐Up: VETMIS report looked only at VETMIS, which will

take over monitoring data from the various aspects of NTA as it is fully rolled out.

The largest single recommendation that came from the BAR III relating to MCC‐supported

graduates (other than internal reporting improvements) was that the training providers

working with MCA‐N be registered with NTA and accredited by NQA. Otherwise, data were

found to be sound and fit for purpose. Furthermore, the data relating to vocational training

providers (VTPs) and their registration with NTA or accreditation with NQA were also found

to be sound and fit for purpose. It was suggested to separate the indicator into each

registration and accreditation and to monitor both separately rather than in aggregate. This

was not deemed to be necessarily beneficial by MCA‐N and was thus not taken into

implementation.48

The DQR Follow‐Up: Quarter 19 report provided the first reporting on the ‘vocational

trainees assisted’ indicator. The indicator presented certain challenges around how ‘assisted’

was considered and whether trainees should be considered assisted only at the selection

point and before drop‐out, or only after the ‘drop‐out’ period. It was determined that

46 MCA‐N, M&E Plan, Annex I, p6 47 For this indicator, MCA‐N is listed is the primary data source and responsible party. However, NTA was consulted as a key stakeholder on this indicator as they provide the bulk of the data that goes into monitoring this data to MCA‐N, which is then compiled with additional data external to NTA for a cumulative indicator. 48 OPM, BAR III, p24‐40


22

assistance might still be considered even if a trainee drops out at a later point. Graduates

from MCC‐supported activities were also considered as well as NTA‐registered and NQA‐

accredited TPs, with no significant data challenges to these.49

The VETMIS review provided an insight into the development and functionalities of VETMIS

as a means of consolidating and monitoring the multitude of data that NTA is trying to

manage. However, due to critical capacity constraints, the full roll‐out of the system has

been slowed. NTA struggles to find adequate capacity to train TPs in the usage of the system

as well as to manage the system within NTA, and to ensure the completion of its full

development (several modules of the system still remain incomplete).50

The largest single challenge to the various indicators and their monitoring was that the NTA

often struggled to get accurate and timely data from the TPs at the reporting periods. This

was seen as primarily a capacity constraint at the TP level in effectively being able to gather

and compile data in a systematic and timely fashion, while similarly relating to capacity

constraints acknowledged at NTA, inhibiting the institution’s ability to build capacity at the

TP levels in order to bolster their overall data quality and reporting structures.

These capacity constraints also translate directly into the completion and roll‐out status of

VETMIS. NTA lacks critical capacity within NTA to manage the system, as well as at the TP

level to effectively implement the system (collect and enter data) at the local levels (with

serious follow‐on implications in terms of the NTA’s performance management capability).


The single most important component for further action is the completion of VETMIS,

through the building of the necessary capacity and prioritisation of related functions and

systems such as M&E. This relates to the full development of the system and its component

modules and its roll‐out to the local TP levels where the data will be collected and entered

on an on‐going basis. In addition, it will be important within NTA to manage the system on

an on‐going basis, such as ensuring timely data uploads, and the sound use of the data

itself for analysis and reporting.

49 Oxford Policy Management, Support for High Quality Data Report: selected indicators in ITT on completed Quarter 19 – DRAFT (2014), p11‐13 50 Oxford Policy Management, Follow‐Up Data Quality Review of Data Collection Systems: Vocational Education

and Training Management Information System (2014), p4, 11‐14


23

4. Tourism Indicators

MCA‐N’s Tourism Project has as its goal to ‘grow the Namibian tourism industry by

improving tourism management and increasing awareness of Namibia as a tourist

destination’.51 This compendium report provides an overview of the following data sources

reviewed for MCA‐N: the system for Arrivals Statistics (MET), Etosha National Park

administrative data (MET), Tourism Jobs Model (NTB), Google Analytics (NTB), and the Bed

Levy Statistics administrative data (NTB).

4.1. Foreign Arrival Records (Ministry of Environment and Tourism)


Tourism is a critical part of the Namibian economy, and the Ministry of Environment and

Tourism (MET) is tasked with promoting ‘biodiversity conservation in Namibia through the

sustainable utilisation of natural resources and tourism development’.52 Given the economic

importance of tourism to the Namibian economy, it is important to have ‘up‐to‐date, reliable

and comprehensive information on foreign visitors to Namibia’.53 In the development of the

MET’s tourism arrival statistics, MET collaborated with the Ministry of Home Affairs and

Immigration (MHAI) to support the data project, while MCA‐N provided additional technical

support.


The Foreign Arrival Records at the Ministry of Environment and Tourism (MET) provided

data for four of the MCA‐N related indicators that have been included in previous reviews,

which are listed below:

Table 8: MCA‐N related indicators: tourism (Arrivals Statistics – MET)54


Leisure tourist arrivals The total number of leisure tourist arrivals recorded per calendar year

Foreign Arrival Records

MET

Tourist arrivals The total number of tourist arrivals recorded per calendar year


MET

Leisure tourist arrivals from the North American market

The number of leisure tourist arrivals from the targeted North American market (United States and Canada) per year


MET

Tourist arrivals from the North American market

The number of tourist arrivals from the targeted North American market (United States and Canada) per year


MET

51 MCA‐N M&E Plan, Annex 4, p7 52 Ministry of Environment and Tourism, Statistical Report on Tourist Arrivals (2011), p5

53 MET, Statistical Report on Tourist Arrivals, 2011, p5

54 MCA‐N, M&E Plan, Annex I, p11‐14


24


The indicators on tourist arrivals were reviewed in the GDQR.55 The review found that the

partners that collect data for these indicators – the MHAI and the Directorate of Tourism

and Gaming (within the MET) – had human capacity constraints, noted particularly for the

DoT. There were also much needed information systems upgrades in order to make the data

collection and entry not only more accurate and efficient, but also more reliable. Also, not all

border posts were covered by the computerised system used to capture information on

tourist entries and exits. Recommendations were therefore provided that these challenges

be addressed.56


Given the critical gaps in human capacity noted by the GDQR, particularly in the areas of

statistical training and data management, it is important for both the DoT and the MHAI to

develop and implement a capacity building plan tailored to the needs of each institution,

with an implicit assumption that they have the capacity to do so.

The review also noted a lack of adequate technological capacity to the arrivals statistics.

This took the form of insufficiently robust data back‐up. Hence, there remained a

considerable risk to data loss and corruption. It should be addressed through the

establishment of a reliable data management system, such as ACCESS or SPSS Data Builder.57

Finally, the computerised system used to capture tourist entries and exits should ultimately

be expanded to all border posts. This would allow for virtually 100% capture of tourist

movements.58 As a result, a much more accurate reflection on tourist arrivals in Namibia

would be ensured.

4.2. Etosha National Park Administrative Data (Ministry of Environment and Tourism)


While the data source discussed above (tourist arrivals) pertained to the significant share

that the tourism industry has in the Namibian economy, the Etosha National Park (ENP) has

possibly the largest share for single tourism destinations within the tourism industry. As

such, assistance to ENP formed a significant component of MCA‐N’s Tourism Project. Its

administrative data provided input to several indicators. It was also the most reviewed data

source under MCA‐N’s data quality reviews.

55 See OPM, GDQR, p90‐104 56 OPM, GDQR, p103‐104 57 OPM, GDQR, p103‐104

58 At the time of the GDQR, it was noted that approximately 90% of border posts in Namibia possessed a

computerised data system.


25


The Etosha National Park’s administrative data system provided data for five MCA‐N

indicators included in data quality reviews, as tabulated below:

Table 9: MCA‐N related indicators: tourism (ENP administrative data – MET)59


Entries and exists through Galton Gate

The number of entries plus exists through Galton Gate

Galton Gate entry/exit records

MET

Tourists to Etosha National Park

The annual number of paying visitors to Etosha National Park

ENP Park Entry Records

MET

Etosha National Park Gross Revenue

The annual total gross revenue generated by ENP, including gate receipts and concession fees

MET MET

Etosha National Park gross revenue from gate receipts

The annual total gross revenue generated by ENP from gate receipts (refunds are not included)

ENP Park Revenue Records

MET

The number of kilometres of roads and fire breaks within Etosha National Park

60

The number of kilometres of roads and fire breaks within Etosha National Park maintained by MET

ENP admin data MET

The number of kilometres of roads and fire breaks in conservancies adjacent to Etosha National Park

The number of kilometres of roads and fire breaks in conservancies adjacent to Etosha National Park maintained by MET

ENP admin data MET


The indicators pertaining to Etosha National Park (ENP) statistics were first reviewed in the

GDQR.61 The most notable finding was the lack of capacity amongst staff with expertise in

statistics. Additional concerns were around the data itself, as all data are collected manually,

and thus risks being damaged or lost. It was noted that there was a plan to computerise the

complete data collection system.62

The first bi‐annual review included indicators on Etosha gross revenue and tourists to the

park.63 For the revenue data, no significant challenges or issues with the data collection were

noted. A recommendation was provided in the way that the indicator was calculated (that it

should exclude revenue from concessions, as the concession fee data was problematic).64

In the second bi‐annual review, the indicators monitoring the use of Galton Gate as well as

the maintenance of roads and firebreaks were reviewed.65 For Galton Gate, the review

59 MCA‐N, M&E Plan, Annex I, p11

60 OPM, BAR II, p26‐27

61 See OPM, GDQR, p106‐115 62 OPM, GDQR, p114‐115 63 See OPM, BAR I, p23‐31

64 OPM, BAR I, p29‐31

65 OPM, BAR II, p22‐31


26

noted frequent opportunity for errors in the collection and transmission of data, as this is

done manually and telephonically. It was also noted that there were certain inconsistencies

in the forms used at Galton with those at other gates, presenting an inconsistent method of

gathering data.66 For the roads and firebreaks indicators, the data were found to be ‘dubious

in quality and the indicator values have a variable meaning as the need for maintenance

fluctuates with seasonality and usage’.67 A considerably more systematic and scientific

system for measuring data for this indicator was noted as being needed, and it was

recommended that the indicator not be used for MCA‐N’s monitoring purposes. It was

subsequently dropped from the M&E Plan.

The indicator on Galton Gate entries was reviewed again in BAR III, which found that the

same system and forms as discussed in the previous review were still in use.68 In particular,

the Yellow Book (shown in Figure 1 below and in Annex 3, Figure 2) is not suited to a gate

that would see several hundred tourists pass through it each day.

The indicator on tourists to Etosha was also included in the BAR III. It found the data to be

prone to errors as all the data are entered at least twice (from entry at the gate to

registering at the reception), and that these data are not easily reconciled with exiting

tourists, due to the manual nature of the process.69

The BAR III also reviewed the indicator on ENP gross revenue from gate receipts.70 The

review found an elaborate process in place to count money on a daily basis, rendering the

data reliable and of a high quality. The only exception is that Galton Gate follows a different

and less systematic method, which needs to be adapted to the same method used

elsewhere in the park.

The DQR Follow‐up: Quarter 19 report provided a final look at ENP as a data source.71 For

Galton Gate, the same system as what is reported on in BAR I and III was found to still be

used for monitoring visitors through the gate. This presented challenges for consistency

when moving between the white forms and the Yellow Book.72 However, in addition to the

problematic forms themselves, the reporting mechanism was found to be feeble: weekly

reporting does not precisely align with monthly reports. It also does not align with monthly

reports provided by the Warden of the Western Park to the Chief Warden. As a result of this

assessment, the DQR Team conducted a review of the numbers reported for Q17‐Q19

(October 2013‐June 2014) in order to get more accurate figures.

66 See Annex 3, Figure 2 and 3 for examples of the data collection forms

67 OPM, BAR II, p30

68 OPM, BAR III, p44‐48 69 OPM, BAR III, p49‐51 70 OPM, BAR III, p52‐56

71 See OPM, Q19, p14‐17, 39‐52

72 OPM, Q19, p39‐44


27

Figure 1: Galton Gate – Yellow Book & white forms

Source: Q19

In monitoring the number of tourists to ENP, these figures were found to be based on

revenue figures (number of permits) rather than actual tourist numbers.73 This does not

provide solid data for tourist numbers, as this method can present a false reality (i.e. tourists

that travel through a tour company who paid in Windhoek would not be counted; a tourist

that stays for 5 days is counted 5 times, rather than as a single tourist).

Revenue figures from gate receipts were similarly diverse. The Otjovasandu office (Galton

Gate) used several forms to record revenue: a book with blue pages with every transaction

recorded, the Monthly Report, and a Covering Advice Book.74 All the gates use a manual

calculation system, which is highly vulnerable to errors and inconsistencies in the time

periods that are reported on.

A key recommendation for Galton Gate is that it moves to the same system as that used by

the other gates in Etosha to count tourists. This will help to ensure consistency in data

collection and comparability across all data. In addition to the form discrepancies at Galton

Gate, the reporting mechanism was not found to be systematic and therefore prone to

errors. This needs to be standardised across all the gates with a systematic format. Tourist

figures are currently based on revenue figures, which do not provide solid data for tourist

figures.

73 OPM, Q19, p45‐46

74 OPM, Q19, p46‐52


28

Revenue figures for the park are monitored and calculated manually. This is highly prone to

errors, as the DQR Team discovered while assessing records in field. The reporting periods

are also at times inconsistent, where a month’s ending point is not precisely followed and

certain days overlap between two months, presenting over‐ or under‐counting. Galton Gate

also needs to come into the same system as that used throughout the rest of the park for

revenue monitoring.

Finally, a computerised system to monitor tourist entries and exits, as well as revenue, is

recommended throughout the park. Such a system would offer the most systematic,

accurate and reliable data possible for all the types of data that are reported on for

monitoring the use of and revenue generated by ENP.


Most of the issues and challenges encountered in the earlier data quality reviews with data

collection in ENP remained consistent throughout the DQR period (2010‐2014). This relates

especially to revenue from gate receipts, tourists entering the park, and Galton Gate specific

tourist monitoring.

As such, an improved system is still needed for monitoring the number of tourists to ENP.

Such a system could also be used to track revenue. A computerised system implemented

across the park would standardise the current data collection systems. This would ensure

much higher quality and accuracy of data collection and reporting, while also offering

opportunity for instant analysis and reporting, quality checks and transparency. But the

implementation of such a system would need to be accompanied by adequate capacity

building to ensure its proper usage.

4.3. Tourism Jobs Model (Namibia Tourism Board)

Overview of the Data Source

The Namibia Tourism Board is the Namibia Government agency responsible for ‘bring

together both the private and public sector in implementing the national policy on

tourism’.75 As such, the NTB’s mission is the following: ‘to market and develop tourism to

and within Namibia that exceeds our visitors’ experience expectations, delivers value to

stakeholders, improves the living standards and sustains the cultural values and way of life

of our people, and enables broad based participation of Namibians in the tourism

industry.’76 The NTB provided data through its tourism jobs model and Google Analytics to

MCA‐N‐related indicators.

For data on jobs created in the tourism sector, the NTB uses data from several sources,

including the Namibia Statistics Agency, the Central Bank, and the Ministry of Finance. As

75 http://www.namibiatourism.com.na/pages/About+NTB (accessed 19 October 2014)

76 http://www.namibiatourism.com.na/pages/About+NTB (accessed 19 October 2014)


29

such, one key data source for the jobs model is the Namibian Labour Force Survey, which

would prove problematic for the related MCA‐N indicator (see section 2.2).


The Tourism Jobs Model from the NTB provided data for the following MCA‐N indicator:

Table 10: MCA‐N related indicators: tourism (jobs model – NTB)77


Responsible Party

Jobs created through tourism

The number of direct jobs existing in the last 12 months within the tourism industry by companies involved in travel and tourism activities, such as hospitality, lodging, food service, equipment rental, guiding, sport hunting, airlines, etc., as defined by NTB

Tourism Jobs Model

NTB


The BAR I report reviewed the ‘jobs created through tourism’ indicator under NTB.78 The review found that the data for this draws heavily from the NLFS; and as noted earlier in this

report, the NLFS has had considerable scrutiny around methodological approaches that it

undertook and the exclusion of subsistence farmers in its employment considerations. In

addition to these specific challenges, the indicator itself would be similarly flawed even if the

methodological issues with the NLFS were resolved due to the lack of comparability over

time. Thus the indicator was seen as sufficiently sound, except for its basis in the NLFS. It

was recommended to be dropped as a MCA‐N indicator. 79 Considerations going forward

For the ‘jobs created’ indicator, the extent to which the data remains based largely on the

NLFS should be considered; and to that degree, the quality and comparability of the NLFS to

that of previous years should be considered for any comparability of the data. The current

editions of the NLFS (2012 and later) are considered methodologically sound, whereas

previous rounds, (namely 2004, 2008) are not. Hence, the indicator could be based on the

latest rounds of NLFS.

4.4. Google Analytics (Namibia Tourism Board)


The NTB website is a key portal both to tourists seeking information about Namibia, as well

as commercial operators in Namibia seeking to promote their products to the international

tourism market. Behind the NTB website, Google Analytics provides data on how the

website is used and accessed, with detailed data on numbers of users, unique users, the

location of users, time spent on the website, and other forms of detailed site usage data.

77 OPM, BAR I, p16‐17

78 See OPM, BAR I, p16‐20

79 OPM, BAR I, p20


30


The NTB website, using Google Analytics, provided data to support a total of 4 MCA‐N‐

related and reviewed indicators, which are tabulated below.

Table 11: MCA‐N related indicators: tourism (Google Analytics – NTB)80


Responsible Party

Unique visits on NTB website

The number of unique visits on NTB website Google Analytics

NTB

Unique visits on NTB website from the North American market

The number of unique visits on NTB website from the North American market (United States and Canada)

Google Analytics

NTB

Average time spent on the NTB website

The average length of time spent on the NTB website per visit

Google Analytics

NTB

Registered users of the NTB website

The number of visitors to the NTB website who enter the website and then register as a user (to receive updates)

Google Analytics

NTB


Google Analytics for the NTB website was reviewed in the BAR I and Q19 reports.81 For the

BAR I report, the data were generally found to be reliable and fit for purpose. The data are

drawn primarily from Google Analytics, which is robust and reliable. The only challenges in

the process of the review was more definitional, relating to how website visits are identified

(cookies on a computer mitigate over‐counting while multiple browser usage would be

registered under unique visits).82

In Q19, the NTB website indicators were again reviewed. The data were once again found to

be robust, given its basis in Google Analytics, which empirically measures the site’s usage.83

The only divergence in the findings related to registered users of the website. The original

intention of the indicator was to measure tourists who sign up to receive updates on tourism

in Namibia. Instead, it more recently began to monitor commercial users that sign up to

have their commercial products (i.e. tours, travel packages, etc) listed and advertised on the

NTB website.84 As a result, no data were available for consistent monitoring purposes on the

indicator after May 2013.


Consistency is needed in the use of the ‘registered users’ indicator. The meaning of the

indicator needs to remains constant, its data source reliable and data providers confident in

their understanding of the indicator. As it stands, the data cannot be of use for monitoring

80 MCA‐N, M&E Plan, Annex I, p14 81 See OPM BAR I, p32‐35 and Q19, p17‐21 82 OPM, BAR I, p32‐35

83 See Annex 3, Figure 6, for an overview of Google Analytics’ features available for the website’s monitoring.

84 OPM, Q19, p20‐21


31

that indicator. In moving forward, if tourist users of the NTB website are still of interest to

the NTB and the broader tourism industry, the website needs to be rebuilt to incorporate

the capacity to allow tourists to register as users, rather than being oriented towards

commercial usage of the site.

4.5. Bed Levy Statistics administrative data (Namibia Tourism Board)


As mentioned above, the NTB’s mandate is to regulate the tourism industry in Namibia and

promote the country as a tourism destination. Given this mandate, the NTB collects monthly

levies and other related statistics through a Tourism Levy Return and Statistics Form.85

Establishments registered with the NTB are required to complete this form along with a

proof of payment of their respective levy by the end of each quarter.86 This data collection

process, therefore, forms the basis for data on the Bed Levy statistics.

Table 12: MCA‐N related indicators: tourism (bed levy statistics – NTB)87


Bed levy income Total bed levies collected Administrative data (NTB)

NTB


The GDQR provided the first review of the bed levy statistics.88 The production of statistics

for this indicator relied on NTB administrative data, and the NTB noted considerably limited

human resources in this area, particularly with statistics. A further complicating element

noted in the review was the fact that the bed levy statics relates largely to financial matters

for the industry. The data for this are collected and kept separately from the statistics and

research unit. The disconnect between the two units prevents the research unit from

helping oversee quality standards for the data. A further issue was a noted discrepancy

between the forms used to collect data from registered establishments and the actual levy

fees that were received.

As a result, recommendations from the review were oriented towards increasing the

agency’s capacity, particularly in the area of statistics and data management. In addition, the

relationship between the research unit and the financial unit that gathers data on the bed

levy should be strengthened. It was also noted that the NTB should more regularly update its

list of registered establishments to identify those that have closed. This would ensure a

higher validity in response rates for their data. It would also show greater alignment in data

on establishments that are registered and those paying the levy.89

85 Please see Annex 3, Figure 5, for a sample of the form. 86 OPM, GDQR, p116 87 OPM, BAR I, p20


89 OPM, GDQR, p126


32

The second review – in BAR I – summarised the issue that NTB was legislatively restricted

from being able to penalise tourism establishments that do not comply with the necessary

levy fees as a part of their registration. At the time of the review, legislation was awaiting

approval that would allow the NTB to send inspectors to check establishments and

determine compliance and impose fines as punishment for non‐compliance.90 Ultimately,

due to the high levels of suspected underreporting, as well as the quality issues noted in the

GDQR, it was recommended that the indicator be dropped from MCA‐N’s monitoring plan.


Per the recommendation noted above the bed levy indicator was dropped for MCA‐N. If it is

to be used by NPC M&E or other entities as part of a monitoring framework, several points

should be taken into account. Firstly, it needs to be determined, whether the legislative

possibilities noted during the BAR I are in place now. They would allow the NTB greater

power to enforce compliance amongst registered establishments. A second point is to

determine what kind of support is needed to increase the overall capacity of the NTB to

manage statistics and data in a way that ensures minimal quality standards for monitoring

and reporting purposes. Moreover, there needs to be data management capacity to ensure

that the various data collected by the NTB are kept secure and managed across the

institution’s departments.

90 OPM, BAR I, p21‐22


33

5. Agriculture Indicators

The data sources used for MCA‐N’s Agriculture Project included in the DQRs related to

measuring change supplemented by MCA‐N’s efforts to contribute to the agricultural sector

in Namibia. These data sources include the Directorate of Veterinary Services (DVS) in the

Ministry of Agriculture, Water and Forestry (MAWF), the Meat Corporation of Namibia

(Meatco), and the Namibia Communal Land Administration System (NCLAS) in the Ministry

of Lands and Resettlement (MLR). The data sources were used i) to ‘capture change in the

average value of slaughtered cattle (by observing the value of sales of slaughtered cattle and

the number of cattle slaughtered in the NCAs)’; ii) to measure ‘change in overall animal

health (by observing the supply of animal health services and diseases/infections

diagnosed)’;91 and iii) to measure changes in parcel registration, perceptions about land

tenure, benefits to households or community groups.’92

5.1. System for Livestock Health Statistics (Ministry of Agriculture, Water and Forestry)


The Directorate of Veterinary Services (DVS) falls under the purview of the Ministry of

Agriculture, Water and Forestry (MAWF), with a mission ‘to maintain and promote animal

health, production and reproduction, and to assure safe and orderly marketing of animals

and animal products through animal disease control’.93 The DVS compiles and manages data

on cattle diseases, which is used for the system for Livestock Health Statistics.


The system for Livestock Health Statistics at the DVS served as a data source for the

following MCA‐N indicators:

Table 13: MCA‐N related indicators: agriculture (DVS – MAWF) 94


Responsible Party

Cattle inspections

Number of cattle inspections (on unique cattle) in the NCAs by a DVS health technician during the last 12 month reporting period

DVS MAWF

Cattle diseases

Number of cattle infections diagnosed which include foot and mouth disease, lung sickness (contagious bovine pleural pneumonia), lymph and skin disease, black quarter, botulism and rabies during the last 12 month reporting period

DVS MAWF

91 OPM, GDQR, p128 92 MCA‐N, M&E Plan, p21

93 http://www.mawf.gov.na/Directorates/VeterinaryServices/veterinary.html (accessed 20 October 2014)

94 OPM, BAR II, p42


34


The GDQR provided the first review of the DVS as a data source, looking at both indicators

listed above.95 The review found that the DVS is considerably constrained by inadequate

human capacity. At the time of the review, data capture, processing and analysis relied on a

single person. The GDQR therefore strongly recommended that the data skills needed at DVS

be decentralised amongst more staff than just the single person.96

Furthermore, on the indicator for cattle inspections the indicator should be changed. The

cattle are not all systematically inspected as this would be impossible. The indicator should

change from monitoring unique cattle inspections to using cattle vaccinated as a proxy to

the inspection indicator. This format is more reliable as data on specific vaccinations can be

monitored. This should in fact be used as a ratio (cattle vaccinated against those not

vaccinated) rather than simply a number of vaccinations.97

The first bi‐annual review also assessed the indicators on cattle inspections and cattle

diseases. Echoing the findings from the GDQR on the cattle inspection indicator, the review

showed that the data are unreliable as it stood with further vagaries around how a cattle

‘inspection’ is defined in the communal areas. Thus vaccinations should be used in the

stead of inspections.

For cattle diseases, the indicator was found to pose problems around interpretation: an

increase in cattle disease could mean that the DVS staff is increasing and able to diagnose

more cattle, or that there is in fact a higher prevalence of disease. As a result, it was

recommended that the indicator should not be used to monitor MCA‐N specific success on

this project. Instead the indicator should be retained for internal purposes of monitoring

progress. The data were seen to still offer useful evidence on outbreaks of diseases, etc.

Thus the indictor was dropped from the MCA‐N logframe, but still provides valuable

information for DVS.98


Future attention to this data source should revolve around the DVS’ capacity to manage and

process data. The constraints noted in the DQR’s point to the much‐needed improvements

in this area. Contingency plans should be in place to manage these challenges, as well as

training plans to mitigate them.

95 See OPM, GDQR, p126‐137 96 OPM, GDQR, p137

97 OPM, GDQR, p137

98 OPM, BAR I, p42‐48


35

5.2. Formal Cattle Slaughter Data (Meat Corporation of Namibia)


The Meat Corporation of Namibia (Meatco) is ‘a meat processing and marketing

organisation, serving markets locally and internationally on behalf of Namibian farmers’.99 It

is considered a state‐owned enterprise and, as it is the largest single entity managing large‐

scale commercial processing of meat in Namibia, its data on cattle slaughter was included to

support MCA‐N’s Agriculture Project and its activities aimed at enhancing the productivity of

the livestock sector in Namibia.


The data provided by the formal cattle slaughter database from Meatco underpinned the

monitoring of the following indicators:

Table 14: MCA‐N related indicators: agriculture (Meatco)100


Cattle slaughtered in the formal market

Number of cattle slaughtered in the formal market

Meatco Meatco

Sales of slaughtered cattle paid to farmers

Value of sales of slaughtered cattle paid to farmers by Meatco

Meatco Meatco


The GDQR found that the overall quality of Meatco’s data on cattle slaughter is

‘satisfactory’.101 However, in considering the data for MCA‐N’s monitoring purpose and the

indicator specifically, Meatco’s data reflects Meatco’s formal slaughters only. This does not

capture other formal slaughter occurring in Namibia, such as at other more local abattoirs

not owned by Meatco, nor the expansive informal sector.102 The indicator was subsequently

dropped from MCA‐N’s monitoring plan.


There are no further action considerations going forward, except for institutions like NPC

M&E to note the above findings when considering Meatco‐sourced indicators for possible

inclusion in results frameworks in the future.

99 http://www.meatco.com.na/ (accessed on 20 October 2014) 100

OPM, GDQR, p142 101

See OPM, GDQR, p138‐144 102

OPM, GDQR, p143‐144


36

5.3. Namibia Communal Land Administration System (Ministry of Lands and Resettlement)


The Namibia Communal Land Administration System (NCLAS) sits within the Ministry of

Lands and Resettlement (MLR). The NCLAS can be used in the process of managing land

rights, including generating village maps, applicant lists that can be displayed during the

notice period, vital information used for approval of applicants, and printing certificates.103


The NCLAS and the MLR provides data for two key indicators for MCA‐N’s efforts to support

reducing the amount of time required to register land, as listed below:

Table 15: MCA‐N related indicators: agriculture (NCLAS – MLR)104


Responsible Party

Average number of days to register a land right

The average length of time (in days) it takes to register a customary land right from when an application is submitted to the TA until the certificate is issued by the CLB

NCLAS MLR

Percentage of rights registered

The number of customary rights that are registered in a given quarter expressed as a percentage of the number of pending applications at that point

NCLAS MLR


The NCLAS was reviewed in the BAR II, within the broader framework of the indicators and

their pertinence to MCA‐N’s intervention.105 With regards to the NCALS specifically, though,

it was noted as capturing various data points along the land rights application process, such

as the Traditional Authority (TA) approval dates. However, it was also lacking some

important information, such as when land parcels are mapped and entered into NCLAS, and

when certificates are delivered to the TAs for owners to pick up.106 At the time of the review,

the NCLAS was undergoing a revision, which is expected to integrate the tracking of

applications.107


Ultimately, the most important point on this data source is the further development and

implementation of the NCLAS. At the time of the latest data quality review, the system’s

revised development had not yet been completed. The gaps noted above in the land rights

application should be addressed in the NCLAS for accurate monitoring of the process, such

that each point in the process is monitored.

103 OPM, BAR II, p34 104 OPM, BAR II, p31 105

See OPM, BAR II, p31‐38 106

See Annex 3, Figure 7, for a poster by MLR that communicates the process around land registration 107

OPM, BAR II, p37

Bi‐Annual Ex‐Post Data Quality Review 37

Annex 1: Data sources and corresponding reviews

Table 16: Data source and corresponding reviews

Data sources

Data quality reviews

GDQR (2011)

BAR I (2012)

BAR II (2012)

BAR III (2014)

EMIS (2014)

VETMIS (2014)

Q19 (2014)

Goal Namibia Household Income and Expenditure Survey (Namibia Statistics Agency [NSA]) √

Namibia Labour Force Survey (Namibia Statistics Agency [NSA]) √ √

Education Education Management Information System (Ministry of Education [MoE]) √ √ √

Directorate of National Examinations and Assessment (Ministry of Education [MoE]) √ √

Admin data (Ministry of Education [MoE]) √

Admin data (Namibia Training Authority [NTA]) √ √ √

Tourism Foreign Arrivals Records (Ministry of Environment and Tourism [MET]) √

Etosha National Park admin data (Ministry of Environment and Tourism [MET]) √ √ √ √ √

Tourism Jobs Model (Namibia Tourism Board [NTB]) √

Google Analytics (Namibia Tourism Board [NTB]) √ √

Bed Levy Statistics (Namibia Tourism Board [NTB]) √ √

Agriculture System for Livestock Health Statistics (Ministry of Agriculture, Water and Forestry [MAWF]) √ √

Formal Cattle Slaughter Data (MEATCO) √

Namibia Communal Land Administration System (NCLAS) (Ministry of Lands and Resettlement [MLR]) √


Annex 2: List of documents consulted

International Monetary Fund, Data Quality Assessment Framework (2012) Meat Corporation of Namibia: http://www.meatco.com.na/ (accessed on 20 October 2014) Ministry of Agriculture, Water and Forestry: http://www.mawf.gov.na/Directorates/VeterinaryServices/veterinary.html (accessed 20 October 2014) Ministry of Environment and Tourism, 2011, Statistical Report on Tourist Arrivals Millennium Challenge Account Namibia, 2014, Indicator Tracking Tables of July and September 2014 Millennium Challenge Account Namibia, 2014, Monitoring and Evaluation Plan National Examination and Assessment: http://www.dnea.gov.na/en (accessed 15 October 2014) Namibia Statistics Agency, 2012, Namibia Household Income and Expenditure Survey 2009/2010 Namibia Statistics Agency, Namibia Labour Force Survey 2004 Namibia Statistics Agency, 2014, Namibia Labour Force Survey 2013 Report Namibia Training Authority: http://www.nta.com.na/?page_id=246 (accessed 16 October 2014) Namibia Tourism Board: http://www.namibiatourism.com.na/pages/About+NTB (accessed 19 October 2014) Oxford Policy Management, 2011, Government Data Quality Review Report Oxford Policy Management, 2012, Bi‐Annual Ex‐Post Data Quality Review, Round 1 Oxford Policy Management, 2012, Bi‐Annual Ex‐Post Data Quality Review, Round 2 Oxford Policy Management, 2014, Bi‐Annual Ex‐Post Data Quality Review, Round 3 Oxford Policy Management, 2014, Follow‐Up Data Quality Review of Data Collection Systems: Vocational Education and Training Management Information System (VETMIS) Oxford Policy Management, 2014, Support for High Quality Data Reporting: selected indicators in ITT on completed Quarter 19 – DRAFT United Nations, Fundamental Principles of Official Statistics (1994)


Annex 3: International Data Quality Standards

3.1 IMF Data Quality Assessment Framework108

108

Double‐click on the document to open the document in full.


3.2 UN Fundamental Principles of Official Statistics


41

Annex 4: Supporting documentation


Figure 2: Visitor data record sheet (Yellow Book) – Galton Gate

Source: OPM, BAR II


Figure 3: Visitor data record sheet (white form) – Anderson Gate

Source: OPM, BAR II


Figure 4: Covering Advice, 23 May 2014

Source: OPM, Q19


45

Figure 5: Tourism Levy Return & Statistics Form

Source: NTB website:

http://www.namibiatourism.com.na/uploadedFiles/NamibiaTourism/Consumer/About_NTB/Industry_

Services/levy%20return%20form.pdf


46

Figure 6: Google Analytics – visits to NTB website (sample taken from Oct‐Dec 2013)

Source: OPM, Q19


47

Figure 7: Poster by MLR to communicate the process around land registration

Source: OPM, BAR II