Prospects of Quality Reporting Implementation for State Household Sample Surveys in Ukraine

The European Conference on Quality in Official Statistics

Rome, 8-11 July 2008

Volodymyr SariogloInstitute for Demography and Social Research,

Kyiv, Ukraine

Main continuous state household sample surveys in Ukraine

- Survey of Economic Activity of Population (SEAP) - since 1999; - Household Living Conditions Survey (HLCS) - since 1999;

- Household Agricultural Activity Survey (HAAS) - since 2000.

These surveys have been organized as integrated: SEAP and HLCS since 1999, SEAP, HLCS and HAAS since 2004.

The State Statistics Committee of Ukraine has accepted the Methodological recommendations on standard quality reporting by results of state household (population) sample surveys (March, 2008). The Strategy of development of the state statistics for the period 2009 – 2012, which is at the stage of the statement now, provides a number of actions on improvement of statistics quality, first of all, relevance, accessibility, clarity and quality reporting.

State regulations on quality reporting

- Relevance;- Accuracy;- Timeliness and punctuality; - Accessibility and clarity;- Comparability;- Coherence.The quality results of population sample surveys are the reliable data received in stipulated terms concerning population characteristics that satisfy the needs of users with the maximal completeness, are comparable with the available data from other sources and accessible for using

Measures of quality

Key quality indicators for quality reporting

- A description of conformity of survey results to needs of the main users (K1); - Unit response rates for main groups of units (K2);- Item response rates for key indicators (K3);- Imputation rates for key indicators (K4);- Coefficients of variation for key indicators (K5);- Length of time between the end of reference period and the date of the final results availability (K6);

- The number of disseminated publications (printed and electronic format) (K7);

- A description of the differences in definitions, classifications, methodology which characterize concrete sample survey results and national and international standards (K8);

- A part of statistics, that are used as sources for others statistical products (K9);

- Comparison of survey results with existing data from others sources (K10);

Standard quality reports for users

Base quality report contains a dynamic (current) information about the quality of survey results, which directly depends from the current survey data (standard errors, refusal and imputation rates etc).

Summary quality report contains a static information on quality, which does not depend from the results of current survey, but determined by methods and procedures of survey designing, organization, data processing, estimation etc.

Comparison of surveys quality (1)

  SEAP HLCS HAASK1 0,50 0,75 0,25K2 0,75 0,75 0,25K3 0,00 0,00 0,00K4 0,00 0,25 0,00K5 0,75 0,75 0,50K6 0,50 0,75 0,25K7 0,75 0,75 0,50K8 0,75 0,75 0,25K9 0,00 0,25 0,00K10 0,00 0,25 0,00I 0,400 0,525 0,200

1.0w j

10

1jjj wKI

0.1,75.0;50.0;25.0;0K j

Comparison of surveys quality (2)

Household response rates, HLCS 2000-2006

Household response rate indices, HLCS 2000-2006

0,90

1,00

1,10

1,20

1,30

2000 2001 2002 2003 2004 2005 2006

Inde

x

Ukraine Cities Towns Rural area Kyiv

Non-participant rate indices, HLCS 2000-2006

0,50

0,80

1,10

1,40

2000 2001 2002 2003 2004 2005 2006

Inde

x

Share of households which refused from the interviewsShare of households which didn’t take part in survey because of other reasons

Quality of statistical weights system (SEAP)

Indicator 1999 2000 2001 2002 2003 2004 2005 2006

Average weight, 743 738 658 647 653 369 367 362

Minimal weight, 185 192 196 187 195 16 52 32

Maximal weight, 2433 3028 2752 2079 2665 4562 4774 6413

Coefficient of variation of weights

36,48 37,75 38,56 38,66 41,42 87,95 87,16 87,33

4,02 3,84 3,36 3,45 3,35 22,75 7,03 11,44

3,28 4,1 4,18 3,21 4,08 12,37 13,00 17,72

Range of variation of weights,

2248 2835 2556 1892 2470 4546 4722 6381

minw

maxw

wCV

min1 w

wR

wwR max

2

R

w

Quality of statistical weights system (HLCS)

Indicator 1999 2000 2001 2002 2003 2004 2005 2006

Average weight, 1921 1897 1889 1868 1812 1753 1694 1659

Minimal weight, 591 384 357 46 45 56 14 9

Maximal weight, 5585 10205 9456 7195 13817 7333 8357 8817

Variation coefficient of weights

31,4 34,96 37,16 38,5 46,08 44,97 46,57 44,40

3,25 4,94 5,29 40,63 40,28 30,30 121,00 184,33

2,91 5,38 5,00 3,85 7,62 4,18 4,93 5,31

Range of variation of weights,

4994 9821 9099 7149 13772 7277 8343 8808

minw

maxw

wCV

min1 w

wR

wwR max

2

R

w

Coherence of estimates with National accounts

0

100

200

300

400

500

600

700

800

900

2006Q1 2006Q2 2006Q3 2006Q4 2007Q1 2007Q2

In a

vara

ge fo

r a m

onth

, per

cap

ita, U

AH

Total resources (excepting of privileges and subsidies), HLCS The disposable income, SNA Wages, HLCSWages (excepting of taxes and obligatory payments), SNA

Directions of future works

Maintenance of calculation of key indicators of quality in each state household survey.Development and implementation of standard quality reports for three base surveys during 2009.Development of procedures for detailed measurement of survey results quality by all quality measures. Development and implementation of procedures of integrated quality estimation, mainly for comparison of different surveys quality levels and revealing of critical quality aspects for each survey.Development of system of indices for measuring of changes of quality with the time.

The end