Report on the consultation of data producersec.europa.eu/...producers-consultation-report.pdf · 1.2 Implementation of the consultation process 4 1.2.1 The questionnaire to NSIs 5

Report on the consultation of data producers

Support in the Impact Assessment for a framework regulation in the area of social statistics ESTAT/F/2014/007

Final version

Zoetermeer & Valencia, January 2016

3

Table of contents

Table of contents 3

1 Introduction 4 1.1 Background to the process of consultation to NSIs 4 1.2 Implementation of the consultation process 4 1.2.1 The questionnaire to NSIs 5 1.2.2 The interviews with selected NSIs 5 1.2.3 Indicators for the consultation phase 6 1.3 Structure of the report of the consultation to NSIs 6

2 Description of the baseline situation 7 2.1 Social statistics: sample size and modes of collection 7 2.2 Response burden 8 2.3 The cost of social surveys 10

3 Perceptions of problems of social statistics 13 3.1 Uncertain quality (lack of coherence and comparability) 13 3.2 Insufficient progress on methods and procedures 14 3.3 Inefficiency of European social statistics (cost and response burden) 15 3.4 Loss of relevance of European social statistics 16

4 Perception of policy options 18 4.1 Perception of policy option 1 19 4.2 Perception of policy option 2 20 4.3 Perception of policy option 3 21 4.4 Perception of policy option 4 21 4.5 Perception of policy option 5 22 4.6 Perceptions of policy options’ impact on cost 23 4.7 Summary of perceptions of policy options 27

Annexes 28 Policy option 1: Not Integrated, Low on innovation, Fixed

programme 46 Policy option 2: Integrated, Low on innovation, Fixed programme 47 Policy option 3: Integrated, High on innovation, Fixed programme 47 Policy option 4: Integrated, High on modernisation, Multi-annual

integrated output programme with outline and burden limitation 48

Policy option 5: Integrated, High on modernisation, Multi-annual integrated output programme with outline and burden limitation 49

4

1 Introduction

1.1 Background to the process of consultation to NSIs

The impact assessment for a framework regulation on social statistics builds on

documentary research by Panteia and DevStat (the Consultants), including the review of

opinions on European social statistics expressed by users and by producers following the

Commission’s approach to what relates to the political validation of new initiatives1.

As part of the consultation — which also includes that of the main institutional users within

the EC and an Open Public Consultation — one dedicated to National Statistical Institutes

(NSIs) of the Member States and EFTA countries was designed, as responsible bodies for

the collection of data and compilation of statistics for national and EU purposes. It was

requested that NSIs’ opinion would also reflect that of other producers of social data in the

National Statistical Systems, and guidelines were given for the response to questionnaires

to take this fact into account.

The aim of the consultation to NSIs is to collect information about:

• the perceptions of the main problems of European social statistics;

• the acceptance or rejection of the policy options proposed;

• an estimate of the impact on cost & burden of the different policy options against the

baseline scenario.

The consultation of NSIs carried out for this impact assessment is not the first approach to

NSIs by the EC: the discussions between Eurostat and the Member States within the

Directors of Social Statistics (DSS) group and other statistical fora during the last years

have indeed considered the theoretical background and practical implications of the work on

the harmonisation of EU social statistics in general and the IESS (Integrated European

Social Statistics) legal framework, in particular.

This document presents the findings of the consultation of NSIs in an aggregated and

anonymised manner. It will also be considered in a consolidated Consultation Report

together with those targeting EC users and the public in general.

1.2 Implementation of the consultation process

According to the Consultation Plan validated by Eurostat and by the Inter-service Steering

Group for the ‘Impact Assessment of new legislation on social statistics towards 2017 and

beyond’ (the SG), the consultation tools comprise (1) an on-line questionnaire and (2) in-

depth interviews with a selection of NSIs. Both instruments having been agreed upon by

Eurostat and the SG. The Consultants and Eurostat decided to additionally carry out in-

depth interviews with a sample of NSIs, following the completion of the e-mail questionnaire

by these and in order to discuss specific aspects of the problem drivers, problems and policy

options in detail.

The following tasks were carried out for the preparation of the consultation tools:

• Preparation of a Consultation Plan;

• Validation by Eurostat and the SG of the Consultation Plan;

• Provision of contact persons in NSIs by Eurostat;

1 Annex 2 presents the new working methods that apply within the EC and between its services for the political validation of new legal initiatives.

5

• Preparation of a draft questionnaire and supporting documents;

• Agreement upon these by Eurostat and the SG;

• Testing the draft questionnaire between the 23rd and 26th of March 2015;

• Implementation of the questionnaire and supporting documents in the online tool

(Euro_survey);

• Notice to NSIs on the impact assessment in the DSS meeting of 25th February and

additional introductory letter sent by Eurostat;

• Preparation of interview scripts;

• Validation of interview scripts by Eurostat and the SG;

• Selection of MS to be interviewed in face-to-face or online meetings;

• Interviews with NSIs, either face to face (France, Germany, the Netherlands, Poland,

Romania) or by telephone or videoconference (Denmark, Portugal and UK) in September

and October 2015.

1.2.1 The questionnaire to NSIs

The collection of information was organised, taking into consideration the problem

definition, the objectives and the policy options2. The questionnaire for the NSIs includes

the following sections:

• Section 1: Identification of respondents, modes of data collection and use of

administrative data;

• Section 2: The problems – including a critical description of the current environment of

the European social statistics. This section aims to:

o rate the importance of each of the problems identified during the initial phase of

the impact assessment, namely:

loss of relevance in changing societies;

inefficiency (cost and response burden);

insufficient progress on methods and procedures;

insufficient quality (in particular coherence and comparability).

o forecast the evolution of each problem if the legal framework is not reformed;

o assess the reasons that might cause the described problem.

• Section 3: Impact of the policy options – provides a description of the policy options and

asks about the preferred and rejected options, but also invites NSIs to add relevant

comments in the context of this Impact Assessment.

• Section 4: Potential cost implications of the policy options - intended to measure the

cost and burden of the proposed options, focusing on the increase or decrease in

comparison to a baseline scenario. It includes three sub-sections:

o Baseline – quantitative assessment;

o The conceptual model (explanatory section);

o Qualitative assessment on potential cost impacts (∆).

Twenty-seven (27) countries sent answers to the questionnaire within the deadline indicated

by Eurostat, including 25 EU MS and Norway and Switzerland (EEA). One MS answered after

the deadline.

1.2.2 The interviews with selected NSIs

A selection of NSIs was made by Eurostat for in-depth structured interviews, balanced in

terms of geography, country size and statistical practices. The MS, dates and persons

interviewed are presented in Annex 4.

2 These were described in an interim report. On the basis of the consultation, several changes were made.

6

The scripts for interviews where sent in advance to the MS. They were based on the

contents of the questionnaire with an introductory section describing the consultation

process, and the following discussion themes:

• Discussion theme 1: Main problems/challenges in the current environment for European

social statistics and their reasons

• Discussion theme 2: Impact of the policy options (with a description of the options

considered in the Interim Report)

• Discussion theme 3: Financial implications - costs derived from the possible

implementation of an integrated European frameworks for social statistics

The policy options were explained in an annex to the questionnaire.

1.2.3 Indicators for the consultation phase

The following table presents the indicators for the monitoring of the consultation phase as

included in the Consultation Plan agreed upon by Eurostat and the SG.

table 1 Indicators for the monitoring of the consultation phase

Activity Verifiable indicator Sources of verification

Preparation of

consultation

materials

Questionnaire to NSIs

prepared

Scripts for in-depth

interviews prepared

Material for consultation

website prepared and

uploaded

Questionnaire prepared (Annex 1)

Script prepared (Annex 3)

Consultation website

(https://ec.europa.eu/eusurvey/runner/NSIs-

Consulation-EU-Social-Surveys)

Carrying out

the

consultation

with producers

(NSIs)

No. of answers received:

25 EU MS and 2 EFTA

Questionnaires filled-in

1.3 Structure of the report of the consultation to NSIs

Considering the questionnaire that has been proposed for the consultation to NSIs, the

following information is provided in this report:

• Section 2: Description of the baseline situation

o Sample size and modes of collection;

o Respondent burden by survey;

o Costs of surveys, by statistical activities.

• Section 3: Perception of problems in the current system of social statistics (see Interim

report I)

o Average rank for each problem;

o Expected evolution of problems in the next 10 years if there is no reform.

• Section 4: Perception of policy options

o Qualitative description of perceptions about of each option;

o Perception of impact on costs and response burden.

7

2 Description of the baseline situation

2.1 Social statistics: sample size and modes of collection

Social statistics are mainly collected from surveys of households and individuals, and with

less prevalence, from administrative registers.

The average sample size of each survey is given in the table below, with an indication of the

minimum and maximum in one country. Countries have expressed the sample size in

number of households (when sampling is based on a household frame), dwellings/addresses

(when based on a list of addresses) and persons (when based on a frame of persons), and

therefore the figures may not be strictly comparable.

An additional source of incomparability of the sample size across surveys is the frequency

with which data collections are implemented: quarterly (LFS), annually (SILC, HHICT), less

than annually (AES, HBS every 5 years or EHIS, HETUS with less frequency). The LFS,

which is carried out quarterly, is by far the largest survey.

table 2 Current Sample Size

Survey Current sample size (in thousands)

Median Average Minimum Maximum

SILC 9,8 10,0 3,8 27,0

LFS 48,5 80,9 10,3 286,0

AES 6,1 7,9 3,0 25,0

EHIS 10,7 12,6 2,6 25,0

HH ICT 5,5 7,5 1,5 20,0

HBS 8,9 13,1 1,2 54,0

HETUS 7,3 9,9 2,1 30,0

Few countries (between 6 and 11 depending on the survey) indicate the intended future

sample size in the next years (without specific reference to the new regulation):

• The increase of the SILC sample ranges from 20% to 130% (more than doubling). Two

MS with a strong regional structure consider doubling or more the SILC sample size;

• The increase of the LFS sample would be 40% in 2 MS and there would be no changes in

another 2. One MS intends to decrease by 10%.

• The increase of the AES sample would be between 10% and 100%, with one MS

decreasing by 30%.

• One country intends to increase the EHIS sample by 90% and one by 50%. The rest do

not change or even decrease (two MS by 10%-30%).

• 0ne country intends to increase HH ICT sample by doubling the size. Two countries will

decrease it by 20%.

• One country intends to increase the HBS sample by 30%. Two will decrease by 20%-

30%. For the others, there is no intention to change.

8

• There are no significant changes in the countries regarding the HETUS sample, except

that some country for which it does not exist will implement one in the next future.

In general, countries already intend to increase — not decrease — the samples, thus

achieving greater precision and allowing for more detail of the data in terms of

disaggregation, especially at the regional level if required.

2.2 Response burden

The modes of collection depend on survey and country, combining face-to-face interviews

with paper questionnaires and personal computers or tables (CAPI), self-administered paper

or web-based questionnaires (CAWI), telephone interviews (CATI). Most countries used a

mix of modes.

With the limitations of measuring the average time per respondent when mixed modes of

collection are used, as well as the differences in surveys whether targeting one or several

individuals or households, it is clear that diary-based surveys (HBS and HETUS) are much

more demanding in terms of time to provide information than the other surveys. From all

other surveys, SILC is the one that requires the most time.

table 3 Time spent per respondent and main mode of collection

Survey Time spent per respondent

(in minutes)

Main mode of

collection

(number of

countries)

Median Average Minimum Maximum

SILC 55,0 48,8 20,0 83,0 CAPI (13), Classical

face-to-face (5),

CATI (5)

LFS 19,0 20,3 4,0 41,0 CATI or CATI/CAWI

(11), CAPI (9)

AES 30,0 32,2 17,0 55,0 CAPI or CAPI/CATI

(13), Classical (4),

CATI (4)

EHIS 35,0 36,9 20,0 60,0 CAPI or CAPI/CATI

(10), Classical (8)

HH ICT 20,0 20,6 5,0 55,0 CATI OR CATI/CAPI

(12)

HBS(*) 97,5 191,2 15,0 600 (**) CAPI (10), Classical

(8), Post Mail or Post

Mail/CAPI/CATI (5)

HETUS 73,0 84,8 10,0 180,0 Classical (5), CAPI or

CAPI/CATI (5)

(*) One country has estimated the time to complete the questionnaire not including the diary. This country

does not report on the duration for filling in the diary

(**) One country has estimated the time to complete the HETUS as 1200 minutes, which is an outlier. The

second largest value is presented.

The mode of data collection is a significant factor in the duration of data collection, as

shown in the graph below for SILC and LFS. The use of innovative technologies clearly

decreases the response time. Technology-aided data collection is the general rule, but still

classical interviews are carried out in the case of complex surveys (e.g. HETUS, HBS).

9

table 4 SILC: Time per respondent (Min.)

table 5 LFS time per respondent (min.)

A simple indicator calculated as the size of sample x median time per respondent (imputing

the median time when not declared), and aggregating at EU level, can rank the social

statistical operations as shown below in terms of aggregate response burden. Supposing a

periodicity of 5 years for HBS and HETUS, 4 years for AES and EHIS, annual operations for

SILC and HHICT and quarterly collection of data for LFS, the annualised response burden

would be as in the table below:

65.2

55

45.8 43.6

20

0

10

20

30

40

50

60

70

Classical Post mail CAPI CATI CATI/CAWI

SILC: Time per respondent (min.)

31 30

2118

11

0

5

10

15

20

25

30

35

CATI/CAPI Classical CAPI CATI Classical/PostMail

LFS: Time per respondent (min.)

10

table 6 Annualised response burden

Survey Response

burden: Number

of person-days

at EU level

(sample size x

median time)

Periodicity/ Frequency Annualised response

burden (days)

LFS 31 558

Quarterly 31 558 (annual total) with a response burden

of 7 890 per quarter

HBS 42 353 Every 5 years 8 471 (supposing one

every 5 years) SILC 8 835 Annual 8 835 EHIS 8 316 Every 5 years 1 663 HHICT 2 695 Annual 1 695 HETUS 7 378 Every 10 years 738

AES 4 039 Every 5 years (carried out

in 2007 and 2011) 808

Any innovation in the data collection systems of HBS, LFS and SILC would have an

important impact to decrease of the total response burden. At the same time, any increase

in the sample requirements or extension of the questionnaires of these three statistical

operations would significantly increase the response burden. At the same time, in case of an

increased sample or frequency of surveys, the response burden would increase, especially in

the case of the most time-consuming surveys (HBS, HETUS).

2.3 The cost of social surveys

The consultation to NSIs includes the provision of baseline estimates for the cost of the

surveys considered.

Estimating the cost of surveys is a complex task, and has been in the debate in NSIs for

decades. Attempts to standardise the methods for cost accounting have been made3, but

they are in general focused on household surveys carried out on a project basis. For

recurrent surveys carried out by NSIs, some costs are spread over the whole statistical

production (e.g. management costs of NSIs, IT infrastructure, training of specialists). In the

case of social statistics using administrative registers, the accounting for costs of the

process of collecting such administrative records is not borne by NSIs but is still covered by

government budgets. Differences in staff salaries, country size in terms of population and

territory add to difficulties with obtaining comparable data. Differences in frequency of

surveys require considering an annualised cost dividing the total cost of a survey by the

number of years between rounds (for instance, the cost of a survey carried out every 5

years is simply annualised by dividing this total cost by 5).

It is important to note that the production process has a significant impact on cost. The use

of modern technologies for data collection (e.g. telephone or web-based collection vs face-

to-face interviews or re-use of administrative registers already collected), efficient data

processing (re-use of software for data entry, validation and tabulation) and data

33 See for instance http://www.ihsn.org/home/survey_budgets or http://unstats.un.org/unsd/hhsurveys/ for developing countries, but largely applicable to developed statistical systems.

11

dissemination (standardised publication processes), etc. can decrease the production costs

in the medium and long term, after the initial investments are absorbed.

Cost accounting by types of expenditure (IT, salaries, printing and sending questionnaires,

etc.) are relatively easy to collect but an analytical accounting per phases of the process is

not immediate. NSIs have, in general, put in place internal systems for obtaining estimates

according to their own needs.

Not all countries were able to provide cost estimates with the breakdown by broad statistical

phases (design, data collection and data treatment & dissemination). No information has

been collected on the disaggregation of cost according to investments for each phase of the

statistical process (e.g. development of new tools for data collection) and current

expenditure (e.g. salaries of interviewers carrying out the data collection). Thus, the figures

presented in this section have to be taken with care, since they do not represent a detailed

accountancy of the cost of social surveys4.

The aggregate cost of the production of the social surveys considered for the IESS amounts

to ca. 300M€ for the implementation of one of each survey in the reporting MS5, equivalent

to 176M€ annually, broken down as follows and annualised with the assumption of the

indicated frequencies:

table 7 Annualised aggregate costs

Survey Aggregated Cost (Mio€) Assumed

frequency

Annualised cost

(Mio€)

LFS 112,9 4 waves per year 112,9HBS 83,5 Every 5 years 16,7SILC 29,8 Annual 29,8HETUS 24,9 Every 10 years 2,5EHIS 21,7 Every 5 years 4,3AES 14,3 Every 5 years 2.9HHICT 7,1 Annual 7,1Total 294,1 176,2

On average, the cost of collecting one unit is around 100€, with substantial differences for

the diary-based surveys and across countries.

The relative cost of the design phase is slightly higher for ‘new’ surveys (AES, HHICT), but

the bulk of the cost corresponds for all surveys to the data collection phase (where a large

number of interviewers has to be mobilised for the traditional modes of face to face

interview, be it on paper or with CAPI).

4 Data cleaning has been necessary before the analysis. For instance, one MS incorrectly reported figures multiplied by a factor of 10, an error that was discovered during the data analysis. 5 For MS that did not provide figures, it is difficult to estimate the total costs of the surveys. If the costs for these missing countries is estimated by that of the country with most similar population size, the total amount would increase in about 10€.

12

table 8 Average Cost

Survey Average cost per unit (€)

Design (in % of the total cost)

Data collection (in % of the total cost)

Data treatment and dissem. (in % of the total cost)

HETUS 196,2 14,7 63,1 22,2HBS 193,7 10,6 69,6 19,8SILC 106,0 11,6 64,3 24,1AES 77,0 16,9 58,2 24,9LFS 66,8 9,8 72,6 17,7EHIS 53,5 15,7 65,0 20,0HHICT 37,8 16,7 58,3 26,3Avg 98,4 13,6 64,5 22,1

13

3 Perceptions of problems of social statistics

The problems identified in the preliminary desk research study (loss of societal relevance of

social statistics, uncertain quality and lack of standardisation, inefficiency of social statistics

in terms of cost and burden, and insufficient progress in methods) are similarly considered

by NSIs in terms of importance. On a scale of 0 (not important at all) to 10 (critical), they

are graded with median-high values 6.0 (average = 6.4), 7.0 (average = 6.5), 6.5 (average

=5.9) and 7.0 (average = 6.5). The large variation of answers (measured by coefficient of

variations of answers of 28%, 33%, 35% and 34%) do not allow to find significant

differences in their order of importance as the table below shows.

table 9 Importance of questions

Question: How important is…

Median Average CV

the problem of less and/or uncertain quality (e.g. lack of coherence and comparability) of the European social statistics today?

7,0 6,5 33%

the insufficient progress on methods and procedures for the European social statistics today?

7,0 6,5 34%

the problem of potential inefficiency of European social statistics today (cost and response burden that could be avoided under a better setup of European statistical legislation)?

6,5 5,9 38%

the problem of a potential loss of societal relevance of the European social statistics today in view of changing users’ needs and expectations?

6,0 6,4 28%

Scale of 0 (not important at all) to 10 (critical)

Detailed results of the tabulation of the received questionnaires are presented in Annex 5.

3.1 Uncertain quality (lack of coherence and comparability)

The lack of coherence across social statistical operations and of comparability (in time,

between countries) is considered a major issue for users of European social statistics,

according to the consultation to that group of stakeholders. NSIs confirm this view (7/10).

NSIs generally agree that progress can be -and has been- made by harmonising questions

across questionnaires (at the national and EU level), but there is no clear consensus on the

impact of a legal action on coherence and comparability (10 countries consider that it would

improve the situation, while 15 do not see any impact).

“the IESS will not solve the whole problem because you can have different implementations in the NSIs. Input harmonisation would help but it is not achievable, at most, the standardisation of some key issues such as the definition of the target population, sample frames...”

NSIs do not consider (4/10) either that the insufficient provisions in the current legal

framework on data collection methods and IT lead to uncertain quality of social statistics,

and in this sense, not reforming the legal framework will perpetuate this situation. However,

eight (8) countries consider that without this legal action, the situation would slightly or

significantly worsen. NSIs consider that the exchange of experiences in IT and data

14

collection solutions among MS is valuable (e.g. through the collaborative networks

ESSnets).

“… in spite of the legal framework innovative data collection methods and IT procedures are developed. Best practices are shared between surveys. Flexibility in choosing the collection modes that ensure the best possible data quality should be addressed”.

“The IESS regulation, at least in its present [draft] status, is not exactly favouring these new methods but it is allowing them, so we can consider that somehow IESS would help. The improvement in household surveys is more on NSIs side. Eurostat can help favouring ESSnets or interchange of experiences but a framework regulation plays a minor role in our opinion”.

“The mode of data collection has an influence [in data quality] as the ESSnet on multi-mode data collection has shown, but it can’t be avoided that countries use different modes”.

However, NSIs rather acknowledge (6/10) that insufficient provisions in the current EU legal framework about administrative data sources and multi-source integration lead to insufficient/uncertain quality of European social statistics (in terms of coherence and comparability). Some countries mentioned that a framework regulation that would provide for better access to administrative data would help much:

“the dialogue between the NSI and owners of administrative data would be improved by an European framework legislation”

and that without legal action, the situation would worsen (13 countries against 12 that consider that it would stay the same). In any case, most countries request more technical work on assessing the quality of administrative sources, on harmonising the concepts used with the statistical standards and validating the records.

“For using administrative sources it is important to have foreseen in the legal framework strong instruments to empower national statistical offices to impose a strategy of modernisation of administrative sources”.

“Quality indicators (coverage, timeliness, coherence...) must also be requested to administrative registers (AR).”

“Documentations about administrative and statistical processes have to be available. Documentation exists describing the differences between administrative and statistical processes in terms of definitions, concepts, coverage, etc. Differences in concepts are thoroughly studied and measures to deal with these differences are taken, when appropriate.

3.2 Insufficient progress on methods and procedures

The methodological progress is considered insufficient for European social statistics (7/10) to satisfy users’ needs and increase the efficiency of the production process. The perception of NSIs is mixed regarding the legal action as a solution: an adequate reform of the legal framework would improve (11 countries) or maintain the situation (14 countries), but still NSIs consider that progress can happen without a framework legislation:

“Common rules, standards and even tools for some phases of the production process of surveys would help to improve the situation. But […] we don´t find it clear to what extend framework regulation could even help”.

“European social statistics need to be up-to-date with the rapid innovation going on in the domain of statistical methodology and IT and we would question whether IESS would help in enabling this.”

“This could progress independently from legal reform”.

“LFS would have been the same without the European legislation […] Progress towards common procedures should not be the aim per se. Sampling details, if established by an European regulation, could be seen as input harmonisation

15

Technical action in the form of the exchange of good practices and development of common approaches among NSIs is valued, but they perceive shortcomings in the ESSnet networks, a recent form of international collaboration in official statistics promoted within the modernisation plans of the European Statistical System6, especially regarding their management and the financial support for transfer of the developed know-how.

Some countries mentioned the effort in agreeing on a legal act as a barrier to tackle technical (statistical) issues at the DSS meetings.

“Generally regulations are a poor way of informing the needs of policy makers faced with fast moving changes in society and the economy. […]. The time taken to draw up regulations such as IESS does not give confidence that they will then be able to improve the situation in a fast changing social landscape”. “the use of regulations doesn’t help – it’s part of the problem rather than a solution”

“Because of IESS is taking too much time in discussions between Eurostat and NSIs, the revision of the LFS modules has not been finalised”.

3.3 Inefficiency of European social statistics (cost and response burden)

NSIs recognise (6.5/10) that European social statistics have a problem of inefficiency in terms of cost (to NSI’s and other institutional data providers) and response burden (for respondents), but again opinions are evenly split on the impact of not reforming the legal framework to increase efficiency.

To a medium extent countries consider that the current legal framework requires questionnaires that are too long. It is clear that using innovative data collection methods and IT (7/10) as well as using alternative sources (6/10) can increase efficiency in cost and burden. Other non-legal actions can improve the efficiency:

“Having standardised sets of questions across surveys would make easier the work of interviewers and better dissemination of results (with less misunderstanding, for example, by journalists when presenting results for different target populations)”.

There is no clear agreement on the fact that decomposing questionnaires in modules would lead to increased cost-efficiency (4/10).

Most countries consider that if Eurostat does not reform the legal framework the situation burden will stay the same or worsen with respect to cost (19 and 6 countries respectively) and burden (17 and 7 countries respectively).

“The largest risk for inefficiency within the current legislative system is the lack of standardization between surveys.”

“It would be necessary to change the legal framework to adopt GSBPM, GSIM, CSPA and other production standards.”

A mention to the rigidity of programmes and the need for stakeholder engagement is worth noting:

“Social statistics in the Member States are extremely costly. A better justification of costs is needed. Priority setting is also required, including the possibility of considering ‘negative priorities’; i.e. what can be left out? In this sense, communication with stakeholders at European level should be increased. New requirements should be communicated in a formal, straightforward, precise and justified manner.”

6 http://www.essnet-portal.eu/

16

3.4 Loss of relevance of European social statistics

Requirements for social statistics develop continuously with new challenges to the changing

societies in Europe like immigration, ageing populations, changing forms of living together,

new challenges in the labour market, risks to people's welfare, globalisation, etc.

The relevance of social statistics is considered to be a problem by most NSIs (3 say it is

critical). Reasons for which the relevance of European social statistics may be at risk are the

following:

• During the crisis, much focus has been put on economic indicators rather than on

social;

• Users want quicker data. However, too timely data may compromise other quality

aspects of outputs;

• Topics such as victimisation, domestic violence and abuses have increasing interest

but are not covered by European social statistics;

• User engagement is weak.

Some countries consider that the possibility of including modules in the current surveys

(e.g. LFS) allows for increasing the responsiveness to user needs.

Fifteen (15) countries consider that the relevance of social statistics be unaffected by no

legal action while 10 consider that the situation will slightly worsen.

“Even with the current legal structure we will take into account new information needs and take them seriously when developing and changing the contents of surveys. From the perspective of an individual survey this seems not to be an essential problem. However, when thinking about the whole sphere of European social surveys and their conceptual coverage and possibilities to describe new phenomena a common legal framework could make a difference”.

“The potential loss of relevance is more related to the lack of ability to face new projects. Is IESS a solution for this? In our opinion it isn´t. As IESS establishes a closed list of domains, anything outside these detailed topics lies outside the scope of the regulation, and should be subject of specific regulation”.

There is more agreement on the slow capacity of the ESS to adapt to new information needs

that will cause loss of relevance (7/10). Multi-annual planning, that could be provided by an

integrated regulation, was mentioned by the NSIs as a need to combine users’ needs

satisfaction with the necessary organisation of the statistical production.

Quoting some responses, more relevance can be obtained by combining data from different

sources, which should be favoured by an integrated regulation:

“The possibility of linking sources and combining data and outputs will offer more input for the interpretation of statistical results and it will increase the quality of reporting and dissemination of data, in order to better satisfy users’ needs and enhance trust to the statistics produced by NSIs”.

“There should be more analysis and evaluation of data and experiments to find ways to combine and link variables from different surveys in order to produce new indicators”.

“Different regulations hamper the multi-source integration. [Country] is more advanced because we use administrative data”.

In fact, a MS that bases most social statistics on administrative registers considers that

relevance is high precisely because outputs are very much related to the administrative

social system:

17

“Social statistics in … are based on administrative registers, very rich and detailed. This is different than in other MS. Local users find social statistics very relevant and rich. For example, income data from SILC are not seen as relevant as those [produced by administrative registers] of the NSI”.

In summary, NSIs consider that the loss of relevance is not so much due to fragmented

legislation than to the inability to quickly react to users’ needs, and that discussions on

legal aspects distract from undertaking technical work such as discussion on new methods.

At the same time, there are opinions that a legal framework that would make the use of

administrative sources easier, would facilitate combining sources and that would allow for

including new relevant topics, would increase the relevance of social statistics.

18

4 Perception of policy options

The consultation to NSIs included the description of the policy options summarised in the

table below. It should be noted that, based on the results of the consultation, changes have

been made in the description of the policy options.

table 10 Description of policy options

Policy option

Integration/ fragmentation of social statistics legislation

Modernisation in view of statistical production (standardisation, quality reporting, survey integration, etc.)

Flexibility in view of statistical output

Outline of transmission obligation in basic act; provision on cost, burden and financing

1 (Business as usual)

Fragmented legislation

Some modernisation possible, but not in an integrated manner across different data collections

Only limited flexibility (e.g. ad-hoc modules), but no integrated data collection programme

- Mostly transmission obligations outlined in the respective base regulation

- Transmission obligations not fully integrated across different data collections

- Often general provision on cost and burden

2 Integrated legislation

EP and Council Regulation provides little support the modernisation of statistical production

Transmission obligations largely fixed in the base act

No provision on cost and burden because it can be assessed completely on the basis of the EP and Council regulation


EP and Council Regulation contains provisions that support the modernisation of an integrated statistical production (standardisation, quality reporting, survey integration, etc.)

Transmission obligations largely fixed in EP and Council regulation




Flexible integrated multi-annual data collection programme via implementing measures

EP and Council regulation contains outline for transmission obligations and provisions to limit cost and burden to administration and respondents




EP and Council regulation contains no outline for transmission obligations and no provisions on cost and burden

19

4.1 Perception of policy option 1

Despite the comments by some countries that the reform of the legal framework will not

completely solve the problems of social statistics, most countries that wanted to comment

on Policy option 1 (6 countries) explicitly rejected it:

“Maintaining the status quo would mean the progressive loss of societal relevance of the data, especially in view of comparability between countries”

“ESS will cease to fulfil its tasks; - entrenching of existing situation on national level; - jeopardised tasks of ESS, risk of competition from the surveys outside of ESS frame”

”Policy Option 1 does not facilitate the procedures towards the modernisation of Social Statistics, it rather complicates the procedures.”

“It is hampering modernisation to have too long surveys, few standardised variables etc., and no firm rules on adding and deleting variables”.

Up to 16 NSIs mention risks for Policy option 1:

• Lack of flexibility with respect to users’ needs (5 countries);

• Progressive loss or comparability and standardisation between countries (3

countries);

• Lack of integration across surveys (3 countries);

• Risk of competition from non-ESS social surveys;

• No enhancement in terms of programming;

• Heavy production process with steadily increasing costs (2 countries);

Only two NSIs consider that the baseline scenario still allows for improvement of the

individual current legal acts in a fragmented way, and one country recalls that further

harmonisation can be achieved without an integrated regulation:

“Even if changes cannot always be implemented easily, the current system has proven to be adaptable with regards to changing user needs. Furthermore there are advantages associated with the current situation; i.e. that the system is a known entity, it is stable and there is no uncertainty with regards to what the statistical output is”.

“This option could perpetuate a lack of harmonisation (e.g. in common variables), but this is not a necessary consequence of a lack of an integrated regulation. These issues could be addressed simply through closer working across Eurostat units in developing individual sources. The legislative framework as a whole is therefore not conducive to achieving less fragmentation necessarily. This goal could be achieved under the current regime if there was more of a focus on output harmonisation and less on input harmonisation. […] Fragmented legislation can be a good thing as it allows for domains such as Time Use to be handled through more flexible arrangements, such as the current gentlemen's agreement”.

The consultation shows that Policy option 1 is not favoured by NSIs, but even in this case

there may still be room for improvement of social statistics by (1) updating the individual

legal acts towards more integration and harmonisation, (2) working on technical issues in

the frame of Eurostat working groups and (3) ensuring proper planning across domains.

20


NSIs view the integration of legal acts as an option to solve the current problems especially

in terms of coherence and comparability across the proposed domains. Two countries

mention that this option is seen as a pragmatic, first step with minimal advantages with

respect to the reform of the individual regulations.

“The advantage of an integrated legislation framework is that it could offer greater coherence and comparability across Social statistics domains. The disadvantage is that [the draft] IESS as it stands is not creating greater integration and has already had to exclude domains that don’t fit well with the framework being proposed. The standardised core variables common across all surveys can act as a hindrance, for example in accommodating these across the different surveys.

According to two MS, rigid programming presents predictability on the expected output and

therefore on costs for NSIs:

“Fixing transmission obligations within the basic act does have the advantage of allowing Member States to plan for, and fund, future requirements in the knowledge that there will not be changes that need to be incorporated in an ad-hoc fashion with potential impacts on costs and burden”

“An integrated legislation […] solves many of the issues associated with the current fragmented legislation, but at the same time it does not risk causing undue burden or costs on member states since the flexibility is still rather low.

The fact that no provisions are made in relation to the modernisation of IT and data

collection does not mean that NSIs will not continue their efforts in this direction.

“We do not see why this scenario is low on modernisation. Having no binding provisions on

modern quality reporting and modern IT-Tools and a harmonised methodology does not necessarily mean that they are not used. So the real question seems to be whether provisions on these aspects are necessary.

Not providing for a control of costs when accommodating for emerging users’ needs is seen

as a major risk of this option.

Up to 9 NSIs mention risks for Policy option 2:

• Loss or comparability and standardisation between countries (3 countries);

• Flexibility is still low (3 countries);

• Does not guarantee integration of surveys;

• Does not ensure a balance between user requests and reduction of budgets;

• The discussions about integrated legislation could delay modernisation efforts;

• The inclusion in the scope of an integrated legislation of additional surveys is

considered by some MS as a risk for higher costs.

From the collected opinions, there is evidence that Policy Option 2 is preferred to Policy

Option 1, but the details have to be defined in order to move forward: an integrated legal

act does not per se improve the situation of social statistics.

21


The introduction of provisions for modernisation while largely fixing the work programme in

an integrated regulation — which would increase the predictability of work load for NSIs —

is welcome, and four (4) statistical producers in countries explicitly favour this option:

“it would achieve goals of integrated legislation, modernisation and harmonisation along with certainty on data outputs”

“The adding of elements to support an integrated statistical production is something which we support. That the statistical output is fixed in the basic legal act is something which we think is an important element in order to make the production system predictable and to ensure that NSIs have control over costs and burden”.

but again, some countries perceived that modernisation of processes aiming at the

reduction of costs and burden can take place without a reform of the legal framework:

“However there is much evidence that even if Eurostat does not reform the legal framework, there would still be positive development in many areas of social statistics in many Member States. High quality social statistics can rely on the advanced and efficient use of data collection and production methods and integration of multiple sources even if the legislation is fragmented”.

With respect to the risks perceived for this option, 6 countries comment:

• The competition from non-official sources due to the increased lack of relevance and

timeliness because of low flexibility (4 countries);

• The loss of comparability;

• Less innovative NSIs will have to face an increase in costs and burden.


Policy option 4 introduces an element of flexibility in the programming of data collections

while providing support/limits to increase in cost and burden. Up to 9 NSIs are explicitly in

favour of this option and 2 are against:

“Policy option 4 seems to be the most relevant to the objectives discussed. The specific option is rather flexible in terms of survey programming, while it also tries to ensure the adoption of provisions that will support the modernisation of statistical production in terms of methodology, new technologies etc. Finally, this option sets certain limits in terms of cost and burden, as well as restriction in the transmission which is important in order to ensure that data will be provided on time”.

“I do support option 4. General provision has to be made that when new data are required and new variables are to be measured, within 2 years (for example) burden on respondents will be brought back to the situation before (meaning clearly outdated variables are stopped).”

“It is our preferable option because of inclusion of the issues of flexibility defined by implementing measures and limitation on cost and burden”.

Policy option 4 seems to be the most adequate but there is a general perception of risks in increasing production costs due to the flexibility needed to satisfy users’ needs without proper planning.

Risks are perceived especially in the context of an increase in costs of higher flexibility without clear constraints to the increasing requirements:

“[…], the introduction of new topics, terms and requirements to the surveys has to be balanced and conformed to the capacity of data producers to cope with additional burden”.

22

“We support many of the points included under policy option 4 but we do not support the increased flexibility of programming statistical output”.

“And it needs to be pointed out that even if there are legal provisions for flexibility this does not necessarily mean that Member States have the capacity for flexibility”.

“[…] more flexible multi-annual data collection programme via implementing measures could add costs and burden for Member States. The important issue is to that this should be restricted by respecting the existing costs and burden faced by MS in terms of budgetary constraints. It is important to note that MS have their own user requirements for policy making based on social surveys that have to be accommodated so there is danger that adding complexity in terms of data collection can have knock-on impacts on cost and burden. There must be a proper impact assessment of new requirements and this needs to be based on user requirements and the needs of the MS themselves. In an ideal world there should be a provisions to limit cost and burden built into IESS built into the regulation but this cannot be in general terms and must be enforceable.”

“Negative priority setting has not been very successful in the European Statistical System. So, the risk of this policy option is a gradual expansion of the statistical programme”.

“There must be a proper impact assessment of new requirements and this needs to be based on user requirements and the needs of the MS themselves”.

“Flexibility is acceptable, but how the minimum requirements that have to be included in the legal framework would be changed? A definition of what is essential and non-essential in the context of statistics is needed. For a Member State transmission of data is essential. Changes need to be justified and the delegated acts might not be the optimal solution.”

Two countries explicitly express their opposition to this option and one country conditions its’ support to the guarantee of financial support in the case of new requirements:

“This option would achieve goals of integrated legislation, modernisation and harmonisation along with increased flexibility to meet user demands. However as a small NSI the relative insecurity in regard to cost would lead us to favour the earlier Option 3. There is no guarantee of financial support or proposals to simplify the awarding of grants”.

The risk of this option can be minimised, in the opinions expressed by NSIs, if (1) detailed provisions are included to limit the introduction of new aspects in social surveys and (2) a well-functioning system is established for identifying negative priorities7 taking into account the needs of national users (to avoid an ever-increasing statistical programme).


Policy option 5 is seen as a ‘maximalist’ option which lacks plausibility. Eight (8) countries

reject this option while only 1 favours it.

The fact that there are no provisions for limitation of costs makes it hard to be accepted by

data producers:

“If resources of data access, time and money was not an issue, option 5 would be nice to have. We would all like to have a statistical system that could cater for both national and comparable European users' needs. Reality is that resources are scarce in all respects; access to data, time and resources (money and staff) - and reality is also that users' needs are different at the national and European level”

“Policy no 5 is not an option without any safeguards against strong increases of costs and response burden”.

“We could not support this option as there would be no safeguards against strong increases in cost and response burden”.

7 The system of ‘negative priorities’ is a decision-making mechanism for the NSIs and Eurostat to discontinue the collection of statistical series based on a trade-off between relevance at the EU level and cost/burden of the production process. They are reflected in changes in the Annual Work Programme of Eurostat.

23

“The statistical output would keep changing, which would mean a large impact on costs, and a recurring impact as the output keeps on changing”.

No limitations for flexibility are also a barrier for acceptance of this option:

“In this option there would be no way for member states to maintain control over costs and burden, something which we are not willing to accept. We also see an apparent risk that this degree of flexibility could lead to a devaluation of the quality of European official statistics”.

“In option 5, since there are no provisions to limit increases in costs and burden the impact on costs is impossible to predict but there is an obvious risk that it would be very large”.

The perceived risks mentioned by 8 countries refer to:

• Incapacity of some NSIs to respond to increased demands (3 countries);

• Lack of national political support for the resources needed (2 countries);

• Lack of control by NSIs to define their statistical programme and costs (2

countries);

• The flexibility without constraints may cause inconsistencies among surveys and

problems in terms of timeliness.

4.6 Perceptions of policy options’ impact on cost

NSIs were asked about their perception on policy options’ impact on the cost of statistical

operations in the medium term (5 to 10 years).

Policy option 1 is not considered to have a significant increase in costs, since it is the

current situation.

All other options presented to NSIs comprised integrated legislation. NSIs were asked to

assess whether an integrated legislation on European social statistics would increase or

decrease the costs related to the statistical phases of: design, data collection, treatment

and dissemination. NSIs were also asked to assess whether, in the absence of cost increase,

an integrated legislation would impact on the responsiveness to users’ needs.

The impact on the cost of a legislative situation in which European social surveys are

integrated and innovative data collection modes and sources are promoted of the different

statistical phases is summarised in the following table:

24

What would be the potential medium-term impact (around 5 to 10 years) of a legislation on European social surveys which is integrated and promotes innovative data collection modes and sources on the cost of statistical phases?

table 11 Potential medium term impact

Design Data collection Treatment & Dissemination

Significantly decrease 1 1

Decrease 10 2(Insignificant) 8 7 9

Increase 12 7 10

Significantly increase 6 1 3

DK/DA 2 2 3

Total general 28 28 28

At the level of production phases:

Impact on the cost of design of statistical operations: In the medium term (5 to 10

years), 18 NSIs consider that an integrated legislation would increase the costs in the

design phase. Some think the impact will be significant (6/28). 8 NSIs consider the impact

would be insignificant. Interestingly, countries for which the impact on cost would be

insignificant include NSIs that base their production of social statistics on surveys and NSIs

that base it on the thorough use of administrative registers.

The cost of design generally takes the form of investments (design of IT tools,

questionnaires, training materials, etc.). It can be recovered with repetitions of data

collection operations.

Impact on data collection costs: the data collection represents the bulk of costs. An

integrated legislation would decrease the cost of data collection for 11/28 countries

(significantly for one MS), but would increase it for 8/28. There is no clear segmentation of

perceptions based on the countries’ use of data collection modes or use of administrative

registers.

Increase in data treatment and dissemination: Results are less mixed with respect to

the impact on data treatment and dissemination: 13 respondents think that costs would

increase or significantly increase, for 9 it would be an insignificant change, and for 3 it

would decrease or significantly decrease costs.

According to their perception, the increase in costs would, in most countries, be borne by

the NSIs (16/28 countries) while 9 of them think that it would be shared with other

authorities e.g. administrative data providers, but with a higher share for the NSI.

Policy Options 2 to 5 are deemed to imply an increase in cost, especially during the

transition to an integrated legislation.

The lack of flexibility of Policy Option 2 would stabilise the costs after some time thanks to a

fixed programme, but could suffer from adaptation to user needs. The possibility for

increasing the number of requirements (data domains, variables) in Options 4 and 5 are

seen as a risk in terms of cost control. Literal comments expressed by the NSIs are grouped

in the table below.

25

table 12 Perceived impacts of the policies on the costs (opinions expressed)

Policy option 1 Policy option 2 Policy option 3 Policy option 4 Policy option 5

General impact on cost and burden

“Minimal impact on costs as it would mean that the current system would still be in place”(

Integration of surveys such as Time Use Survey and Household Budget survey into IESS will significantly increase costs and burden. Increase of costs to NSIs (4) in transitioning to a new system (3). The situation would eventually stabilise due to the low flexibility of the output (1) Insignificant or no change (2)

increase of cost on EU level and partially on NSO level (4) Increase in costs in transitioning to new integrated legislation and modernisation The situation would eventually stabilise due to the low flexibility of the output (1) Moderate risk of increase in costs. NSI would be affected.

Increase of cost on EU level and on NSO level in the beginning and maybe progressive decrease of costs. Significantly increase of costs for NSIs (3) Implementing Policy Option 4 may have an increase both in cost and administrative burden for the NSIs. Furthermore it may have an increase on the respondent burden.

Increase of costs for NSIs (4), unsustainable or critical (2) , increase both in cost and administrative burden for the NSIs … this would be borne by the NSI, other government departments and devolved administrations Moderate risk of increase in costs. NSI would be affected. Increase in costs in transitioning to new integrated legislation, modernisation and adapting to increased flexibility around data outputs

Impact of flexibility on cost and burden

Increase [of cost and burden] could be mitigated through effective planning and across the social statistics domains.

Additional variables (e.g. core variables) and burdensome precision requirements will be costly to implement

There are risks to fixing requirements and cost and burden requirements within the basic act but this can be assessed by MS and planned for to mitigate the potential risks. The risks of continuous increase of topics; insufficient material and human resources can have very significant impact on costs, mainly for NSI. There is a possibility that at times, the output data may not be optimally fulfilling the users' needs and also it may take time to react to new needs.

[The option should], include not only a maximum of variables to change, [but] also include a line saying that within 2 years (example) burden on respondents and NSI will be back as before. Meaning that when new variables come, old and less relevant variables have to go. The statistical output would keep changing, which would mean a large impact on costs, and a recurring impact as the output keeps on changing. Increase in costs in transitioning to new integrated legislation, modernisation and adapting to increased flexibility around data outputs

If no provision for burden or cost is made, we can arrive in the same situation as option 1, … may have an increase on the respondent burden. There is no safeguard against strong increases.

26

From the responses of countries, it is not clear that the integration of surveys and the use

of innovative data collection modes can free resources that may improve the responsiveness

and user orientation. Under the assumption of no increase in the total cost, no survey is

above the mid-point of a scale from 0 (no improved responsiveness) to 10 (improved

responsiveness):

table 13 Improved responsiveness and user-orientation thanks to survey integration and the use of

innovative data collection modes and sources,

Scale from 0 to 10 the potential effect

SILC 5,0LFS 4,2AES 4,6EHIS 4,2HHICT 4,2HBS 4,2HETUS 3,2

27

4.7 Summary of perceptions of policy options

To summarise the perceptions of NSIs on policy options, it can be stated that:

• Not undertaking a legal reform will generally keep the situation as it is or worsen it. No possibilities for ‘inertial’ improvement are generally perceived; the current system can still be an option but would require changes anyway in the individual legal acts;

• An integrated legislation would favour the increased harmonisation of concepts. This could also happen with a fragmented legislation, but more technical work is needed to achieve greater coherence, comparability and integration of statistical operations;

• Multi-source integration would also be favoured by an integrated regulation. In particular if it provides improved access by NSIs to administrative registers. In parallel, technical work should be done to ensure quality assessment of administrative sources and increase their international comparability;

• In the medium term (5 to 10 years), most NSIs consider that an integrated legislation would increase the costs in the design phase, as well as during the data treatment and dissemination. But, there is also relative majority of countries that consider that data collection costs — the bulk of production costs for European social statistics at the national level — would decrease.

However:

• Relevance of social statistics is not so much related to the legal

architecture, as to the capacity of NSIs to address new information needs;

• The modernisation of social statistics through the use of modern data collection methods (especially multi-mode) and enhanced use of IT will happen independently of the legal architecture. For instance, through exchange of good practices and collaborative networks (e.g. ESSnets);

Necessary aspects that any legal architecture should ensure are:

• Multi-annual planning, within an integrated or a fragmented legislative architecture, is important for better organisation of production and control of costs by MS;

• Better access to administrative sources and more technical work on their quality assessment would allow decreasing production costs;

• Limitations to the flexibility of introducing new topics for measurement as well as an operational method for negative priority identification, involving the users of national and European data, have to be made explicit in the legal architecture, so that MS can control the increased production costs;

• Flexibility of MS in adopting the optimal solutions for data production has to be maintained. Output-harmonisation is preferred to input-harmonisation;

• MS keep control on essential aspects such as data transmission deadlines, periodicity of surveys and geographical breakdown of the output.

28

Annexes

Annex 1: Questionnaire to NSIs

Annex 2: List of respondents to the questionnaire

Annex 3: Script of Interviews with Directors of Social Statistics within the National

Statistical Institutes

Annex 4: Calendar of in-depth interviews and persons interviewed

29

Annex 1. Questionnaire to NSIs

Introduction

The production of European social statistics is subject to a changing environment in

line with the overall process for the modernisation of European statistics more in

general. This changing environment is characterised by: - new data needs and expectations from the users; - increasing competition on information market; - availability of new data sources (such as administrative data); - pressure on resources of national administrations (cost of statistical

operations) - increasing concern on response burden; - innovation in methodology and IT.

A critical review of the current environment for European social statistics reveals a

series of problems that represent real challenges for moving on in the process of

modernisation of social statistics. Four main problems are considered and described

below. 1. Loss of societal relevance of the European social statistics in view of changing

users’ needs and expectations 2. Lack of coherence and comparability of European social statistics 3. Inefficiency of European social statistics (cost and response burden) 4. Insufficient progress on methods and procedures for the European social statistics

For each main problem, you will be asked two questions: 1. How important is, for your institution, each of the main problems today (current

situation of European social statistics)? 2. Imagine Eurostat does not reform the legal framework: would the problem

improve, stay the same or worsen in the next 10 years?

For answering this second question, please think of the possible evolution of the

characteristics of the changing environment of European social statistics, namely

of the increasing data needs and expectations from the users, increasing

competition on the information market, availability of new data sources, pressure

on resources (cost and response burden) and innovation in methodology and IT.

In a second step you will be asked to help us assess what reasons might be behind

the described problems.

In the section 'POLICY OPTIONS' you will have the opportunity to directly express your

views about different policy options for EU legislation in the area. In the section

'COST' we try to assess the cost implications of those options. Given the known

challenge of comparing information about administrative cost and burden from

different EU Member States, the information to be provided will be kept limited in

detail and precision.

Parts of this questionnaire might be relevant for other statistical authorities in your

country or for other departments within the National Statistical Institute. We would be

grateful if you could consult or try to engage these whenever necessary.

Please note that the information you provide for this consultation will be interpreted

as representing the views of your National Statistical Institute and can be published as

such.

30

Section 1: Identification of respondents, modes of data collection and use of

administrative data

Q.1.1 Please provide your contact details below:

Family name, First name (mandatory)

Title (optional)

Organisation/institution (mandatory)

Function (optional)

Country (mandatory)

Address (optional)

E-mail address (mandatory)

Telephone number (mandatory)

Would you like to receive by e-mail a summary of the main outcome of this

consultation?

Yes

No

Q.1.2 Which data collection mode(s) does your institution use or intend to use

in the next five years for the following surveys?

ACTUAL USE TODAY: Please estimate the share of data records in % (rough

estimate) obtained by each mode of data collection.

In cases where the NSI attempted to use successively different modes (e.g. CAWI

first, and if no reply: classical face to face second) please refer to the collection mode

that finally provided the complete information.

The total for each row should add up to 100%

Survey

Share of data records in % (rough estimate) obtained by…

Actual use today

Post

mail

survey

(paper;

self-

adminis-

tered)

Classical

Face to

face

CAPI

face

to

face

CATI

CAWI

(self-

adminis-

tered)

Administrative

data Other

Current

sample

size for

the

whole

nation

Total

EU-SILC 100

EU-LFS 100

AES 100

EHIS 100

HH ICT 100

HBS 100

HETUS 100

INTENDED FUTURE USE: Please estimate the share of data records in % (rough

estimate) obtained by each mode of data collection

31

The total for each row should add up to 100%

Survey

Share of data records in % (rough estimate) obtained by…

Intended future use

Post

mail

survey

(paper;

self-

adminis-

tered)

Classical

Face to

face

CAPI

face

to

face

CATI

CAWI

(self-

adminis-

tered)

Administrative

data Other

Current

sample

size for

the

whole

nation

Total

EU-

SILC

100

EU-

LFS

100

AES 100

EHIS 100

HH

ICT

100

HBS 100

HETUS 100

where:

EU-SILC - European Union Statistics on Income and Living Conditions, Eurostat

EU-LFS - European Union Labour Force Survey, Eurostat

AES - Adult Education Survey, Eurostat

EHIS - European Health Interview Survey, Eurostat

HH ICT - Survey on ICT usage in households, Eurostat

HBS - Household Budget Survey, Eurostat

HETUS - Harmonised European Time Use Survey, Eurostat

32

Q.1.3 What is the current sample size for the whole nation and the intended

future sample size?

Current sample size (in 1000)

Intended future sample size (in

1000) – only answer if

different from the current

sample size

EU-SILC

EU-LFS

AES

EHIS

HH ICT

HBS

HETUS

Q.1.4 Would you like to provide any further information/comment on the

question in the section 'Modes of data collection and use of administrative

data'?

Yes

No

33

Section 2: The problems

Main problem 1:

Loss of societal relevance of the European social statistics in view of changing

users’ needs and expectations

European social statistics needs to: • target existing user needs • adapt to newly emerging user needs • offer data that meet expectations of users in today's 'internet age'

More information on MAIN PROBLEM 1: click here

Q.2.1 How important is the problem of a potential loss of societal relevance of

the European social statistics today in view of changing users’ needs and

expectations?

Please rate it on a scale from 0 to 10, where:

0 – not important at all 1 – insignificant 10 – critical

0 1 2 3 4 5 6 7 8 9 1

0

X

Q.2.2 Is the situation likely to improve, stay the same or worsen in the next

10 years, if Eurostat does not reform the legal framework?

Significant

improvement

Slight

improvement

Stay the same Slightly worse Significantly

worse

34

Reasons behind the loss of societal relevance of the European social statistics

HYPOTHESIS 1

There is a loss of societal relevance of data if legal fragmentation leads to

fragmented output

Key issues:

The societal relevance of European social data is at risk if legal fragmentation leads

to: • difficulties to combine the data available from different sources, create coherent

data pools, and integrate the data to better meet existing data needs; • problems to compare results for the same variable when data comes from

different sources; • an unrelated production of separated datasets ('stovepipes'); • a lack of technical standardisation: data linking is difficult due to the different

types of datasets, different coverage, deviating definitions of variables, different reference dates and formats, etc.

More information on HYPOTHESIS 1: click here

Q.2.3 To what extent does the fragmented legislation, production and

compilation of European social statistics reduce the overall value of European

social statistics today?


0 = I cannot see any direct cause 1 = to a very small extent 10 = to a very large extent

0 1 2 3 4 5 6 7 8 9 10

Q.2.4 If Eurostat does not reform the legal framework, how will this situation evolve?

Significant

improvement

Slight

improvement


worse

HYPOTHESIS 2:

There is a loss of societal relevance of data if rigid legislation leads to

outdated contents

Key issues:

The societal relevance of European social data is at risk: • if it takes too long to adapt the contents to new information needs that

relate to new societal challenges and changes (e.g. on globalisation, migration or sustainability);

• if the legislative setup is a barrier for agile response to changing societal information needs and requires that (a) changes must be made for each domain separately and (b) due to lengthy legislative procedures;

• because today's information market can meet the demand for quick and inexpensive data, albeit at (sometimes) dubious quality (e.g. low accuracy);

35

• because under the influence of the Internet, users are getting more and more used to being able to find in a timely way the exact information they were looking for.


Q.2.5 To what extent do you consider that the slow capacity to adapt to new

information needs reduce the overall value of European social statistics

today?


0 – I cannot see any direct cause 1 – to a very small extent 10 – to a very large extent

0 1 2 3 4 5 6 7 8 9 10

Q.2.6 Is the situation likely to get improve, stay the same or worsen in the

next 10 years, if Eurostat does not reform the legal framework?

Significant

improvement

Slight

improvement


worse

Q.2.7 Would you consider any other reason important for the loss of societal

relevance of the European social statistics?

YES

Please explain:

NO

_____ END OF QUESTIONS ON MAIN PROBLEM 1______

Main problem 2:

Lack of coherence and comparability of European social statistics

European social statistics needs to: • ensure full coherence of the overall output of social statistics • make it possible to integrate and compare data from different surveys • address quality issues related to innovative production methods and IT,

and alternative data sources


Q.2.8 How important is the problem of less and/or uncertain quality (e.g. lack

of coherence and comparability) of the European social statistics today?



0 1 2 3 4 5 6 7 8 9 10

36

Q.2.9 Is the situation likely to improve, stay the same or worsen in the next 10 years, if Eurostat does not reform the legal framework?

Significant

improvement

Slight

improvement


worse

Reasons behind the lack of coherence and comparability of European social

statistics

HYPOTHESIS 3:

A lack of standardisation leads to less and / or uncertain data quality

(coherence and comparability)

Key issues:

The coherence and comparability of the data is compromised if: • the legal fragmentation causes a lack of standardisation of concepts, variables,

breakdowns, classifications; • there are no standards to assess the quality of sampling frames; • adaptation of technical matter related to standardisation is time consuming

because it is not internalised into the European Statistical System (and therefore requires heavy legal procedures);

• more and more NSIs are incorporating new sources in their production process, but methodology, definitions and specifications differ from one data collections to another-


Q.2.10 To what extent do you consider that the lack of standardisation is

causing a decrease of the data quality (coherence and comparability) today?



0 1 2 3 4 5 6 7 8 9 10


Significant

improvement

Slight

improvement


worse

37

HYPOTHESIS 4:

There will be less and/or uncertain data quality in the future if the EU legal

framework not adapted to innovative data collection methods and IT

Key issues:

The quality of European social statistics will decrease and/or become difficult to assess

if the EU legislative framework: • does not promote the use innovative data collection methods and IT tools; • makes the adaptation of technical matter time-consuming because it is not

internalised into the European Statistical System (and therefore requires heavy legal procedures);

• develops no quality assessment that takes innovative data collection methods and IT tools appropriately into account;

• becomes outdated given the fact NSIs are increasingly working on or with these methods.


Q.2.12 To what extent do you consider that insufficient provisions about

innovative data collection methods and IT in the current EU legal framework

lead to insufficient/uncertain quality of European social statistics today?



0 1 2 3 4 5 6 7 8 9 10

Q.2.13 If Eurostat does not reform the legal framework, how will this

situation evolve?

Significant

improvement

Slight

improvement


worse

Q.2.14 According to your experience, what quality issues should be addressed

in order to use innovative data collection modes for the purposes of European

social statistics?

1500 character(s) maximum (1500 characters left)

Please just provide key words

HYPOTHESIS 5:

There will be less and/or uncertain quality (in terms of coherence and

comparability) in the future if the EU legal framework is not adapted to a

situation where new sources and multi-source integration are used to their

full potential

38

Key issues:

The quality of European social statistics will decrease and/or become difficult to assess

if the legal framework: • does not promote the use of alternative data sources and the integration

of multiple data sources; • does not address the fact that the divergence of concepts used in administrative

data sources can hamper the production of EU-wide comparable data ; • makes the adaptation of technical matter time-consuming because it is not


• develops no quality assessment that takes the use of alternative data sources and the integration of multiple data sources into account;

• becomes outdated given the fact NSIs are increasingly working on or with multiple data sources and their integration.


Q.2.15 To what extent do you consider that insufficient provisions about

administrative data sources and multi-source integration in the current EU

legal framework lead to insufficient/uncertain quality of European social

statistics (in terms of coherence and comparability) today?



0 1 2 3 4 5 6 7 8 9 10

Q.2.16 If Eurostat does not reform the legal framework, how will this

situation evolve?

Significant

improvement

Slight

improvement


worse

Q.2.17 According to your experience: What quality issues should be addressed

in order to use administrative data for the purpose of European social

statistics?

Please just provide key words (open text maximum ½ A4 page):

Q.2.18 Would you consider any other reason important for the lack of

coherence and comparability of European social statistics?

YES

Please explain:

NO


39

Main problem 3:

Inefficiency of European social statistics (cost and response burden)

European social statistics needs to be efficient in terms of: • taxpayers' money used for their production • the burden to those who reply to surveys


In this section, the term 'inefficiency' does not refer to a possible lack of

efficiency within the current business processes, but rather to the gap of the

efficiency that current business processes can provide to the efficiency

possible if alternative, more modern business processes would be applied

(e.g. modernisation of data collection, production and dissemination

systems).

Q.2.19 How important is the problem of potential inefficiency of European

social statistics today (cost and response burden that could be avoided under

a better setup of European statistical legislation)?



0 1 2 3 4 5 6 7 8 9 10



Significant

improvement

Slight

improvement


worse

40

Reasons behind the Inefficiency of European social statistics (cost and

response burden)

HYPOTHESIS 6:

There is inefficiency if EU legislation does not support survey integration and

the usage of modern data collection tools

Key issues:

There is inefficiency in the production of European social statistics if the EU legislative

framework: • does not promote the use innovative data collection methods and IT tools; • makes the adaptation of technical matter time-consuming because it is not


• becomes outdated given the fact NSIs are increasingly working on or with these methods.


Q.2.21 To what extent do you consider that the use of innovative data

collection methods and IT can increase efficiency in terms of…



[Q.2.21.A] … COST to the NSI?

0 1 2 3 4 5 6 7 8 9 10

[Q.2.21.B] If Eurostat does not reform the legal framework, how will this

situation evolve?

Significant

improvement

Slight

improvement


worse

[Q.2.21.C] … BURDEN to the respondent?

0 1 2 3 4 5 6 7 8 9 10

[Q.2.21.D] If Eurostat does not reform the legal framework, how will this

situation evolve?

Significant

improvement

Slight

improvement


worse

41

Q.2.22 To what extent do you consider that the cost-reducing potential of

innovative data collection methods can be realised better when it is possible

to decompose one long questionnaire into several shorter questionnaires ?



0 1 2 3 4 5 6 7 8 9 10

Q.2.23 Do you consider that the EU's current legal setup is too inflexible, i.e.

that it requires questionnaires that are too long to support alternative data

collection modes and instead reports about variable crossings that are

probably not really required?


0 – I cannot see any direct link between innovative data collection and the length of questionnaires 1 – I don't think at all that the current EU legislation on social surveys is too inflexible 10 – I think very much that the current EU legislation on social surveys is too inflexible

0 1 2 3 4 5 6 7 8 9 10

HYPOTHESIS 7:

There is inefficiency if EU legislation does not support use of alternative

sources and integration of multiple sources

Key issues:

There is inefficiency in the production of European social statistics if the EU legislative

framework: • does not promote the use of alternative data sources and the integration

of multiple data sources; • does not address the fact that the divergence of concepts used in administrative

data sources can hamper the production of EU-wide comparable data ; • makes the adaptation of technical matter time-consuming because it is not


• becomes outdated given the fact NSIs are increasingly working on or with multiple data sources and their integration.


Q.2.24 Compared to today's situation in your country: to what extent do you

consider that the use of alternative data sources and their integration can

increase efficiency in terms of…



42

[Q.2.24.A] … COST to the NSI?

0 1 2 3 4 5 6 7 8 9 10

[Q.2.24.B] If Eurostat does not reform the legal framework, how will this

situation evolve?

Significant

improvement

Slight

improvement


worse

[Q.2.24.C] … BURDEN to the respondent?

0 1 2 3 4 5 6 7 8 9 10

[Q.2.24.D] If Eurostat does not reform the legal framework is undertaken,

how will this situation evolve?

Significant

improvement

Slight

improvement


worse

Q.2.25 Would you consider any other reason important for the inefficiency of

European social statistics (cost and response burden)?

YES

Please explain:

NO


43

Main problem 4:

Insufficient progress on methods and procedures for the European social

statistics

European social statistics needs to be up-to-date with the rapid innovation

going on in the domain of statistical methodology and IT.


Q.2.26 How important is the insufficient progress on methods and procedures

for the European social statistics today?



0 1 2 3 4 5 6 7 8 9 10



Significant

improvement

Slight

improvement


worse

Q.2.29 Do you have any further comments on the issue that European social

statistics need to be up-to-date with the rapid innovation going on in the

domain of statistical methodology and IT?

YES

Please explain:

NO


Final question in this section

Q.2.29 Do you consider that this list of the main problems of European social

statistics is complete?

YES

NO

Q.2.30 If no, what other problems of European social statistics do you have in

mind and what would be their potential causes?

Please describe them

44

Section 3: Impact of the policy options

The system of policy options has been designed as a cascade of options where the

next higher option contains the features of the option below, plus additional aspects.

The cascade describes increasing levels of ambitions, innovation, but therefore also

uncertainty.

An overview of the policy options is presented in the table below. A more detailed

description of policy options can be obtained by clicking on the number in the table.

45

Relationship between the hypothesis and the policy options

There is a close relationship between the hypothesis described above and the different

policy options. Although the relationship is not a perfect one-to-one relationship, it

can be validly described by this table.

For the crossing of routing 6 'does legislation support survey integration' it has to be

quantified to what extent the cost-reducing potential of innovative data collection

methods can be realised better when it is possible to decompose one long

questionnaire into several smaller questionnaires.

[1] Given that policy option 2 describes integrated legislation, promotes some

standardisation of variables and classifications. However, this standardisation would

fall short of issues related to the modernisation of methodology, IT related issues,

data sources, quality reporting etc.

Features common to policy options

All policy options maintain the current basic legal setting of European statistics: • The ‘European Statistical Law’ – Regulation 223/2009, which establishes the

scope, purpose, governance, and decision-making structures of European statistics

• The multi-annual European Statistical Programme (ESP), renewed every 5 years, and which defines the work programme for European statistics for that period. In order to implement the ESP, the Commission adopts annual work programmes which set out the annual objectives and the expected results.

The current legal base of European statistics is set by Regulation (EC) Nº 223/2009 of

the European Parliament and of the Council of 11 March 2009[1], which in particular

updates the Council Regulation (EC) No 322/97 on Community Statistics, and other

relevant acts, and establishes the ESS. Regulation (EC) No 223/2009 constitutes the

legal basis for the preparation of the European statistical programme, providing the

framework for the development, production and dissemination of European statistics,

the main fields and the objectives of the actions envisaged for a period not exceeding

five years.

[1] http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2009:087:0164:0173:en:PDF

46

Policy option 1: Not Integrated, Low on innovation, Fixed programme

Policy option 1 describes the current situation, i.e. the situation if there was no policy

change (baseline scenario). It should be noted that over the next years, the baseline

would be dynamic regardless of not acting on the issues at hand. Changes would occur

in either case.

Fragmented legislation for social statistics

There is no framework legislation on European social data collections, but a set of

unrelated Council/EP regulations for each data collection, namely for the Labour Force

Survey (LFS), EU Statistics on Income and Living Conditions (EU-SILC), Adult

Education Survey (AES), European Health Interview Survey (EHIS), and Survey on ICT

usage in households (ICT-HH). Two European surveys are conducted on the basis of

an informal agreement only, namely the Household Budget Survey (HBS) and the

Harmonised European Time Use Survey (HETUS). This legal fragmentation makes the

adoption of common legal provisions for several domains complicated and uncertain,

and thereby reduces the system’s capacity to adapt to both new data requirements

and to modern production environments.

Not promoting innovation

The current regulations contain few provisions that relate to the challenges of new

data sources, or to the modern methodology and information technology that are

increasingly used in the Member States to integrate data from different sources. This

can lead to a situation where European statistical legislation becomes gradually aloof

to the technical realities of data production in the Member States. As a consequence,

modernisation and technological progress in the statistical production in the Member

States might gradually undermine EU-wide data comparability.

Fixed programme

The current domain specific regulations fix the statistical programme of European

social data collections8. They regulate both, programme and technical implementation

for that particular domain. The indiscriminate mixture of programming (policy) and

implementation (technicalities) in the current legal setup can reduce the quality of

decision-making. The current legal setup often refers very technical matter to the

Parliament and Council, diverting their attention away from the essential task to

balance European information requirements against cost and response burden. This

results in legislative delays and rigidity that could be overcome if European statistical

legislation corresponded better to the statistical governance based on the principal of

division of responsibilities as envisioned in the EU's statistical law.

[1] There is some degree of flexibility in the measurement of emerging topics through

the so-called ‘ad hoc’ modules of the LFS. The Regulation 0577/1998 of 9 March 1998

on the organisation of a labour force sample survey in the Community specifies that a

further set of variables may be added to supplement the information from the core

questionnaire of the EU-LFS. The ad hoc modules are set by Commission regulation by

group of three years. See http://ec.europa.eu/eurostat/statistics-

explained/index.php/EU_labour_force_survey_-_ad_hoc_modules.

8 There is some degree of flexibility in the measurement of emerging topics through the so-called ‘ad hoc’ modules of the LFS. The Regulation 0577/1998 of 9 March 1998 on the organisation of a labour force sample survey in the Community specifies that a further set of variables may be added to supplement the information from the core questionnaire of the EU-LFS. The ad hoc modules are set by Commission regulation by group of three years. See http://ec.europa.eu/eurostat/statistics-explained/index.php/EU_labour_force_survey_-_ad_hoc_modules.

47

Q.3.1 Do you want to comment on policy option 1?

Q.3.2. Do you perceive any risks related to the implementation of this policy

option?

Policy option 2: Integrated, Low on innovation, Fixed programme

Integrated legislation for social statistics

Policy option 2 describes an integrated EP and Council regulation encompassing all

important aspects of statistical data collections (e.g. definition and list of variables,

definition of statistical units, data sources and data collection, methodology, reference

periods, frequency, regionalisation, precision requirements, other quality

requirements). The regulation might refer issues of technical implementation to

implementing acts.

Policy option 2 will realise the advantages of integrated legislation, e.g. increase the

consistency of definitions and specifications of variables, classifications, statistical

populations and observation units. There would be a consistent use of terminology.

Policy option 2 would avoid overlaps in the programme.

Not promoting innovation

Still, option 2 would contain few provisions that relate to the challenges of new data

sources, or to the modern methodology and information technology that are

increasingly used in the Member States to integrate data from different sources.

Fixed programme

The regulation under policy option 2 would fix the statistical programme of European

social data collections in quite some detail, including the variables, periodicities,

regional coverage, precision requirements, transmission deadlines etc. There would be

little flexibility to target the output optimally to existing user needs and to react to

new needs for data.



option?

Policy option 3: Integrated, High on innovation, Fixed programme



important aspects of statistical data collections, and referring to issues of technical

implementation to implementing acts.

Promoting innovation

In contrast to policy option 2, policy option 3 would contain provisions that relate to

the challenges of new data sources, or to the modern methodology and information

technology that are increasingly used in the Member States to integrate data from

different sources. As a matter of principle, the regulation would be open to

conventional data sources (e.g. surveys) and alternative data sources like

48

administrative registers, and lay the ground for an appropriate quality assessment and

reporting. There would be provision on sampling frames. Provisions on feasibility

studies would support future development of processes and output.

Fixed programme

However, the regulation under policy option 3 would still fix the statistical programme

of European social data collections in quite some detail, and have little flexibility to

target the output optimally to existing user needs and to react to new needs for data.



option?

Policy option 4: Integrated, High on modernisation, Multi-annual

integrated output programme with outline and burden limitation



important aspects of statistical data collections, and referring issues of technical


High on modernisation

Policy option 4 would contain provisions that relate to the challenges of new data

sources, or to the modern methodology and information technology.

Multi-annual integrated output programme with outline and burden limitation

In contrast to policy option 3, policy option 4 would provide more flexibility to target

the output optimally to existing user needs and to react to new needs for data. It

would authorise delegated acts to set up a multi-annual planning of statistical output,

and implementing acts to provide the technical specifications of the data collections.

An increased flexibility of programming statistical output is desirable from a user's

point of view, but also from the producers' viewpoint because it can bring the

potential of innovative data collection methods and IT to reduce cost to their full

potential. However, flexibility of programming also translates into insecurity in regard

to cost and response burden for data producers.

Flexibility in programming has to be understood not only as the possibility to enlarge

the scope for data collection, but also to reduce it (for instance, as a result of the

identification of ‘negative priorities’ in the multi-annual programme9).

For example, where there are provisions on possible changes to the programme, it is

stated that those should not impose significant administrative burden or cost to the

Member States or on the respondents to surveys. In addition, policy option 4 would

contain provisions on financial support of the EU towards the cost of developments of

9 Since 2010, the priority setting mechanism also involves the identification of “negative priorities”. Activities which were declared negative priorities have been removed or delayed because of four main reasons: lack of resources; time constraint; technical or administrative problems; revisions or rescheduling.

49

data collection, specified outputs and/or methodology. This may include EU financial

support (grants) for statistical activities such as:

• The development and/or implementation of data collections for social

statistics, including sampling frames, which may be particularly expensive

during the first four years;

• Developing new statistics or methodologies, including feasibility and pilot

studies;

• Collecting statistics with a one-off character or implemented for the first time.

Moreover, policy option 4 would outline the output possibly required on the basis of

the regulation in terms of topics that could be covered by the different domains of

social statistics. The topics would be specified in further detail. The basic act would

further outline the range of possible periodicities, regional coverage, precision

requirements, transmission deadlines etc. The limitation of the possible scope of the

programme would limit the range of possible increases to cost and burden. The

regulation would limit the rate of change by limiting the maximum number of changes

within a given time to a specified maximum. Provision of minimum delays for an

advance planning would round up the safeguards.



option?

Policy option 5: Integrated, High on modernisation, Multi-annual

integrated output programme with outline and burden limitation

Policy option 5 would have the same features like policy option 4, but contain no

safeguards against strong increases of cost and response burden, neither directly, not

via a limitation of the scope of the data collection programme. The European

Commission would be empowered to adopt implementing measures specifying

(including enlarging) the work programme, without providing for financial support to

Member States.



option?

Q.3.11 A multi-annual planning (as foreseen in policy options 4 and 5) can

guarantee the early, bottom-up involvement of the National Statistical

Institutes. Do you think that a multi-annual planning of the concerned social

surveys can facilitate acceptance of any draft legislation?

YES

NO

50

Q.3.12 Do you wish to add any other comment that might be interesting in the

context of this Impact Assessment?

Please detail

Baseline – quantitative assessment

This section intends to collect information about overall current annual costs of the

social surveys broken down by broad activity groups (based on the GSBPM phases).

Q.4.1 Please fill-in the table below indicating the total annual cost for each of

the seven statistical data collection and the break down (%) by statistical

activities.

Note: For those surveys that are conducted more than once per year, please indicate

the average cost for the whole year.

For surveys that are not conducted every year, please indicate the average cost for

one complete cycle when the survey is conducted (i.e. do not annualise the cost by

dividing it by the number of years of the period form on survey cycle to the next).

Cost to the NSI

Response

burden

Sample

size

Time

spent

per

respon-

dent

Total annual cost Design Data

collection

Treatment

and

dissemination

In Mio

Euros In % of total costs

In

1000

In

minutes

EU-

SILC 100

EU-LFS 100

AES 100

EHIS 100

HH ICT 100

HBS 100

HETUS 100

Explanation on the abbreviations for the surveys: click here

Explanation on cost: click here

Explanation on response burden: click here

Explanation on activity groups: click here

This questionnaire considers a grouping of the UNECE GSBPM (The Generic Statistical

Business Process Model) phases and considers the following 3 categories of activities: - ‘Design’ – which comprises the GSBPM phases ‘Specify needs’, ‘Design’ and

‘Build’; - ‘Data collection’ – corresponding to GSBPM phase ‘Collect’; - ‘Treatment and dissemination’ – which comprises the GSBPM phases ‘Process’,

‘Analyse’, ‘Disseminate’ and ‘Evaluate’

51

Conceptual model (this section is explanatory only)

The change in costs (Total ∆) can be decomposed according to the different features

of the scenarios. Since policies are described in cascade by adding features we

consider an additive model for the Total ∆, as described in the table below.

Where:

∆ Integrated legislation

The change only due to integrating the legislation (e.g. cost of legal advisors,

transposition into national law, etc.).

∆ Modernisation in view of statistical production (low)

The change only due to considering a few provisions that relate to the challenges of

new data sources, or to the modern methodology and information technology to

integrate data from different sources.

∆ Modernisation in view of statistical production (high)

The change only due to applying new methods, sources, IT, etc. It is expected that

the initial investment is high, but it is largely recovered in the long run.

∆ Flexible statistical programming

The change due to modifying (increasing or decreasing) the work programme. Under

the hypothesis of fixed programme (policy options 2 and 3) it is 0 (zero). • Policy option 4 contains provisions that limit the increase of cost and response

burden (e.g. there would be an outline of transmission obligation in the basic act and provision on cost, burden and financing)

• Policy option 5 contains no provisions that limit the increase of cost or response burden

Response burden

The conceptual model does not take into account the cost due to changes to the

response burden.

52

The composition of ∆ by policy option

Explanation on general expectations: click here

Qualitative assessment on potential cost impacts (∆)

Q.4.2 Imagine a situation where the legislation on European social surveys is

integrated and promotes innovative data collection modes and sources: what

would be the potential medium-term impact (around 5 to 10 years) on the

cost?

Significantly

increase

Increase (Insignificant) Decrease Significantly

decrease

Design

Data

collection

Treatment &

Dissemination

Q.4.3. If the potential medium-term impact on the cost will increase or

significantly increase, who will cover these costs?

The NSI exclusively

The costs will be shared between the NSI and other authorities concerned e.g.

administrative data providers

Please detail and (if possible) mention the % share of costs between the NSI & other

authorities

Q.4.4 Survey integration and the use of innovative data collection modes and

sources can create synergies and free resources which could be employed in

53

order to improve the responsiveness and user-orientation of surveys. Under

the assumption that new legislation on European social surveys generates no

change in the total cost, please rate on a scale from 0 to 10 the potential

effect (improved responsiveness and user-orientation thanks to survey

integration and the use of innovative data collection modes and sources).

where:

0 – not important at all

1 – insignificant

10 – very substantial

0 1 2 3 4 5 6 7 8 9 10

EU-SILC

EU-LFS

AES

EHIS

HH ICT

HBS

HETUS

Q.4.5 What would be the impacts of the risks identified in the previous

section (in the answers to Q.3.2, Q.3.4, Q.3.6, Q.3.8 and Q.3.10) on the costs?

And whom would affect?

For the risks related to the implementation of Policy option 1, please detail





Q.4.6 Please provide any information on the cost and/or burden of the

surveys that you might consider interesting in the context of this impact

assessment.

You could either add any relevant information in the text box below or

provide separate files.

54

ANNEX TO THE QUESTIONNAIRE

More information on MAIN PROBLEM 1:

The role of official statistics in the current society is fundamental for decision-makers.

At European level, European statistics are important for the development,

implementation, monitoring and evaluation of EU policies.

Requirements for social statistics develop continuously with new challenges to the

changing societies in Europe like ageing populations, changing forms of living

together, new challenges in the labour market, risks to people’s welfare, globalisation,

etc. Since some of these topics are interrelated and mutually dependent, cross-domain

statistics and integrated data sets are increasingly demanded. EU legislation needs

to target existing user needs well and respond to emerging user needs.

Moreover, Internet and social media have raised users’ expectations to

receive information quick and personalised service10. 'Data fluent' users demand

more quantity and quality data, ready for re-utilisation, as a result of increased

availability of data processing and visualisation software. The culture of 'Open Data'

forces the European Statistical System to increase the dissemination of anonymised

micro-data.

BACK TO QUESTIONNAIRE

More information on HYPOTHESIS 1:

The measurement of complex phenomena which are related to different statistical

operations (e.g. globalisation, which may have an impact on social areas such as

labour, public health and migration) suffers from the technical fragmentation- which in

turns is caused by legal fragmentation. The societal relevance of data is at risk

because of a limited capacity to combine the data available in ESS. Technically, this is

possible as NSIs and other organisations that used new sources (i.e. administrative

data) or methods (i.e. statistical matching techniques) have proven. However, the

'stovepipe' model of production of separated datasets makes it hard to create

coherent data pools (data warehouses), and to present the available statistical data to

better meet existing data needs. This problem is enlarged by the fact that the various

datasets are not standardised in regard to technical matter: linking is difficult due to

the different types of datasets, different coverage, definitions and identifiers,

references dates and formats. Changes in classifications (administrative or statistical)

make it more complicated.



The emerging information needs for the identification of new societal challenges and

changes (for example on ageing populations, changing labour markets, globalisation,

migration or sustainability), and the increasing use in EU policies of quantitative

objectives and thresholds making use of statistical indicators increasingly confront

Eurostat and the NSIs with high quality requirements – including timeliness – for new

indicators. Moreover, under the influence of the Internet, users are getting more and

more used to being able to find in a timely way the exact information they were

looking for. Today's information market can meet the demand for quick and

inexpensive data, albeit at (sometimes) rather dubious quality.

Currently, it takes the ESS considerable time to start the production of new data. New

user needs may require changes in the definition of variables collected in the social

surveys, new relevant breakdowns, cross-tabulation from different sources or other

10 Personalised in the sense that for example crosstabs are provided at a requested level of detail, not in the sense that ‘new data’ is collected.

55

methodological innovations. Any of these technical changes has to be reflected in the

lengthy modification of legal acts.

The Heads of the Statistical Offices in the EU Member States have acknowledged the

need for social statistics to become more flexible and responsive to requests for new

or improved data, and for the ESS to strengthen its capacity for reaction and

adaptation.



EU legislation needs to ensure full coherence of the overall output of social

statistics. It should be possible to integrate and compare data from different surveys.

Currently, this is often not the case because definitions of statistical variables, their

breakdowns and classifications are set in survey-specific pieces of legislation.

Moreover, modernisation and technological progress in the statistical production in the

Member States – an increasing reality – might gradually undermine EU-wide data

coherence and comparability if EU legislation does not relate sufficiently well to

innovative production methods and IT, alternative data sources, and to quality issues

around them.



Legal fragmentation causes a lack of standardisation of concepts, variables,

breakdowns, classifications. This is strengthened by the fact that technical matter is

not internalised into the ESS (i.e. decided by the ESS institutions). The work on

standardisation of social variables has identified a number of variables that appear in

different statistical operations. Some of them have achieved a high level of

standardisation, while other are still in process and still present discrepancies across

surveys.

If the legal framework that provides definitions, formats, etc. (often bound to one

particular survey) is not adapted to the process of standardisation, the statistical

quality can decreases, in particular in regard to overall coherence and comparability.

This problem is enlarged by new data needs and expectations of users.



If a legal setup sets no provisions for the use of innovative data collection methods

and the integration of alternative data sources, statistical quality might decrease

and/or become difficult to assess given the fact that NSIs are working on or with these

methods.



Any divergence of concepts used in sources other than statistical surveys (such as

administrative registers) can hamper the production of cross-nationally comparable

social data. Given the fact NSIs are increasingly using multiple data sources,

statistical quality might decrease and/or becomes difficult to assess, if the legal

framework sets no provisions for the use of alternative data sources and the

integration of multiple data sources.


56


EU legislation should avoidable cost to the Member States and burden to the

respondent. The requested output (data to be provided to Eurostat) should be

produced using no more than the needed input. For instance, using registers collected

for administrative purposes or combining sources could avoid collecting (and

processing) information from respondents more than once by different surveys. This

concerns both, NSIs' budgets (including time spend by NSI staff) and the burden

(time and money spend) for respondents.

Administrative burden has become an important political issue in the past two

decades. In addition to the factual burden, the perceived burden (‘annoyance’) is

relevant. A high perceived burden could for instance result in deceasing response

rates. The economic crisis further increases the importance of budgetary efficiency.

Most NSIs have been forced to cut back on expenses. In part, they use innovative

methodology for data collection and processing and innovative sources to do so.



Modern data collection and pooling methods allow integrating and/or modularising

surveys without compromising the quality of the data needed by the users. Such

approaches promote the use of innovative data collection methods and IT tools that

can reduce cost and response burden. NSIs are working on such approaches.

Consequently, the use of modern data collection techniques should be sufficiently

supported by the European legislation because the fixed programming results in a

slow adaption to required changes.



NSIs are using increasingly alternative sources (especially administrative data) and

integrate different data sources in view of producing consistent statistical output. This

is done in a quest to substantially reduce the running cost of statistical production.

The current legal does not have adequate provisions to govern these changes.

The EU legislative framework can lead to inefficiencies in term of costs for NSIs and

burden for respondents if it does not promote the use of alternative data sources and

the integration of multiple data sources.



EU legislation should support the integration of business processes which is

increasingly happening in the NSIs. Modern statistical production comprises

increasingly integrated business processes in order to exploit the strongly growing

capacities of information technology. Such integrated business processes comprise

e.g. multi-source/cross-domain data integration, integrated management of registers

and statistical frames, creation/maintenance of common infrastructure, common

standards and tools for data processing, exchange and dissemination and standardised

quality assessment. Very often, developments in one MS could be implemented in

others, leading to cost reduction, with the condition that procedures are standardised.

The work on standardising the statistical business process undertaken by Eurostat,

UNECE and the NSIs (such as the definition of the Generic Statistical Business Process

Model – GSBPM - and the Common Statistical Production Architecture – CSPA), aims

at facilitating the adoption of common languages and tools for the production of

statistics, but this work is currently done without any supportive legislative basis.


57

Abbreviations for the surveys

EU-SILC - European Union Statistics on Income and Living Conditions

EU-LFS - European Union Labour Force Survey

AES - Adult Education Survey

EHIS - European Health Interview Survey

HH ICT - Survey on ICT usage in households

HBS - Household Budget Survey

HETUS - Harmonised European Time Use Survey


Information about costs to the NSIs

For the estimation of the total costs of the survey, each NSI has to take into account

the following cost items: personal costs, running costs, investments. NSIs are kind

asked provide an estimate of the share by statistical activities, but not at the level of

the cost items. However, all cost items should be covered by the estimate.

Personal costs should be estimated taking into account the number, dedication and

category of staff intervening in the operation in all its phases. Support personnel of

the NSI not dedicated to a statistical activity (maintenance, drivers, etc.) should not

be included.

Under running costs the following can be included: transport, publication and

communication, outsource services.

Investments should include those intangible and tangible assets addressing directly

the statistical operations (e.g. IT developments, hardware and software).


Information about response burden

The response burden shall be estimated as time needed to fill in the questionnaires

multiplied with the total sample size for the respective survey in the Member State.

The response burden is generally borne by respondents. Member States might have an

overall estimate of the time needed to fill in one questionnaire and about the sample

size. The calculation of the estimated total response burden will be done by Eurostat.


Information on broad activity groups

The phases of statistical operations, according to the General Statistical Business

Process Model (GSBPM) consist in: 1. Specify needs: Identification and confirmation of statistics’ needs, new

information request, statistical outputs required to meet the user needs and data sources availability checks.

2. Design activities: detailed design of statistical outputs, variables, collection methods and instruments, design frame and sample, design the statistical processing methodology to be applied (including specification of routines for coding, editing, imputing, estimating, …)

3. Build: design collection instruments and its preparation and testing; new and enhance existing components and services needed for the ‘process’ and 'analyse' phases (dashboard functions and features, information services, workflow frameworks,…); components and services needed for the dissemination; technical testing of programmes and routines; test of the statistical business process (small-scale data collection, processing and analysis of the collected data); and activities to put the configured processes

58

and services, into production ready for use (documentation about the process components – manuals, and training).

4. Collect: creation of frame and sample selection (including coordination of samples between instances of the same statistical business process; quality assurance and approval of the frame, …); set up collection (including the collection strategy, planning and training activities, agreeing terms with any intermediate collection bodies, …); run and finalise the collection.

5. Process: activities to clean of data and their preparation for analysis (i.e. data integration; data classification and code; data revision and validation; edit and imputation; derivation of new variables and units; calculation of weights; calculation of aggregates; and finalisation of data files).

6. Analyse: production and examination of statistical products and prepare them for dissemination (i.e. data transformation into statistical outputs, (e.g. production of indices, trends, recording of quality characteristics); output validation and explanation; application of disclosure controls; and finalisation of outputs 'fit for purpose' and reaching the required quality level.

7. Disseminate: update output systems (formatting and loading data and metadata ready to be put into output databases); production of dissemination products; manage release; promote dissemination and manage users support.

8. Evaluate: evaluation of the success of a specific instance of the statistical business process by: gather evaluation inputs (e.g. feedback from users, process metadata; system metrics; staff suggestions); conduct evaluation (i.e. evaluation report including recommendations for changes if appropriate); and agreement of an action plan based on the evaluation report (including also consideration of a mechanism for monitoring the impact of those actions).


General expectations: • ∆ Integrated legislation ≈ 0

• ∆ Modernisation in view of statistical production < 0 (cost reduction)

• ∆ [Modernisation in view of statistical production and Integrated legislation] < ∆ [Modernisation in view of statistical production] (the potential of innovation to reduce cost becomes stronger if the European social surveys are integrated)

• ∆ Flexible multi-annual statistical programme > 0, but limited for policy option 4 below any significant increase, i.e. the possible increase is smaller than a given ε

• ∆ Flexible multi-annual statistical programme > 0, and not predictable for policy option 5

Please note that changes in the cost of producing statistical data due to the

modernisation processes can occur in any phase of the statistical production chain.

Some examples are given below: - Modernisation of design phase: developing new statistical operations (e.g. to

respond to the ‘GDP and beyond’ initiative), costs of studying the adequacy of new data sources for the production of statistics, improvement of statistical registers, improvement and integration of questionnaires improvement of sampling techniques to ensure coordination of samples, new tools for building questionnaires (e.g. design of catalogues of variables), improving the coherence across social surveys (e.g. standardisation of variables, classifications and breakdowns across surveys), etc.

- Modernisation of data collection phase: technical innovation - use of advanced IT tools for data collection (tablets, web questionnaires, optical lecture, etc.), organisational innovation in data collection (e.g. centralisation of operations, exchange of international flow data between NSIs), combination of

59

administrative and survey data, use of Big Data sources, costs of increasing samples to respond to emerging needs for representative statistical information about population segments or territories, etc.

- Modernisation of treatment and dissemination: adoption of improved estimation methods (e.g. small area estimates), use of data modelling techniques, data matching and linkage, new IT tools for dissemination (e.g. visualisation), innovative dissemination of metadata, etc.

According to the intensity of modernisation, the costs can be classified as low or high:

- ∆ Modernisation in view of statistical production (low) The change only due to considering a few provisions that relate to the challenges of new data sources, or to the modern methodology and information technology to integrate data from different sources.

- ∆ Modernisation in view of statistical production (high) The change only due to applying new methods, sources, IT, etc. It is expected that the initial investment is high, but it is largely recovered in the long run.


60

Annex 2. List of respondents to the questionnaire

Name Organisation/ institution Function Country

Kytir, Josef Statistik Austria Director of Social Statistics Austria

Merckx, Lydia STATISTICS BELGIUM DIRECTOR SOCIAL STATISTICS AND DEMOGRAPHY Belgium

Kostova, Magdalena National Statistical Institute Director of Demographic and Social Statistics Directorate Bulgaria

Deckovic-Vukres, Vlasta Croatian Institute of Public Health Head of Department Croatia

Vlajčević, Mario Croatian Bureau of Statistics Head of the Department Croatia

Onisiforou, Koulia STATISTICAL SERVICE OF CYPRUS HEAD OF THE DIVISION OF DEMOGRAPHY, TOURISM AND SOCIAL STATISTICS Cyprus

Ploug, Niels Statistics Denmark Director Social Statistics Denmark

Kommel, Kutt Statistics Estonia Leading Statistician-Methodologist Estonia

Tarkoma, Jari Statistics Finland Director, Population and Social Statistics Finland

Lenglart, Fabrice INSEE Director of social statistics France

Brand, Ruzth Federal Statistical Office, Germany Director for Health, Social Statistics, Education, Households Germany

Ntouros Giorgos Hellenic Statistical Institute Head of Special Household Surveys Section Greece

Menesi, Eva Hungarian Central Statistical Office Expert Hungary

McMahon, Richard Central Statistics Office Assistant Director General, Social & Demographic Statistics Ireland

Sabbadini, Linda Laura ISTAT Director of Department Italy

Maranda Behmane Central Statistical Bureau of Latvia Director of Social Statistics Department Latvia

Ambrozaitienė Dalia Statistics Lithuania Deputy Director General , Social Statistics Issues Lithuania

Hury, Jérôme STATEC Head of social statistics Luxembourg

Nørgaard, Elisabeth Statistics Norway Director of Social Statistics Norway

Beata Jakubczak Central Statistical Office of Poland Head of European Union Section/ International Cooperation Department Poland

Pereira, Leonor Instituto Nacional de Estatística (Statistics Portugal)

Director of Demographic and Social Statistics Department Portugal

61

Name Organisation/ institution Function Country

Pisica, Silvia National Institute of Statistics Romania Director general, social statistics Romania

Ivancikova, Ludmila Statistical Office of the Slovac Republic Director, Directorate of Social Statistics and Demography Slovakia

Argüeso, Antonio Instituto Nacional de Estadística (INE) Director for socio-demographic statistics Spain

Eklund, Inger Statistics Sweden Director, Statistics Sweden Sweden

Schwyn, Markus Swiss Federal Statistical Office Head of Division Switzerland

Van der Laan, Paul STATISTICS NETHERLANDS Senior policy adviser The Netherlands

Goodwin, Guy Office for National Statistics Director of Social Statistics United Kingdom

62

Annex 3: Script of the interview with Directors of Social Statistics within the

National Statistical Institutes

Introduction

Why do we carry out this interview?

European Social statistics are essential for major policies in the European Union, in

particular social policies. They are currently developed in a legal framework which includes a

number of Regulations and other legal acts. A process of re-designing the legal framework

has been discussed for long between Eurostat and the National Statistical authorities of the

Member States. This requires carrying out an Impact Assessment of the initiatives, in

particular of that of passing a ‘Framework Regulation for the Production of European

Statistics on Persons and Households’.

As part of this Impact Assessment which is being implemented by Eurostat, stakeholders

have to be consulted. Information is required about the impact of the re-design options of

the current legislation on the different groups of stakeholders.

The consultation of the primary stakeholder of this process (i.e. the NSIs) includes a

questionnaire11 and an in-depth interview with a sample of NSIs. The selection of the

participating NSIs has been made considering a series of criteria, such as good

representativeness of the EU geography and characteristics of the national statistical

systems (e.g. register-based, traditional surveys, etc.)

What information is collected through the interviews?

The consultation searches for in-depth qualitative information - supported by quantitative

information where appropriate - about the potential impact of a re-designed legal framework

for European social statistics.

Issues which are of interest include:

• The main problems in the current environment for producing social statistics and

reasons behind such problems;

• Alternative policy options to address the problems;

• The impact of the policy options at national level (incl. change in costs due to

possible new provisions).

Who else is being interviewed?

Stakeholders of this process can be broadly classified in two categories:

• data producers: this category comprises the authorities responsible for the collection

and compilation of social statistics. It mainly includes National Statistical Institutes

(NSIs) of Member States and Eurostat.

• data users: it includes institutional users – national administrations, other organisations,

or professional staff working within the Commission services or DGs and other EU

institutions, as well as other external users such as academics and mass media.

Among the first category (i.e. data producers), the following NSIs are interviewed:

11 Available at: https://ec.europa.eu/eusurvey/runner/NSIs-Consulation-EU-Social-Surveys

63

- CBS Netherlands;

- CSO Poland;

- DESTATIS Germany;

- INE Portugal;

- INSEE France;

- INSSE Romania;

- ONS United Kingdom;

- Statistics Denmark.

As part of this category of stakeholders, the consultation plan includes the consideration of

professional analysts working with EU social statistics that are staff of the following services

and/or organisations and agencies:

- Directorates General of the EC:

▪ DG EMPL;

▪ DG SANTE;

▪ DG EAC;

▪ DG REGIO;

▪ DG HOME;

▪ DG ECFIN;

▪ DG JUST.

- European agencies:

▪ Eurofound;

▪ CEDEFOP;

▪ European Institute for Gender Equality (EIGE);

▪ European Agency for Safety and Health at Work (OSHA).

- and Member(s) of ESGAB.

In addition, the consultation process also includes an Open Public Consultation

addressed to users in general.

What are the contents of the interview?

The following is an indicative script/checklist with suggested discussion topics. However, the

interview will be in the form of an open discussion with one of the consultants selected by

Eurostat to carry out the impact assessment.

Interviewees are able to suggest new topics or disregard some of those suggested if not

particularly relevant in each case.

The contents of the interview and the opinions expressed are confidential and will be treated

only in aggregate and anonymised form.

Would you agree that we record this interview?

[IF YES, RECORD THE SENTENCE “THANK YOU FOR ALLOWING US TO INTERVIEW AND

RECORD YOUR OPINIONS. THEY WILL NOT BE INDIVIDUALLY DISSEMINATED, AND WILL BE

TREATED ONLY AGGREGATED AND ANONYMISED”]

INTERVIEW SCRIPT

DISCUSSION THEME 1: Main problems/challenges in the current environment for

European social statistics and their reasons

64

The production of European social statistics is subject to a changing environment in line with

the overall process for the modernisation of European statistics. This changing environment

is characterised by:

- new data needs and expectations from the users;

- increasing competition on information market;

- availability of new data sources (such as administrative data);

- pressure on resources of national administrations (cost of statistical operations)

- increasing concern on response burden;

- innovation in methodology and IT.

A critical review on the current environment for European social statistics reveals a series of

problems that represent real challenges for moving on in the process of modernisation of

social statistics. Four main problems are considered and described below.

1. Loss of societal relevance of the European social statistics in view of changing users’

needs and expectations

2. Lack of coherence and comparability of European social statistics

3. Inefficiency of European social statistics (cost and response burden)

4. Insufficient progress on methods and procedures for the European social statistics

QUESTIONS – DISCUSSION THEME 1:

Let’s discuss about the relevance of European social statistics

Could you please indicate how important is the problem of a potential loss of societal

relevance of the European social statistics today in view of changing users’ needs and

expectations? Which statistical operations should be most urgently renewed in your country

to satisfy users’ needs?

In your opinion, how would the current situation evolve (in the next 10 years) if Eurostat

does not reform the legal framework?

Do you agree that the potential loss of societal relevance of European social statistics could

be caused by any of the following (A or B) and to what extent?

A) Fragmented output as a consequence of legal fragmentation

B) Outdated contents as a consequence of rigid legislation?

Would an Integrated European framework for Social Statistics tackle these root causes or

not? If yes, to what extent? Would this entail important changes in national legislation?

Would you consider any other reason important for the loss of societal relevance of the

European social statistics?

65

Let’s discuss about the coherence and comparability of European social statistics.

Do you consider that there are problems of quality (e.g. lack of coherence and

comparability) at the level of the European social statistics? Have you identified important

problems of coherence and comparability:

- Between your country and other countries’ data?

- Between statistical operations in your country?



Do you agree that possible coherence and comparability issues at the level of European

social statistics could be caused by any of the following (A, B or C) and to what extent?

A) Lack of standardisation (of concepts, classifications, etc.);

B) Outdated data collection methods and IT tools (not innovating);

C) New sources and multi-source integration are not used to their full potential (e.g.

administrative registers).

Would an Integrated European framework for Social Statistics tackle these root causes or

not? If yes, to what extent? Would this entail important changes in national legislation?

What quality issues/obstacles should be addressed in order to use innovative data collection

modes (e.g. from social media or other big data sets) for the purposes of European social

statistics?

What quality issues/obstacles should be addressed in order to use administrative data for

the purpose of European social statistics?

Would you consider any other reason important for the lack of coherence and comparability

of the European social statistics?

Let’s discuss about the inefficiency (cost and response burden) of European social

statistics. We will go back to specific issues on cost of the different policy options later

during this interview.

Could you please indicate how important is the problem of potential inefficiency of European

social statistics today in terms of cost?

Could you please indicate how important is the problem of potential inefficiency of European

social statistics today in terms of response burden?



Do you consider that any of the following options (A or B), could cause inefficiencies at the

level of European Social Statistics and to what extent?

A) Insufficient support, through EU legislation, to survey integration and use of modern

data collection tools;

B) Insufficient support, through EU legislation, for the use of alternative sources and

integration of multiple sources.

Do you consider that the EU's current legal setup requires questionnaires that are too long

and asks for variable crossings that are probably not really demanded by users?

Would EU legislation containing adequate provisions for supporting survey integration and

use of modern data collection tools and the use of alternative sources and integration of

66

multiple sources, increase efficiency in terms of: (1) cost to the NSI and/or (2) burden to

the respondent? Would this entail important changes in national legislation?

Do you think that decomposing long questionnaires into modules or shorter questionnaires

can help in reducing cost with innovative data collection methods (e.g. administrative data)?

Would you consider any other possible cause of inefficiency of the European social statistics?

Let’s discuss about the insufficient progress on methods and procedures for European

social statistics.

Do you think that the progress on methods and procedures for the European social statistics

is sufficient?

Do you think that progress is hampered by the current legal setting?



Additional questions on problems and reasons

Can you think of any other problem of European social statistics?

If yes, what would be their potential cause?

Are there any national initiatives that you would like to mention in relation to the

improvement of the system for social statistics that could be useful for other Member

States?

Are there any specificities of your national statistical system that should be especially taken

into account?

Resuming questions

We will resume the first discussion theme with a series of discussion items that could mirror

an outsider perception (e.g. European legislators) on the European social statistics:

Do you think that the currently available European social statistics satisfy the information

needs? Is any particular area that we could consider especially problematic?

Are there new information domains that require producing new or improved statistics? Is

there sufficient capacity in the European Statistical System for being responsive to users’

needs?

67

DISCUSSION THEME 2: Impact of the policy options

During the Impact Assessment exercise, Eurostat with the support of our consultants’ team

has developed five (5) alternative policy options with regard to the legal environment for

European Social Statistics. These policy options, described in detail in an interim report, act

with different strength in the following 4 dimensions:

• Integration of the different legal acts now in place;

• Promotion of innovation in methods and tools;

• Definition of the programme of statistical operations to be carried out, from a

largely fixed to a more flexible one;

• Provision for limiting costs and burden and European financing of operations.

[SHOW TABLE AS MEMORY CARD] (see supporting description for more detailed information):

Policy option

Integration/ fragmentation of social statistics legislation

Modernisation in view of statistical production (standardisation, quality reporting, survey integration, etc.)

Flexibility in view of statistical output

Outline of transmission obligation in basic act; provision on cost, burden and financing

1 (Business as usual)

Fragmented legislation

Some modernisation possible, but not in an integrated manner across different data collections

Only limited flexibility (e.g. ad-hoc modules), but no integrated data collection programme

- Mostly transmission obligations outlined in the respective base regulation

- Transmission obligations not fully integrated across different data collections

- Often general provision on cost and burden


EP and Council Regulation provides little support the modernisation of statistical production

Transmission obligations largely fixed in the base act




Transmission obligations largely fixed in EP and Council regulation





EP and Council regulation contains outline for transmission obligations and provisions to limit cost and burden to administration and respondents




EP and Council regulation contains no outline for transmission obligations and no provisions on cost and burden

68

Do you want to comment on any of the policy options?

Do you perceive any risks related to the implementation of any of the policy options?

Do you consider that a multi-annual planning (e.g. as proposed in policy option 4 and 5) of

the concerned social surveys can facilitate acceptance of any draft legislation?

DISCUSSION THEME 3: Financial implications – costs derived from the possible

implementation of an Integrated European framework for social statistics

Finally, we would like to discuss with you about the impact of some policy options on the

cost of producing statistics. While more detailed information on possible derived costs is

collected through the online questionnaire, during this interview the focus will be mostly on

the costs implications derived from the introduction of new data collection as compulsory.

The current system is regulated by a number of legal acts governing different statistical

operations such as the Labour Force Survey, the European Survey on Living Conditions and

other, while some surveys are carried out on the basis of gentlemen agreements. In

addition, there are non-legal standards set by manuals, etc.

We would like first discuss about the costs of the current system in YOUR COUNTRY.

Do you have accurate cost estimates of the costs of surveys and other statistical operations?

How do you produce cost estimates?

Which are the main sources of cost for the social statistics operations currently in place?

Where do you think that savings could be made by modernising the production process or

the organisation of work?

We would like finally to discuss about the costs in the case the system is modified.

Which costs would increase or decrease if an integrated legislation is passed...

A) In relation to the modernisation and streamlining of the production of social statistics

… due to the research needed to adapt to new methodologies (e.g. modifying

questionnaires, developing questionnaire modules)?

… due to integration of different sources (e.g. administrative registers)?

… due to renovation or modification of IT applications for data processing?

… due to organisational changes (e.g. in the fieldwork)?

… due to changes in the data collection technology?

B) In relation to requirements introduced to answer new policy needs

… due to increased precision requirements for some variables and sub-populations or higher

disaggregation of results (e.g. territorial breakdown)?

C) In relation to the introduction of data collections previously not covered by European

legislation as compulsory?

Have you estimated the difference in cost, response burden and user needs satisfaction that

an integrated regulation would imply?

69

Description of policy options:

Policy option 1: No policy change (no integration) – baseline scenario

Policy option 1 describes the current situation, i.e. the situation if there was no policy

change (baseline scenario). It should be noted that over the next years, the baseline

would be dynamic regardless of not acting on the issues at hand. Changes would occur

in either case.

Fragmented legislation for social statistics

There is no framework legislation on European social data collections, but a set of

unrelated Council/EP regulations for each data collection, namely for the Labour Force

Survey (LFS), EU Statistics on Income and Living Conditions (EU-SILC), Adult

Education Survey (AES), European Health Interview Survey (EHIS), and Survey on ICT

usage in households (ICT-HH). Two European surveys are conducted on the basis of

an informal agreement only, namely the Household Budget Survey (HBS) and the

Harmonised European Time Use Survey (HETUS). This legal fragmentation makes the

adoption of common legal provisions for several domains complicated and uncertain,

and thereby reduces the system’s capacity to adapt to both new data requirements

and to modern production environments.

Some modernisation, but not integrated across different data collections

The current regulations contain few provisions that relate to the challenges of new

data sources, or to the modern methodology and information technology that are

increasingly used in the Member States to integrate data from different sources. This

can lead to a situation where European statistical legislation becomes gradually aloof

to the technical realities of data production in the Member States. As a consequence,

modernisation and technological progress in the statistical production in the Member

States might gradually undermine EU-wide data comparability.

Limited flexibility (e.g. ad-hoc modules), but no integrated data collection programme

The current domain specific regulations fix the data collection programme of European

social data collections12. They regulate both, programme and technical implementation

for that particular domain. The indiscriminate mixture of programming (policy) and

implementation (technicalities) in the current legal setup can reduce the quality of

decision-making. The current legal setup often refers very technical matter to the

Parliament and Council, diverting their attention away from the essential task to

balance European information requirements against cost and response burden. This

results in legislative delays and rigidity that could be overcome if European statistical

legislation corresponded better to the statistical governance based on the principal of

division of responsibilities as envisioned in the EU's statistical law.

Policy option 2: Integrated, Low on modernisation, Little flexibility of data

output



important aspects of statistical data collections (e.g. definition and list of variables,

12 There is some degree of flexibility in the measurement of emerging topics through the so-called “ad hoc” modules of the LFS. The Regulation 0577/1998 of 9 March 1998 on the organisation of a labour force sample survey in the Community specifies that a further set of variables may be added to supplement the information from the core questionnaire of the EU-LFS. The ad hoc modules are set by Commission regulation by group of three years. See http://ec.europa.eu/eurostat/statistics-explained/index.php/EU_labour_force_survey_-_ad_hoc_modules.

70

definition of statistical units, data sources and data collection, methodology, reference

periods, frequency, regionalisation, precision requirements, other quality

requirements). The regulation might refer issues of technical implementation to

implementing acts.

Policy option 2 will realise the advantages of integrated legislation. It would e.g.

encourage evolutionary progress towards consistency of definitions and specifications

of variables, classifications, statistical populations and observation units. There would

be a consistent use of terminology. Policy option 2 would avoid overlaps in the

programme.

Low on modernisation

Still, option 2 would contain few provisions that relate to the challenges of new data

sources, or to the modern methodology and information technology that are

increasingly used in the Member States to integrate data from different sources. Data

collection would be assumed to continue to happen the 'classical' way. There would be

no provisions to advance on the standardisation of methodology, issues related to

innovative IT and its application, and modernised quality reporting. Statistical

concepts (e.g. for variables) might fall short of meeting the requirements of an

integrated data collection environment.

Little flexibility of data output

The regulation under policy option 2 would fix the data collection programme of

European social data collections in quite some detail, including the variables,

periodicities, regional coverage, precision requirements, transmission deadlines etc.

There would be little flexibility to target the output optimally to existing user needs

and to react to new needs for data.

Policy option 3: Integrated, High on modernisation, Little flexibility of data

output






In contrast to policy option 2, policy option 3 would contain provisions that relate to

the challenges of new data sources, or to the modern methodology and information

technology that are increasingly used in the Member States to integrate data from

different sources. As a matter of principle, the regulation would be open to

conventional data sources (e.g. surveys) and alternative data sources like

administrative registers, and lay the ground for an appropriate quality assessment and

reporting. There would be provision on sampling frames. Option 3 would encourage an

evolution towards the development of standards for statistical concepts, methodology,

IT and quality reporting that increase the system's fitness in view of a highly

innovative production environment. Provisions on feasibility studies would support

future development of processes and output.

Little flexibility of data output

However, the regulation under policy option 3 would still fix the data collection

programme of European social data collections in quite some detail, and have little

71

flexibility to target the output optimally to existing user needs and to react to new

needs for data.

Policy option 4: Integrated, High on modernisation, Multi-annual integrated

output programme with outline and burden limitation






Policy option 4 would contain provisions that relate to the challenges of new data

sources, or to the modern methodology and information technology.

Multi-annual integrated output programme with outline and burden limitation

In contrast to policy option 3, policy option 4 would provide more flexibility to target

the output optimally to existing user needs and to react to new needs for data. It

would authorise implementing measures to set up a multi-annual planning of

statistical output, and implementing acts to provide the technical specifications of the

data collections.

An increased flexibility of programming statistical output is desirable from a user's

point of view, but also from the producers' viewpoint because it can bring the

potential of innovative data collection methods and IT to reduce cost to their full

potential. However, flexibility of programming also translates into insecurity in regard

to cost and response burden for data producers.

Flexibility in programming has to be understood not only as the possibility to enlarge

the scope for data collection, but also to reduce it (for instance, as a result of the

identification of “negative priorities” in the multi-annual programme13).

As safeguard against strong increases of cost and response burden due to the

introduction in the programme of new topics for statistical measurement, policy option

4 contains provisions that limit both.

For example, where there are provisions on possible changes to the programme, it is

stated that those should not impose significant administrative burden or cost to the

Member States or on the respondents to surveys. In addition, policy option 4 would

contain provisions on financial support of the EU towards the cost of developments of

data collection, specified outputs and/or methodology. This may include EU financial

support (grants) for statistical activities such as:

• The development and/or implementation of data collections for social

statistics, including sampling frames, which may be particularly expensive

during the first four years;

• Developing new statistics or methodologies, including feasibility and pilot

studies;

• Collecting statistics with a one-off character or implemented for the first time.

13 Since 2010, the priority setting mechanism also involves the identification of “negative priorities”. Activities which were declared negative priorities have been removed or delayed because of four main reasons: lack of resources; time constraint; technical or administrative problems; revisions or rescheduling.

72

Moreover, policy option 4 would outline the output possibly required on the basis of

the regulation in terms of topics that could be covered by the different domains of

social statistics. The topics would be specified in further detail. The basic act would

further outline the range of possible periodicities, regional coverage, precision

requirements, transmission deadlines etc. The limitation of the possible scope of the

programme would limit the range of possible increases to cost and burden. The

regulation would limit the rate of change by limiting the maximum number of changes

within a given time to a specified maximum. Provision of minimum delays for an

advance planning would round up the safeguards.

Policy option 5: Integrated, High on modernisation, Multi-annual integrated

output programme with outline and burden limitation

Policy option 5 would have the same features like policy option 4, but contain no

safeguards against strong increases of cost and response burden, neither directly, not

via a limitation of the scope of the data collection programme. The European

Commission would be empowered to adopt implementing measures specifying

(including enlarging) the work programme, without providing for financial support to

Member States.

73

Annex 4: Calendar of in-depth interviews and persons interviewed

Name Organisation/ institution Function Country Date and mode of interview

Ploug, Niels Statistics Denmark Director Social Statistics Denmark Teleconference, 12 October 2015

Lenglart, Fabrice

INSEE Director of social statistics France Face to face interview, 2 October 2015

Brand, Ruzth (with staff from International Cooperation Unit)

Federal Statistical Office, Germany

Director for Health, Social Statistics, Education, Households

Germany Face to face interview, 19 October 2015

Beata Jakubczak (with staff from Social Statistics, Methodology and Coordination Depts. )

Central Statistical Office of Poland

Head of European Union Section/ International Cooperation Department

Poland Face to face interview, 23 October 2015

Pereira, Leonor (with staff from Social Statistics)

Instituto Nacional de

Estatística (Statistics

Portugal)

Director of Demographic and Social Statistics Department

Portugal Videoconference, 8 October 2015

Pisica, Silvia (with staff from Social Statistics)

National Institute of Statistics Romania

Director general, social statistics Romania Face to face interview, 14 September 2015

Argüeso, Antonio

Instituto Nacional de Estadística (INE)

Director for socio-demographic statistics Spain Preliminary interview for testing the consultation tools, 15 January 2015

Van der Laan, Paul

STATISTICS NETHERLANDS Senior policy adviser The Netherlands Face to face interview, 5 October 2015

Goodwin, Guy (with staff from Social Statistics)

Office for National Statistics Director of Social Statistics United Kingdom Teleconference, 9 October 2015

74

Annex 5: Selected results of the consultation

Question

Median Average CV

Significant improvement

Slight improvement

Stay the same

Slightly worse

Significantly worse

Nr of responses

Q.2.1 How important is the problem of a potential loss of societal relevance of the European social statistics today in view of changing users’ needs and expectations? 6,0 6,4 28% Q.2.2 Is the situation likely to improve, stay the same or worsen in the next 10 years, if Eurostat does not reform the legal framework?

0 1 15 10 2 28

Q.2.3 To what extent does the fragmented legislation, production and compilation of European social statistics reduce the overall value of European social statistics today? 5,0 5,0 57%


0 1 13 12 2 28

Q.2.5 To what extent do you consider that the slow capacity to adapt to new information needs reduce the overall value of European social statistics today?

7,0 6,7 35%


0 1 13 12 2 28

75

Question

Median Average CV


Slight improvement

Stay the same

Slightly worse

Significantly worse

Nr of responses

Q.2.8 How important is the problem of less and/or uncertain quality (e.g. lack of coherence and comparability) of the European social statistics today? 7,0 6,5 33%


0 3 15 8 2 28

Q.2.10 To what extent do you consider that the lack of standardisation is causing a decrease of the data quality (coherence and comparability) today?

6,0 6,1 38%


0 3 15 8 2 28

Q.2.12 To what extent do you consider that insufficient provisions about innovative data collection methods and IT in the current EU legal framework lead to insufficient/uncertain quality of European social statistics today? 4,0 3,8 71%


0 1 19 6 2 28

Q.2.15 To what extent do you consider that insufficient provisions about administrative data sources and multi-source integration in the current EU legal framework lead to insufficient/uncertain quality of European social statistics (in terms of coherence and comparability) today?

6,0 5,3 60%


0 3 12 8 5 28

76

Question

Median Average CV


Slight improvement

Stay the same

Slightly worse

Significantly worse

Nr of responses

Q.2.19 How important is the problem of potential inefficiency of European social statistics today (cost and response burden that could be avoided under a better setup of European statistical legislation)? 6,5 5,9 38%


0 3 12 11 2 28

Q.2.21.A To what extent do you consider that the use of innovative data collection methods and IT can increase efficiency in terms of COST to the NSI?

7,0 6,8 32%

Q.2.21.B If Eurostat does not reform the legal framework, how will this situation evolve?

0 3 13 10 1 27

Q.2.21.C To what extent do you consider that the use of innovative data collection methods and IT can increase efficiency in terms of BURDEN to the respondent?

7,0 6,4 40%

Q.2.21.D If Eurostat does not reform the legal framework, how will this situation evolve?

0 2 15 10 1 28

Q.2.22 To what extent do you consider that the cost-reducing potential of innovative data collection methods can be realised better when it is possible to decompose one long questionnaire into several shorter questionnaires?

4,0 4,2 74%

77

Question

Median Average CV


Slight improvement

Stay the same

Slightly worse

Significantly worse

Nr of responses

Q.2.23 Do you consider that the EU's current legal setup is too inflexible, i.e. that it requires questionnaires that are too long to support alternative data collection modes and instead reports about variable crossings that are probably not really required? 5,0 5,4 58%

Q.2.24.A Compared to today's situation in your country: to what extent do you consider that the use of alternative data sources and their integration can increase efficiency in terms of COST to the NSI? 6,0 6,0 44%

Q.2.24.B If Eurostat does not reform the legal framework, how will this situation evolve?

0 3 19 4 2 28

Q.2.24.C Compared to today's situation in your country: to what extent do you consider that the use of alternative data sources and their integration can increase efficiency in terms of burden to the respondent? 7,0 6,6 37%

Q.2.24.D If Eurostat does not reform the legal framework, how will this situation evolve?

2 1 17 5 2 27

78

Question

Median Average CV


Slight improvement

Stay the same

Slightly worse

Significantly worse

Nr of responses

Q.2.26 How important is the insufficient progress on methods and procedures for the European social statistics today? 7,0 6,5 34%


0 3 14 6 5 28

79

Panteia

Bredewater 26

PO BOX 7001

2701 AA Zoetermeer

The Netherlands

tel: +31 79 322 20 00

fax: +31 79 322 20 01

[email protected]

www.panteia.nl

Documents

Report on the consultation of data producersec.europa.eu/...producers-consultation-report.pdf · 1.2 Implementation of the consultation process 4 1.2.1 The questionnaire to NSIs 5