Upload
gabriella-gwenda-kennedy
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
The Future of Statistical Data Collection?Challenges and Opportunities
Johan Erikson (Statistics Sweden)
Gustav Haraldsen (Statistics Norway)
Ger Snijkers (Statistics Netherlands)
Seminar on New Frontiers for Statistical Data Collection31 October – 2 November 2012, Geneva, Switzerland
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 2
Outline
1. Fundamental challenges
2. How do we meet these challenges?
3. Are we prepared? • Traditional survey perspectives:
- Management- Methodology
• Beyond the traditional survey framework
4. Conclusions
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 3
1. Fundamental challenges
Official Statistics in a changing world!
Three main challenges:
1. A shift in the balance of power between survey organisations and respondents
2. New competitors may make Official Statistics redundant
3. Globalisation of the economy
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 4
A shift in the balance of power
• Dillman et al. (2009):
“Surveys are now respondent driven, rather than driven primarily by the needs of
survey organizations.”
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 5
A shift in the balance of power• Traditionally – data collectors in control - Responding to survey was seen as a duty – not any more - The change can be seen clearly by studying the attitude of
respondents in different age groups
• Business organisations focus more on production and competiveness
• The means of communicating are changing• Respondents are getting harder to reach, and can
control who they want to be reached by
Conclusion: We need to adapt to respondents more and more in orderto get their cooperation using all kinds of comm. means.
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 6
New competitors
• Groves (2011) - “Organic vs. designed data”:
“We’re entering a world where data will be the cheapest commodity around, simply because
society has created systems that automatically track transactions of all sorts.”
• E.g. transactions on social media:3.2 million new entries/day, in the Netherlands
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 7
New competitors
• Data are everywhere: “Big Data” Conferences• New competitors use these data to produce new
statistics/information in real time:- E.g. Google’s stock prices index (GOOG)
• This challenges the way traditional (NSI) statistics are produced:- founded in survey methodology, with a focus on quality
• The new information maybe directed at different users
Conclusion: • What is our position in the information market?• Can we produce timely, relevant and competitive stats?
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 8
Globalisation
• UNECE (2011) - The Impact on globalization on National Accounts:
“The increasingly global nature of economic transactions and arrangements presents a
challenge to the application of national accounts concepts and the use of data collection and compilation systems for
measuring developments in the domestic economy.”
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 9
Globalisation
• Traditional indicators that show a society’s development may no longer be valid
• The concepts and units (needed to compile these indicators) are becoming irrelevant to globalised businesses:
- If the concepts are irrelevant to providers, data will be hard to get - Offices in any part of the world may have to be contacted
Conclusion: • The definitions, measurement and compilation of
traditional indicators needs to be reviewed• The concepts and units need to be revised, • The data collection methods and procedures need to be
adapted in international perspective (relates 1st challenge)
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 10
Data Collection Strategy - 3 steps:1. Re-use of available data
• Data sharing & data warehousing
2. Use of new registers and other secondary sources • Traditional government-based registers • Information from private businesses
• “Organic/Big Data” sources
3. Primary data collection:1. EDI technologies, like XBRL
2. Web surveys 3. traditional modes: paper, CATI, CAPI
• Using new communication technologies • Reciprocity: report back to respondents
Multi-source designs
Mixed-mode designs
2. What is done? The answer to these challenges so far
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 11
3. Is this sufficient? Are we prepared re: challenges?
• To answer this question we looked at the internal processes in our Statistical Offices
• What are we actually doing …?• What is our short-term focus?
• The Quality Diamond: (Haraldsen; in: Snijkers, Haraldsen, Jones & Willimack, 2013, Wiley)
Integrates four quality perspectives on statistics:
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 12
SpecificationsCoverageSamplingNonresponseMeasurementProcessing
RelevanceAccuracyTimelinessPunctualityAccessibilityClarityComparabilityCoherence
(Eurostat Quality Dimensions, 2011)
Question order effectsMode effectsCross-cultural effects
CostsTechnologyEthicsTimelinessPRB
The Quality Diamond• Management perspective• Methodology perspective
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 13
Non-response
6 Managing customer demands and expectations
Mixed-mode effectsData Collection Strategy effects
1 Costs2 Timeliness3 Flexibility of process planning4 Innovation of process5 Response Burden7 Planning staff8 Systems and tools9 Culture
The Quality Diamond:The Management Perspective
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 14
Non-responseMeasurementCoverageSampling
Accuracy
Mixed-mode effectsQuestionnaire effectsData Collection Strategy effects
Response BurdenInnovation of data collection methods
The Quality Diamond:The Methodology Perspective
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 15
Differences in focus:
• Management (mostly) interested in becoming better (more efficient) within traditional survey framework.
Short-term challenges, while running the surveys: - Standardisation and integration of processes - Innovating the current processes within the Data
Collection Strategy, like planning mixed-mode designs
• Methodology driven by quality considerations: - Reducing survey design effects and Total Survey Error - Survey process constraints not much considered
• The two perspectives hardly overlap
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 16
Beyond the traditional survey framework• Short-term focus – based on traditional
framework
We need to go beyond that framework: Considering challenge 1: balance of power
• The end of probability sampling? - High non-response levels and low representativeness:
responsive designs, web panels, post-survey adjustments - Use of registers: no sampling involved; non-sampling errors?
Towards multi-source/mixed-mode designs• The end of standardised questionnaires/data collection?
- From collecting data to collecting {data & metadata} usingdata capture methods (like XBRL)
- From standardised interviewing to conversational interviewingand data mining, using new technologies (smart phones, Skype)
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 17
Beyond the traditional survey framework
• Official data requirements and concepts – based on stove-pipe approach: traditional indicators, separate surveys
We need to go beyond that:• The end of traditional indicators to measure societal
developments?Considering challenge 3: Globalisation
- What information do we need? - How do we measure that information?
• The end of official statistics? Considering challenge 2: New competitors - What will be our position in the information market?
New Frontiers for Stastictal Data Collection, 2 November 2012, Geneva 18
4. So, are we prepared?
• Standardisation and integration of data collection processes are a step in the right direction… but not enough
• Data Collection Strategies towards multi-source/mixed-mode designs are a step in the right direction… but not enough
Managers and methodologists need to collaborate, having common short-term and long-term focuses: a communication perspective!
We need to go beyond the traditional survey framework… and act now! The future is already here!