SURFing with Statistics New Zealand
Nathaniel Pihama and Deborah Brunning
Statistics New Zealand
Statistics Teachers' Day30 November 2007
What you will see today
• SURF for Schools
• The Statistics New Zealand website
• Table Builder
..and some ideas on how to use them!
The First SURF: a Synthetic Unit Record File for Schools
Overview:
• Confidentiality – Big Picture
• SURF???– What is it?– How and why did we make a SURF?
• What teachers and students can do with the SURF
What is a Unit Record File?
Other names• Data set• Unit Record Data set• Microdata
Example: A Dataset:Name Gender No.Sibs Age;yrsAmy F 1 14Ben M 2 13Cher F 1 15
Confidentiality
Safe
UnsafeUseless Useful
The pocket
Raw dataset
Non release
Confidentialised Unit Record Files
Confidentiality methods include:Categorical Data• Global recoding• Local recoding
• Numerical Data • top/bottom coding, • capping,• rounding,
Before Top and Bottom Coding
0
2
4
6
8
0 10 20 30 40 50 60 70 80 90
Salary: k$/yr
FreqAfter Top and Bottom Coding
0
2
4
6
8
0 10 20 30 40 50 60 70 80 90
Salary: k$/yr
Freq
What is the SURF?
• Data from 200 synthetic respondents.
• Target population is those aged 15-45 in paid employment.
• 7 variables
personid gender
qualification age hours income marital ethnicity
1 female school 15 4 87 never european
2 female vocational 40 42 596 married european
3 male none 38 40 497 married maori
4 female vocational 34 8 299 never european
5 female school 45 16 301 married european
6 male degree 45 50 1614 married european
7 female none 36 12 201 other european
What does the SURF look like?
The first 7 of 200 complete unit records
SURF- the variables
How to start SURFing?• The gender gap (Level 3 and 4)
» Do more females have higher qualifications than males?
» Is this different from how it was in the past?
• Am I average? (Level 4) » What defines the average person?
• Under pressure? (Level 5)» Are people who have never been married different
from married people?
• Equal Pay! (Level 6) » Are males and females paid equally?
• Money for nothing (Level 7) » Investigating hours worked by employees in a
company• Should I do a degree? (Level 8)
» Investigation into whether getting a degree helps improve earning power
http://www.stats.govt.nz/schools-corner
• A large company is concerned that it has too many employees who do not work a 40-hour week.
• You have been hired to investigate the working patterns of the employees.
Task -Money for Nothing
Further Analysis- Hours by Gender
SURF CURF
Related variables – Hours by Marital Status
SURF CURF
Related variables – Hours by Age GroupSURF CURF
How ‘school friendly’ is SURF????
• SURF Excel spreadsheet
• Records are in random order – First 30 records could be used for manual
data analysis
• Use ExcelHow???????
Add Age_10 variable
Add random numbers (then paste special as values)
An example of how to take a stratified sample
Use filter – copy and paste records
Sort on random numbers
Filter function
Pivot Tables
Pivot Tables
Box plots !!!!
QUARTILE functionSORT by Marital
Regression and Residuals• Trend line in a scatter plot
– Good for quick visual check
– Provides equation & R-sq
– But no residuals
• Plot the data (XY scatter)
(tidy plot up)
• Add TrendlineChart menu > Add trendline
Options tab
y = 17.058x + 0.3487
R2 = 0.6323
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 20 40 60 80
Hours
Inco
me
Regression and Residuals
• Using Excel functions
– SLOPE(), INTERCEPT(), RSQ()
Copy cell ref into formula bar
Regression and Residuals- Easy to create predicted values and residuals
(can copy formula and use $)
Regression and Residuals- Plot the residuals
-600
-400
-200
0
200
400
600
800
1000
1200
Hours
Res
idu
als
The Statistics New Zealand Website
Stats NZ Products and Services• Schools Corner
– Full of resources based on the curriculum.
• Information releases – Hot Off the Press – Full of highlights, commentary, technical notes and tables!
• New Zealand in Profile– Quick stats of New Zealand for 2007
• Analytical reports– Contain in depth analysis, background and technical
information
• Table Builder– Customisable tables of released survey data
Stats NZ Products and Services• Schools Corner
– Full of resources based on the curriculum.
• Information releases – Hot Off the Press – Full of highlights, commentary, technical notes and tables!
• New Zealand in Profile– Quick stats of New Zealand for 2007
• Analytical reports– Contain in depth analysis, background and technical
information
• Table Builder– Customisable tables of released survey data
Stats NZ Products and Services• Schools Corner
– Full of resources based on the curriculum.
• Information releases – Hot Off the Press – Full of highlights, commentary, technical notes and tables!
• New Zealand in Profile– Quick stats of New Zealand for 2007
• Analytical reports– Contain in depth analysis, background and technical
information
• Table Builder– Customisable tables of released survey data
Stats NZ Products and Services• Schools Corner
– Full of resources based on the curriculum.
• Information releases – Hot Off the Press – Full of highlights, commentary, technical notes and tables!
• New Zealand in Profile– Quick stats of New Zealand for 2007
• Analytical reports– Contain in depth analysis, background and technical
information
• Table Builder– Customisable tables of released survey data
Battle for the ‘greener suburb’:
an example of using case data from Table Builder
• Problem – the statement of the research questions
• Plan – planning the procedures used to carry out the study
• Data – the data collection process
• Analysis – the summaries and analyses of the data to answer the questions posed
• Conclusion – the conclusions about what has been learned.
The statistical investigation cycle:(Wild and Pfannkuch, 1999)
Battle for the ‘greener suburb’:an example of using case data
• Comparing the ‘traveling to work’ habits of area units within Auckland.
• Which area has the ‘greener’ workers?– Walking / Running / Cycling
– Public transport
– Carpooling?
– Working at home?
Battle for the ‘greener suburb’:where to find the data
• We want a data source that contains information about modes of travel to work by area units.
• Luckily, we have the 2006 Census of Population and Dwellings on Table Builder!
• So this is some of what Statistics New Zealand has to offer for teachers.
• Do you know about: – Statzing?– CensusAtSchool?– Statistics and Research?