Embed Size (px)
Citation preview
Collecting Information About Every Kind of Household from a Self-Administered
Questionnaire
Sarah Grady, American Institutes for Research Jeremy Redford, American Institutes for Research
2012 AAPOR Conference
Orlando, Florida May 20,2012
2
Data
National Household Education Survey (NHES) from the National Center for Education Statistics (NCES)
Mode change from CATI to self-administered mail questionnaires
2011 Field Test
Parent and Family Involvement in Education Survey (n= 5830)
3
Overview
Challenge:
Collect information about primary parent figures in sampled child’s household
- Adapt a “mother/female guardian” and “father/ male guardian” section to collect information about two parents, regardless of gender or formal relationship (e.g. stepparent, grandparent, nonparent guardian)
- Parent 1/ Parent 2 sections created and tested using cognitive interviews. Would they work in practice?
– Split panel experiment to find out
4
Research Questions and Methods - 1
GOAL: Evaluate the “Mother/female guardian” and “Father/male guardian” sections from the Mainline form against the “Parent 1” and “Parent 2” sections from the Alternate form.
Survey Fielded
Mainline
(2009 Pilot Test
versions)
Alternate
(versions revised after
2010 cognitive
interviews)
Total
Early Childhood
Program Participation
1,820 510 2,330
Parent and Family
Involvement in
Education
4,650 1,190 5,830
Total 6,460 1,700 8,160
5
Research Questions and Methods - 2
Mainline Questionnaire
6
Research Questions and Methods - 3
Alternate Questionnaire
7
Research Questions and Methods - 3
• RQ1: Did questionnaire redesign affect the item-level
estimates related to characteristics of students’ parents?
Table 1. Frequencies resulting from split panel experiment
• RQ2: Did questionnaire redesign allow us to collect information from households with different parent types?
Table 2. Frequencies resulting from split panel experiment
• RQ3: Did questionnaire redesign affect item nonresponse?
Table 3. Item nonresponse rates from split panel experiment
8
RQ1 Results: Frequencies for Item-Level Data
We compared estimates on selected items: - Marital status / Marital and partner status - Relationship to child - Education - Employment
9
RQ1 Results: Frequencies for Item-Level Data
Mom Dad
PFI Mainline
PFI Alternate
PFI Mainline
PFI Alternate
Marital Status Married 73 75 86 88
In a registered domestic partnership
n/a
-
n/a
-
Living with a partner 5 4 4 5 Separated 3 4 1 1
Divorced 10 8 5 4
Widowed 1 1 - - Never married 8 7 3 2
10
RQ2 Results: Collecting Information About
Different Types of Parent Figures
• 2 parent households
Mother and Father
Same-sex parents
Unknown two-parent
• Not 2 parent hhs
Grandparent
Single parent
Other
Looked at reported parents and analyzed the percentage of students with:
Compared Mainline, Alternate; 2007 CATI estimates; 2010 ACS
11
RQ2 Results: Collecting Information About
Different Types of Parent Figures
2011 NHES Field Test
Mother/Father (Mainline)
Parent 1/ Parent 2(Alternate)
2 Parent households 72 71 Mother and father households 72 68
Same-sex parent households - 2 Unknown gender two-parent households
n/a -
Other parent combinations 28 29
Grandparent-headed households 2 3
Single-parent households 23 24 All other households 3 2
12
RQ3 Results: Item Nonresponse Rates
2011 NHES Field Test
Mother/Father
(Mainline)
Parent 1/ Parent
2(Alternate)
Marital status 1 1
Relationship to child 4 1
Education 5 2
Employment 4 1
13
Discussion
• Questionnaire redesign from Mother/ Female guardian and Father/ Male guardian sections to Parent 1 and Parent 2 sections resulted in:
Similar substantive estimates
Higher prevalence of same-sex parent households and grandparent-only households; lower prevalence of mother and father households
Lower average item nonresponse in the parent section
14
Next Steps
Areas for future research
- Work on marital status item
- Look at Early Childhood study data from Field Test
- 2012 NHES results- first SAQ ABS full-scale administration
- Compare 2012 full-scale to 2007 CATI
- More investigation of ACS and additional extant data
Additional Information
17
RQ2 Results: RDD and ACS
2007 NHES
PFI
2011 NHES Field Test 2010 ACS
Mother/Father (Mainline)
Parent 1/ Parent
2(Alternate)
2 Parent households 73 72 71 67
Mother and father households
73 72 68 67
Same-sex parent households
- - 2 -
Unknown gender two-parent households
n/a n/a - n/a
Other parent combinations
27 28 29 33
Grandparent-headed households
3 2 3 1
Single-parent households
23 23 24 31
All other households
1 3 2 -
18
Instruments- Mainline questions used in analysis
53. Is this person the child’s…
Birth mother,
Adoptive mother,
Stepmother,
Foster mother,
Grandmother, or
Other female guardian?
55. What is the current marital status of this child’s
mother or female guardian?
Mark [X] ONE only.
Married
Living with a partner
Separated
Divorced
Widowed
Never married
65. What is the highest grade or level of school that she
completed?
Mark [X] ONE only.
8th grade or less
High school, but no diploma
High school diploma or equivalent (GED)
Vocational diploma after high school
Some college, but no degree
Associate’s degree (AA, AS)
Bachelor’s degree (BA, BS)
Some graduate or professional education, but no degree
Master’s degree (MA, MS)
Doctorate degree (PhD, EdD)
Professional degree beyond bachelor’s degree (MD, DDS, JD,
LLB)
67. Which of the following best describes her
employment status?
Mark [X] ONE only.
Employed for pay or income
Self-employed
Unemployed or
out of work GO TO question 69.
Stay a home
mother GO TO question 70.
Retired GO TO question 70.
Disabled or GO TO question 70.
unable to work
19
Instruments- Alternate questions
57. Is this parent or guardian the child’s…
Biological parent
Adoptive parent
Step parent
Foster parent
Grandparent
Other guardian
59. What is the current marital or partner status of this
parent or guardian?
Mark ONE only.
Married
In a registered domestic partnership or civil union
Living with a partner
Separated
Divorced
Widowed
Never married
69. What is the highest grade or level of school
that this parent or guardian completed?
Mark ONE only.
8th grade or less
High school, but no diploma
High school diploma or equivalent (GED)
Vocational diploma after high school
Some college, but no degree
Associate’s degree (AA, AS)
Bachelor’s degree (BA, BS)
Some graduate or professional education but no degree
Master’s degree (MA, MS)
Doctorate degree (PhD, EdD)
Professional degree beyond Bachelor’s degree (MD, DDS,
JD, LLB)
71. Which of the following best describes this person’s
employment status?
Mark ONE only.
Employed for pay or income
Self-employed
Unemployed or out of work
Stay at home parent
Retired
Disabled or
unable to work
