Survey Research

Brian J. Piper, Ph.D., M.S.

Goals

• History• Sampling• Variations• Creating Good (or Bad) Surveys

History

• Charles Darwin, in preparing Expression of Emotions in Man & Animals (1872) sends out survey:– “Is contempt expressed by a slight protrusion of

the lips and by turning up the nose?”

1809-1882

Grief

Sir Francis Galton

• Pioneer in behavioral genetics (eugenics)• Interested in nature & nurture• Leading Question:

– “How far do your scientific tastes appear innate?”

Research Question Determines Sampling Strategy

• Convenience Sample: participants obtained based on convenience of investigator

• Representative Sample: participants carefully chosen so that they represent some larger population

Areas that use convenient samples

• Social Psychology (Experimental)• Jury Duty• Brain Imaging

– fMRI: claustrophobia, metal artifacts– PET: claustrophobia, radioactivity

• Taboo/Sensitive (Illegal) Behaviors– Sex: incest, masturbation, homosexuality– Suicide/Mental Health– Drug Use

Areas that (aim to) use representative samples

• Political Polling• Neuropsychological Test Development

(Norms, e.g. BRIEF)

Behavioral Rating Inventory of Executive Function (BRIEF)

• Please rate whether the following behaviors are never, sometimes, or often a problem (1-3 points):– Emotional Control (EC) : overreacts to small problems– Inhibit (INH): interrupts others– Shift (SHI): becomes upset with new situations– Working Memory (WM): when given three things to do, remembers only the first or last– Initiate (INI): has trouble coming up with ideas for what to do in play time– Plan/Organize (PO): gets caught up in details and misses the big picture– Organization of Materials (OM): cannot find things in room– Monitor (MON): does not check work for mistakes

– Behavioral Regulation Index + Metacognition Index = Global Executive Composite

Norms: Mean= 50.0, SD = 10Gioia GA, Isquith PK, Guy SC, et al. (2000). Behavioral Rating Inventory of Executive Function: Professional Manual. Psychological Assessment Resources: Lutz.

*** p < .0005, ** p < .005, * p < .05 .

“Clinically Significant” Behavioral Problems in Exposed Children

Piper et al. (2011) Pharmacology, Biochemistry, & Behavior.

Types of Sampling

• Random sampling: every member of a population has an equal probability of being in the sample

• Stratified sampling: sample is made to match population on characteristic(s)

• Cluster sampling: obtain subset of representative subset (cluster)

Pros & Cons of Surveys: Monitoring the Future

• National survey, conducted annually by NIDA of secondary school (8th, 10th, or 12th),

• Missing data (drop-outs = 15%, not attending) = under-estimation

• Concern about labels (inadvertent marketing)• Self-report

– Knowledge (e.g. steroids, diet pills, club drugs)– Memory (e.g. How many drinks?)– Taboo

http://www.monitoringthefuture.org/

Demographics

• Compares use by:– Year (1975 – present)– Sex– Ethnicity– Parental education– Region– College plans

?

Year & Grade

Illicit = Scheduled drugs including marijuana, heroin, cocaine, methamphetamine, LSDNot Illicit: EtOH, cigarettes, inhalants, Ritalin for medical purposes

Monitoring the Future

• Advantage– Cost effective– Detailed– Information not otherwise available

• Disadvantage– Representative?– Self-report (ideally verified)

Interviews

• Alfred Kinsey (1894-1956) founded the field of sexology with extensive interviews with males (1948, N=5,300!) and females (N=6,000!, 1953)

• Masturbation, hetero/homosexuality• Studied groups• Over-represented prisoners, under- represented African-Americans

2 min clip: http://www.youtube.com/watch?v=KkGzoBSG1cY

Phone Surveys

• Pros: flexible, cheaper than face to face• Cons: caller-ID (over-representation of elderly?)

Electronic Surveys

• Pros: very inexpensive• Cons:

– Brevity– Confidentiality– Uncontrolled environment

Cronbach’s Alpha

• A measure of the internal consistency of items on a survey (0 to 1.00)

1916-2001

Paper versus Online

Piper & Corbett (2012). Nicotine & Tobacco Research, 14, 191-199.

Don’t know

• Pro: provides respondent an out• Con: missing data

Conclusion

• Survey construction: use others (avoid leading questions & double-barreling) & pilot– Reliability– Validity

• Sampling: plan

Options