Slide 1

When creating survey instruments, there are six main things to consider.

Slide 2

Creating Survey Instruments 1. Substance What do you really want to know? What questions do you need to ask to find these things out?

Slide 3

Creating Survey Instruments 2. Number How many questions should you ask? How many is too many? Respondent fatigue

Slide 4

Creating Survey Instruments 3. Organization/ Order: Where to put the stuff on the survey Standard order is: 1. Intro 2. Warm-up 3. Substantive questions 4. Demographic or sensitive information

Slide 5

Creating Survey Instruments 4. Form: Types of questions Open-ended questions Close-ended questions Mix of both Pros & Cons

Slide 6

Creating Survey Instruments 5. Layout: Look of the survey instrument Is it appealing? Amount of time? Coding

Slide 7

Creating Survey Instruments 6. Wording: How you ask questions and phrase things Leading questions Double-barreled questions Clarity in Wording Social desirability element

Slide 8

Recap of Survey Instruments 1. Substance 2. Number 3. Organization/ Order 4. Form 5. Layout 6. Wording Survey Instrument

Slide 9

Slide 10

Slide 11

Slide 12

Slide 13

Pre-testing Real or True Pre-test Pseudo Pre-test

Slide 14

Population What is the population? What is a case? Cases Population

Slide 15

Samples Sample: Part of a population Case Census v. sample Simple random sample Every member of population equal chance of being picked

Slide 16

Samples 2 main considerations: How to pick cases How many cases These help ensure reliability of the results Representative sample Confidence interval (CI)- wiggle room At most want to see +/- 3% Confidence level (CL)- certainty in CI Usually 95% is acceptable

Slide 17

Evaluating Data Who to trust Major polling organizations Gallop Nielsen Big media outlets Wall Street Journal NY Times Washington Post 3 major TV networks (CBS, NBC, ABC)

Slide 18

Evaluating Data Who not to trust Unless CI is given, dont trust without looking it up General media outlets Other TV networks, newsprint surveys, radio stations, Internet sites

Slide 19

Representative Sample The National Housing Quarterly Survey done in 2011 found 64% of Americans think owning their own home is a safe investment. In 2010, 70% thought it was a safe investment, while in 2003, 83% it was. *CI +/-3% Is there a significant change from 2010 to 2011? How about from 2003 to 2011?

Slide 20

Representative Sample YearAs many asAs few as 201167%61% 201073%67% 200386%80% *95% of time= 95/100 people respond same way

Slide 21

Representative Sample We want to know how racial minorities in the U.S. view gay marriage. A poll conducted to assess Americans attitudes toward gay marriage has an insufficient sample size to evaluate individual racial minority groups reliably. However, in surveying Americans, support for gay marriage is essentially identical among whites (53%) and nonwhites (54%). CI +/-3% -Why cant the polling company reliably report the racial breakdown?

Slide 22

Representative Sample Would need a large enough sample size for each racial group Thus, CI not reliable for the racial breakdown Poll has sufficient sample size to represent Americans in general Cant guarantee 9.5 /10 Asians, Blacks, Latinos, would respond the same way as the nonwhite respondents Latinos might actually be at 20%, Blacks at 60%

Slide 23

Measures Of Central Tendency Mean Median Mode

Slide 24

Measures of Central Tendency Mean: Average Formula: Add all, divide by total number (N) What is the mean of the following college students ages? 18, 18, 19, 21, 22, 24, 52 Using formula: 174/7 =24.85

Slide 25

Measures of Central Tendency Median: Middle Formula: Odd numbermid point; Even number number of points +1 divided by 2 What is the median age for these college students? 18, 18, 19, 21, 22, 24, 52

Slide 26

Measures of Central Tendency Mode: Frequency Formula: Identifying the modal points What is the mode age for these college students? 18, 18, 19, 21, 22, 24, 52 Using formula: That which occurs most

Slide 27

Measures of Central Tendency If these are the ages of the college students, why does matter whether the mean, median, or mode is reported? Ages: 18, 18, 19, 21, 22, 24, 52 Mean-24.58 Median-21 Mode-18

Slide 28

Why Median? The Economist noted that The National Association of Realtors reckons that the median house price fell to a near-nine-year low. Why did the NAR report the median house price? Think in terms of the student ages: 18, 18, 19, 21, 22, 24, 52 Mean-24.58 Median-21 Mode-18

Slide 29

Why Median? Mean=average Median=middle point Reporting mean includes outliers Reporting median controls for outliers

Slide 30

Measures Of Dispersion Nominal: No numerical value Names Examples? Ordinal: Numerical value Rank ordered Bracketed Examples? Interval: Numerical value Know distance between actual numbers Examples?

Slide 31

Nominal Names of places Louisville and Covington are cities in KY. Columbus and Akron are cities in OH. Assigned numbers, but they are meaningless: Akron (1) Columbus (2) Covington (3) Louisville (4)

Slide 32

Ordinal What do we know from this information? Louisville is the biggest city in KY. Covington is the 5th biggest city in KY. Columbus is the biggest city in OH. Akron is the 5th biggest city in OH.

Slide 33

Interval Louisvilles population is 597,337 Covingtons population is 40,640 Columbuss population is 787,033 Akrons population is 199,110 What do we know from this information?

Slide 34

Data TypeInformationMeasurement Nominal Louisville, Columbus, Akron, etc. Mode # times name appears Ordinal Louisville: Largest in KY Columbus Largest in OH Mode, median Also, city rank in exclusive category Interval Louisville: Population of 597,337 Columbus: Population of 787,033 Mode, median, mean Can also figure out median between diff. categories Also, actual average of populations What is the average population size of the top 5 cities, top 5 KY cities, top 5 OH cities, etc.

Slide 35

Recap of Measures Data Type Appropriate Measure of CT Nominal Mode OrdinalMode, median IntervalMode, median, mean

Slide 36

Issues with Reporting Data Is the confidence interval given? If so, you can assume a CL was also used Are figures given consistent? Mean or median? Are examples anomalous or representative? False cause and hasty generalization fallacies

Slide 37

Which Methodology? Qualitative or Quantitative?

Slide 38

Which Methodology? You want to know how people will vote in the upcoming election.

Slide 39

Which Methodology? You want to determine which US foreign policies in the last decade have had the biggest impact on trade.

Slide 40

Which Methodology? You want to know whether adding new sewer lines or repaving the roads in your district is a better use of city money.

Slide 41

Which Methodology? You want to see what impact putting wells in developing countries has had on the number of people stricken with diarrhea.