Effective Surveys

  • View
    2.001

  • Download
    0

Embed Size (px)

DESCRIPTION

Slide show presented at May, 2007 in-service for UNH CE educators

Transcript

  • 1. Designing Effective Questionnaires
  • 2. Survey/Questionnaire Data
    • Use type appropriate to population, topic, and budget.
      • In-person
      • Telephone
      • E-mail / web
      • Self Administered / Mail
  • 3. Survey/Questionnaire Data
    • Outputs (Activities)
      • Teaching and facilitation
      • Participant background
      • Location
      • Time and date of program
      • Marketing
    • Learning outcomes and intended behavior change
    • Future programming (needs assessment)
  • 4. Donts!
    • Dont ask a question if you already know the answer!
    • Dont ask a question if youre not prepared to do anything about the answer!
    • Dont ask nice to know questions.
    • Ask questions only if you know what you will do with the answers.
  • 5. What do you really want to know?
    • Take three minutes and write down four or five things you would really like to know from a questionnaire you are designing.
  • 6. Asking good questions
    • Questions should be clear and precise
    • Make it easy to complete
    • Know your audience
    • Protect anonymity of respondents
    • Proof read and edit
    • Pilot test!
  • 7. Caution End-of-Session Questionnaires
    • Self-reports
    • One-point-in-time
    • Although they may measure perceptions and intentions, they wont measure behavior change
  • 8. Other considerations
    • Data analysis
    • How will you use the results?
    • Who needs the results?
    • Timing of the survey/questionnaire
  • 9. Designing Questionnaires
    • Keep it short and on point.
    • Use questions that have been used and tested by others if possible
    • Be sure questions have face validity.
    • Include important demographic questions.
    • Make sure ALL questions are germane to the evaluation.
    • Pre-test all questionnaires on same population.
  • 10. Types of Question/Format
    • Open-ended
    • Yes or no
    • One best answer
    • Select all that apply
    • Rating or Ranking
    • Retrospective (post-then-pre)
  • 11. Open-ended
    • Free text
    • Provide richer detail
    • Must be clear in what you are looking for
    • More difficult to get people to fill out sometimes
    • Takes more time to analyze
  • 12. Yes or No
    • Simple
    • Use with information that has no middle ground
      • Do you own your own home?
      • Are you a U.S. citizen?
  • 13. One-best answer
    • Provide only one of many possible answers
    • Of the following, which topic would you MOST like to receive information about?
    • a. Pasture Management
    • b. Farm Estate Transfer
    • c. Dairy Cattle Nutrition
  • 14. Select all that apply
    • When you want to have more than one answer to a question
    • Which appliances do you own? (check all that apply)
          • __ Blender __ Waffle Iron
          • __ Toaster __ Coffee Maker
          • __ Mixer __ Food Processor
  • 15. Rating Questions
    • To what extent do you think you can apply the information learned here today?
    • Not at all A Great Deal
    • 1 2 3 4 5 6
  • 16. Ranking Questions
    • Of the topics covered today, please rank their usefulness in order, with 1 being most useful and 4 being least useful.
    • ___ Setting limits for teenagers
    • ___ Challenging behaviors
    • ___ Consequences
    • ___ Listening
  • 17. Scavenger Hunt
    • Using the UW Extension Publication Collecting Evaluation Data: End-of-Session Questionnaires
  • 18.
    • An example of a grid with dichotomous pairs of words to rate an instructor
    • An example scale asking how the level of the workshop fit participant experience level
    • An example of a retrospective survey on public policy education
    • Sample category scales
    Find the following:
  • 19. More about questions
    • Avoid double barreled questions
      • Did you find the workshop helpful and interesting?
    • Dont use leading questions.
      • Do you agree that funding for Extension should increase? yes or no
    • Dont use abbreviations and jargon
  • 20. Questions continued
    • Use ranges for choices instead of precise numbers. Put choices in logical order.
    • Be very clear
      • How many times did your club meet last year?
    • Avoid questions that are too demanding.
      • Rank the following 15 items in order of importance to you.
  • 21. Yet more on questions
    • Avoid making assumptions.
      • How many children do you have?
    • Avoid bias.
      • How would you rate the workshop? a. excellent; b. good; c. satisfactory
    • Make response categories clear.
      • How many acres do you own? 0-10, 10-20, 20-50, 50-100
  • 22. Question Order
    • Question order does influence results.
    • Begin a questionnaire with something that isnt too difficult to answer, but will be of interest to the participant.
    • Put easiest (demographic information) at the end this is where people sometimes lose interest and if the questions at the end are less demanding, they are more likely to finish.
  • 23. Pre/post Tests
    • Use an identical knowledge test, before the educational program and then after.
    • Might ask specific questions about a subject, like, How many servings of fruits are recommended for an adult every day by the new food pyramid?
    • Might ask them to rate their knowledge of something, Please rank your understanding of the new food pyramid recommendations.
  • 24. Pre/post test considerations
    • Need to p