Upload
jeremy-faull
View
230
Download
4
Tags:
Embed Size (px)
Citation preview
Summer Camp
Designing engaging questionnaires to deliver high response rates
Presented by
Steve Bax
Managing Director
Bax Interaction Limited
Summer Camp
Overview
This webinar will explore the key principles behind effective questionnaire design. It will include current thinking on how best to engage potential respondents and thereby improve completion rates.
Summer Camp
Content
1. Introduction
2. Defining the research process
3. Questionnaire design and piloting
4. Engaging the respondents
5. Research design checklist
6. Questions
Summer Camp
Research phases
Analysis, reporting and
reviewphaseR
eso
urc
es r
equ
ired
DefiningPhase
Piloting and DesignPhase
Engagement including
pre-qualification
Reminders
Start Time Finish
Summer Camp
Why use questionnaires?
Primarily to acquire quantitative information from respondents
Also provide qualitative insights
Reasons include:-
Identifying, evaluating and sizing existing and new markets
Testing reactions to new products and services
Measuring performance
Satisfaction surveys
And more
Summer Camp
Defining
Summer Camp
Defining
Purpose
Research objectives The ‘shopping list’
Data to be collected Research vs.. volunteered
Target audience(s)
Delivery mechanisms
Previous experience
Summer Camp
Research objectivesThe objectives are key and must generate the topics for investigation.
The wording can imply a certain method
E.g. to measure = quantitative
E.g. to explore = qualitative
Consider other terms: goal, aim, purpose and hypothesis
Summer Camp
Stop and think
What is my ‘shopping list’?
What data do we need at the end of the process?
Is it pure marketing research or volunteered data?
Who do we need to gather data from?
What are the implications on method and design?
What have we learnt from previous or others’ research successes or failures?
Summer Camp
Selection of delivery mechanisms
Summer Camp
Online surveys
Advantages
Low cost
Fast
International coverage
Creative opportunities
Effective for qualitative and quantitative data collection
Disadvantages
Declining response rates
Incomplete cover
Biased responses possible
Summer Camp
Written and postal surveys
Advantages
Geographical reach
Control over the process
Quantity of data that can be gathered
Respondent perceptions
Disadvantages
Cost due to postage rates and printing vs. online
Slow to administer
Response rates
Self-selecting samples - bias
Summer Camp
Telephone based questionnaires
Advantages
Low cost vs. face to face
Fast
International coverage
Control over process
Understanding checking possible
Disadvantages
Refusal rates
Sugging and frugging perceptions
Limited duration
Summer Camp
Multi mode approachesSnap Surveys and Silver Dialogue findings with National Trust visitors
Summer Camp
MMR ‘DiaLogs’ APP
Summer Camp
Design and piloting
Summer Camp
Design
Fundamentals
Relevance
Usability
Layout and appearance
Question and response construction
Methodology
Qualitative vs.. quantitative building blocks
Multi-mode approaches
Sampling
Open and closed question mix
Piloting
Summer Camp
Key design principles
1. .. the rigour, integrity and quality of data. Neither interviewers nor respondents should be confused or bored by any questionnaire.
2. The guiding rule can be expressed as ‘doing justice to how people would normally talk about things’.
Source: MRS – Questionnaire design guidelines
Summer Camp
‘Funnel’ sequence
As a rule – progress from the general to the specific
1. Gain understanding of broad context of opinions held
2. THEN the nature and strength of opinion in a given area
3. Step by step to the detail underpinning these
ALWAYS PILOT TEST
Summer Camp
Questionnaire construction
Target audience
Develop question topics
Select question and response formats
Select wording
Determine sequence
Design layout and appearance
Clear instructions
MRS/DPA considerations
Summer Camp
Layout and appearance
Introduction
Spacing
Numbered headings and sections
Numbered questions
Length
Quality of production – on and offline
Use of sequencing, routing, piping and a variety of response mechanisms
Clear instructions
Use of colours to indicate changes in areas being questioned
Font and text size
Summer Camp
Types of question
Behavioural - market size, market share, usage rate, awareness
Attitudinal - image and attitude surveys, brand mapping, help build market share
Classification - all surveys
Summer Camp
Behavioural questions
Have you ever ……?
Who do you know …………?
How many ………….?
When did you last ………?
Do you have ………?
Who does …………?
Summer Camp
Attitudinal questions
What do you think of ……?
Why do you …………?
Do you agree or disagree ………?
How do you rate ………?
Which is best (or worst) for ………?
Summer Camp
Question / response formats
Closed questions
Open-ended questions
Scales
Variety is key
Summer Camp
Closed questions
Dichotomous
Two possible answers
Multiple choice
Three or more answers! Collectively exhaustive Mutually exclusive
Pros and cons
Summer Camp
Open-ended questions
Advantages
Full answer
Exploratory
Disadvantages
Misinterpretation
Speed
Coding / analysis
Completely unstructured - interviewer or respondent writes answer
Three key open-ended questions are useful:
1. What is good? 2. What is bad? 3. What improvements could be made?
Summer Camp
Classification questions Examples
Age
Gender
Household status
Marital status
Social class
Education level
Industrial occupation
Number of employees
Location
Neighbourhood
Summer Camp
Use of scales
Forced vs.. non forced
Forced Doesn’t allow ‘middle ground’
Non forced Allows neutral views e.g. ‘neither expensive or inexpensive’
Balanced vs.. unbalanced
Most balanced e.g. equal number + as –
Unbalanced used where ‘end piling’ is expected e.g. Car safety features
Summer Camp
Question wording
Ambiguous questions
Double barrelled
Leading / loaded questions
Checking questions
Choice of statements to measure
Implicit assumptions
Language
Tone
Summer Camp
Design variations
Qual stage
Final report
Quant stage
A B C
D E
F
Desk research
Summer Camp
Ensuring relevance to the audience
Screening
Sampling
Research currency
Timeliness
Clear explanation of reason target chosen and mutual benefit
Summer Camp
Online design options
Progress indicators
Forward and back buttons
Self checking
Filtering
Routing
Piping
Radio buttons
Sliders
Summer Camp
Online design options
Video / visuals / documents as stimuli and intuitive response mechanisms
Audio comments
‘Click and drag’
Select and deselect in a visual context
Voting on visual contexts
Hotspot mapping
Timed responses
Summer Camp
The power of pilot testing
Defining parameters
Improving design aspects
Adding key questioning areas missed
Assessment of mix of closed vs.. open questioning
Ensuring understanding and logic
Checking completion timing
Checking perceived response likelihoods
Summer Camp
Engagement
Summer Camp
Engagement
Pre-qualification
Landing timing
Reminders’ timing and impact
Incentivisation
Current environment
Summer Camp
Pre-qualification – “What’s in it for me?”
Why should the target respond?
Transparency and consent
Justification Relationship building Mutual benefit Sampling should be used to reduce data fatigue
Relevance critical
Incentive?
Pre-contact
Multi-mode approaches
Attention grabbing headlines and titles needed
Summer Camp
Response rate trends
39
Summer Camp
Timing
Initial survey invitation – online
Monday pm optimal*
Reminders
Avoid over-use of reminders
2 reminders appear to generate optimal additional returns currently** Typically 50% of initial response following reminder 1 and… 50% of response to reminder 1 following reminder 2
Sources: *Lightspeed Research survey timing tests / **Bax Interaction further education surveys results 2012/2013
Summer Camp
Incentivisation
Consider target audience(s)
Perceived value
MRS rules
Include in pilot testing
Previous experience
Comparable surveys
Summer Camp
Current environment
Time poor audiences
Data fatigue – many are over researched
Instant gratification culture
Low attention spans
Desire for intuitive design
Desire for simplicity
Base of ‘tech savvy’ respondents growing
Digital environment changing attitudes to research
Summer Camp
Getting it wrong? Ofsted example
(Please tick) strongly agree
agree disagree strongly disagree
unable to comment
1. My child enjoys school.
2. My child is making good progress because the teaching is good at school.
3. I feel that my child is safe and well cared for at school.
4. Behaviour in school is good.
5. The school is well led and managed.
6. The school takes account of children’s views.
7. The school seeks the views of parents/ cares and takes account of their suggestions and concerns.
8. Additional comments:
Summer Camp
Design checklist- getting it right
Engage your targets to gain optimal response rates
Summer Camp
Checklist - 1
1. Methodology
Relevance to the target audience?
Delivery mechanisms – right for the target audience?
Expected response rates to be measured?
Has previous experience been applied?
Summer Camp
Checklist - 2
2. Questions
Number of and logical order?
Relevance and appropriateness to the audience and the required outcomes?
Tone / language?
Ease of understanding?
Clarity of instructions?
Classification questions at the end?
Summer Camp
Checklist - 3
3. Responses
Variety and use of scales and other mechanisms for scoring responses?
4. Testing
Has a pilot test to identify possible improvements including length, layout, question order and whether content is meeting the objectives been done?
Summer Camp
Checklist - 4
5. Introduction
Timing for completion?
Explanation of purpose?
Gaining consent?
Explanation of how respondents’ data will be used?
Incentives being offered?
Meeting regulatory requirements?
Summer Camp
Checklist - 5
6. Layout
Attractiveness to the respondents?
Colour, fonts, branding, visual appeal?
Clear routing instructions?
Effective use of filtering and piping?
Use of numbered questions and sections?
Summer Camp
Checklist - 6
7. Structure
Is there the right mix of open and closed questions?
8. Post survey planning
How will the results be analysed, findings presented and followed up?
Summer Camp
Q and A?
Summer Camp
Contact detailsSteve Bax DMS MCIM AMRS FCMC Chartered Marketer
Bax Interaction Limited
Research - Strategy - Training Consultancy
Cambridge
www.baxinteraction.co.uk
T . 01223 864011
M. 07778 407676