Influences behind STAR in Scotland Key survey questions Building a questionnaire Survey methods Timing Census, sampling and checking for bias Benchmarking
Outline
Developed in Scotland with SHBVN
Established framework for satisfaction measures
Over 200 landlord comparators
Historic timeline of data – 2011 and before
Inclusive development process involving:◦ Landlords, tenant groups, market research
companies, other survey experts
Free to access guidance & support from HouseMark and SHBVN
What’s good about STAR?
‘Conducting a comprehensive satisfaction survey – a guide’ from SHR / Ipsos MORI
Very similar approach to STAR on most aspects of the survey
Some close-to-identical key questions
Consistency with the Scottish Household Survey
What’s good about the Social Housing Charter satisfaction measures?
Aspect of survey framework
Checklist
Flexible questionnaire
Response options except overall sat question
Permitted survey methods
Frequency / timing adapted slightly to match
Margin of error adapted slightly to match
Representative samples / weighting
Wording of core questions and
Original STAR alignment with Charter requirements
STAR in Scotland
code
Charter
Reference
Question
CorS1 CI1(B) Taking everything into account, how satisfied or dissatisfied are you with the service provided by [your landlord / landlord name]?
CorS2 CI10 Overall, how satisfied or dissatisfied are you with the quality of your home?
CorS4 CI29 Taking into account the accommodation and the services [your landlord / landlord name] provides, do you think the rent for this property represents good or poor value for money?
STAR / Charter equivalent questions
Some Charter questions had significantly different wording from the same themed question in STAR
Decided sufficiently distinct for each version to be included in a survey
So for each themed pair of questions you can:◦ Comply with the Social Housing Charter indicator
◦ Benefit from an additional STAR question enabling direct comparisons with the rest of the UK
The problem for STAR …
Theme Origin Question
Neighbourhood
STAR Overall, how satisfied or dissatisfied are you with your neighbourhood as a place to live?
CharterOverall, how satisfied or dissatisfied are you with [your landlord / landlord name]’s management of the neighbourhood you live in?
Involvement
STARHow satisfied are you that [your landlord / landlord name] listens to your views and acts upon them?
CharterHow satisfied are you with opportunities given to you to participate in [your landlord / landlord name]’s decision making process?
Repairs
STARGenerally, how satisfied or dissatisfied are you with the way [your landlord / landlord name] deals with repairs and maintenance?
CharterThinking about the LAST time you had repairs carried out, how satisfied or dissatisfied were you with the repairs service provided by [your landlord / landlord name]?
Keeping you informed
STARHow good or poor do you feel [your landlord / landlord name] is at keeping you informed about things that might affect you as a resident?
CharterHow good or poor do you feel [your landlord / landlord name] is at keeping you informed about their services and decisions?
Same theme – different wording
Charter ref
Question Service user
CI9(B)
Thinking about when you moved in, how satisfied or dissatisfied were you with the standard of your home?
New tenants
CI33
Taking everything into account, how satisfied or dissatisfied are you with the factoring services provided by [your landlord / landlord name]?
Owners of factored properties
CI28
How satisfied or dissatisfied were you with the overall quality of the temporary or emergency accommodation you were provided?
Users of local authority temporary or emergency accommodation
CI37
How satisfied or dissatisfied are you with [your landlord / landlord name]'s management of your site?
Users of gypsy / traveller sites
Charter questions not in original STAR
Use the exact question wording contained in STAR Questions for Scotland
Use the specified response options – as important as the question wording
Remember – for core and charter questions
Have you decided when you will conduct your large scale survey?
If so, what methodology do you plan to use?
Discuss
Build your own survey
• Starting point is the core and charter satisfaction questions
• Always ask the overall satisfaction question first
• Add to the core/charter questions with:• From optional questions from STAR
Questions for Scotland• Your own questions
• Add demographic questions as necessary
• Group questions by theme
As long as it’s statistically robust and representative, STAR in Scotland can be done by:◦ Post◦ Telephone◦ Face-to-face
On-line not acceptable for SHR: “extremely unlikely” to be representative
Permitted survey methods
Neither HouseMark, SHBVN nor SHR want satisfaction data which is too far out of date
Survey data should not be more than three years old
That is a maximum – you may conduct surveys more frequently if you wish
Take account of:◦ Data validity for comparisons◦ Utility value for business decisions◦ Cost
Keep data fresh!
Census – sent out to all tenants
If under 2,500 units, you should conduct a census and do not need to select a sample
For larger landlords, see section 3.3 of SHR / Ipsos MORI guidance on sample selection
Whether census or sample, always check make-up of respondents for evidence of bias e.g. proportionally more older tenants responding than in your tenant population?
Weighting can correct problems of bias – see section 4.4 of SHR / Ipsos MORI guidance
ARC return asks for unweighted and weighted results
Census, sampling and checking for bias
Will you conduct the survey in-house or externally commission the work?
Who in your organisation will be responsible for leading the project?
Discuss
Current E-form relates to STAR
A STAR in Scotland E-form will be introduced this year
Comparisons of STAR results will be available via:◦ PI Tracking◦ STAR benchmarking◦ Core and dashboard – alongside HouseMark cost
and performance measures
HouseMark Benchmarking
Annual benchmarking
Results available on query tool
Results available to use within report builder
Results used to inform VFM ranking
Sharing Good Practice
SHBVN Benchmarking
Unlikely all survey/feedback needs will be met by STAR in Scotland
Audit and review your whole framework for feedback and surveys
Avoid duplication and survey fatigue
Helpful to have a good base of tenant profile information◦ provides a variety of baselines for checking for bias◦ avoids having to repeatedly ask demographic questions
in surveys
Other considerations
www.housemark.co.uk/starscotland A Guide to STAR in Scotland STAR Questions for Scotland List of survey suppliers STAR in Scotland logos Links to relevant SHR documents
Coming soon Frequently Asked Questions Register for notification of updates
More info on STAR in Scotland