What’s the impact?

Read this if you want to learn:

1. What do we mean by impact?

2. Why it is important to show impact?

3. How to measure the changes you want to see

4. The 4 steps you can work through:a. Planningb. Collecting your datac. Analysing itd. Using it

What do we mean by impact?The word impact can be replaced with words like ‘change’, ‘effect’, ‘outcome’, ‘benefit’ and ‘result’, but they all largely mean the same thing. You’ll hear people talking about demonstrating and measuring impact, which means taking a step back and thinking about the difference you make through the work you do and collecting data to show this. Your impact may be the difference that you make to those who directly use your service, to those in the local community more generally, or to wider society. The impact you highlight can be wide ranging and can affect many aspects of people’s lives.If someone asked you right now “What does your project achieve?”, what would you say to them? How do you know how it’s going, what works well and what doesn’t?

Why is it important to show impact?Collecting data on the difference your project is making has many benefits:• Helps you think through the problem you’re tackling• See how you are doing and how you can improve • Helps plan what you are going to do next • Tell your story and inspire others • Attracts further funding and investment• Raises awareness in the local community

How can we show the impact?To explore how you’re doing, what’s going well and what could be improved, it’s useful to start by asking a few questions:

• What need is your project meeting? For example, some young people have nothing to do in the evening on the estate so we are running street football and street cricket sessions for them

• What's the problem you’re trying to tackle? Because some young people have nothing to do, they are littering, breaking windows, fighting and getting into trouble

• Who benefits from what you’re doing? Young people on the estate aged 12 – 19, parents, residents, local businesses , sports clubs

• What do you (and others) do that makes a real difference? The young people learn sports skills and we help them get along better with each other. We help them see that team work can be positive and that there are adults that care about them.

• What is your projects’ specific goals? We want young people to feel better about themselves, grow in confidence and wanting to explore their future options.

• How will you know if you’ve achieved them? More young people asking us about coaching qualifications and wanting more information about sports or educational qualifications, more confidence, fewer reports of anti-social behaviour (ASB) and other low level criminal activity on the estate.

How can we show the impact?It’s useful to break down the whole process into 4 steps:1. Planning it: what will success look

like, what do you need to measure to see how you’re doing

2. Doing it: gathering the evidence you need

3. Analysing it: what is the evidence telling you, assessing how you’re doing

4. Using it: to learn, to improve, to share and celebrate your successes

Planning it

Doing itAnalysing it

Using it

Step 1: Planning itYou can map out what you want to change and how to get there using the diagram below. This a simple Theory of Change, a tool which helps you identify the changes you want your project to make and then encourages you to work backwards to think about what this would look like (outcomes) and the steps involved in making that change happen (activities). You could do this using post-it notes and large bits of paper, perhaps as a group exercise with different people from your project.

Goals Outcomes Activities

What long-term change do you want to make?What will success look like?

What changes or benefits result from the project? What differences do you want to make? What outcomes are you hoping for to support your goal?

What are the things you do or fund to make a difference?What activities will you do to support your outcomes?

e.g. fewer reports of Anti-social behaviour (ASB) and other low level criminal activity on the estate, lower level of local young people unemployed

e.g. local young people more confident, more young people using new skills, knowledge and confidence to gain qualifications and find out about other opportunities, more friendships

e.g. street football and street cricket sessions, developing resources coaching information, signposting to other opportunities

What do you need to know?Building on our Theory of Change, we can start by thinking about the evidence we need to collect to evaluate a project’s impact. We can break down questions into four areas:• Inputs: What resources go into a programme and how much do they cost? • Activities: What do you do that actually makes a difference? Try to be as specific as

possible about how these lead to outputs and contribute to outcomes.• Outputs: What is produced through these activities? How many people have

benefited? • Outcomes: What difference have you made? What changes/benefits can you see? How

have people changed? Collecting evidence of outcome can be challenging if you are trying to follow up with people after a project has finished – they may have moved on, or not be willing to give you further information

What do you need to know?Now, you can map the project’s outcomes, outputs, activities and inputs using post-it notes and large sheets of paper or simply put them into an Excel table.It’s important to remember that outputs tell you what is produced through activities, while outcomes are the changes or benefits that result from the whole project. Simply measuring outputs does not tell the whole story, e.g. 25 local young people attending a sports session (output) does not necessarily mean more local young people gaining qualifications (outcome/impact).

Outcomes/Impact Outputs Activities Inputs

the changes or benefits that

result from the project

what is produced

through those activities

what the project does to make a

difference

what resources go into a

programme and how much they

cost

e.g. local young people more

confident, more young people

using new skills, knowledge and

confidence to gain qualifications and

find out about other

opportunities, more friendships

e.g. 12 sports sessions held, 2 introduction to coaching

sessions held, 25 packs

produced and handed out

e.g. street football and street cricket

sessions, developing resources, coaching

information, signposting to

other opportunities

e.g. 1 coach @ £30 per hour,

sports equipment,

football shirts, cost of

resources etc.

How do we know? We will have a lower level of local youth unemployment (long-term goal)

Specific changes or results (outcome) Declined About the same Changed/

increased a littleChanged/increased a lot

How do we know? Our evidence and examples

(what signs/indicators have we seen, what have we measured, what have people told us)

More local young people have gained sports skills

Number of participants who attended sessions and then were able to demonstrate skills – survey of participants before and after sessions

More local young people have gained a qualification in sports coaching or other in the past 6 months

Number of participants who attended sessions and then got qualifications within 6 months

Survey of participants before and after session, and follow-up after 6 months

More young people who have participated in the sessions have gained employment

Number of participants who attended sessions and then gained employment within 6 months

Survey of participants before and after session, and follow-up after 6 months

Try to decide on good things that you could measure to show how close you are to achieving your goals.

For each outcome, you can come up with observable measures (which can be called indicators) to show the change the project is making and how close you are to achieving your outcomes. We suggest keeping it simple to start with, so focus on measuring only one or two outcomes.

You can use this table to think through how you will know if you’re achieving your intended outcome.

What kind of information can you collect?

There are two main types of data:• Quantitative e.g. numbers• Qualitative e.g. what people say, sounds, pictures

Quantitative and qualitative data can be put together to tell a great story, which is often more persuasive than just a table of figures. Collecting a mixture of numbers and personal accounts can bring your whole story alive. Do think about what kind of data you need – different outcomes require different methods of data collection so it is worth discussing the measures with your stakeholders before you start.It important to remember that not everything can be counted very easily e.g. enthusiasm or passion, but you might still might want to record it with a photo. Experiment and have fun!

Step 2: Doing itTo help you gather your evidence, you can start by asking a few questions:• What information do you already have?• What information do you still need to collect?• How are you going to collect it? • Who/where are you collecting it from?• Who will collect the evidence – someone from your group or somebody else?• When does it need to be collected by?

How can you collect the data?To help you gather your evidence, you can start by asking a few questions:• What information do you already have? E.g. number of participants attending

sports skills and introduction to coaching sessions• What information do you still need to collect? E.g. age, sex, previous

qualifications/education, contact details etc. of session participants• How are you going to collect it? E.g. Survey before and after sessions, follow-up of

participants 6 months later by e-mail and phone

Depending on the funding and how the commissioners you are working with measure impact, there may be set definitions and rules for collecting data. What is important is that you have definitions and rules which everyone applies in the same way.

How can you collect evidence?Showing physical change to a place Take photographs of before and after: Take

a photo

Showing physical changes to a place, or talking to stakeholders about their experiences

Make a video (you don’t need any fancy equipment)

Showing fitness changes before and after for young people

Use SurveyMonkey or another website to create a short and clear feedback survey. A simple questionnaire before and after a workshop means you can to compare answers.

Being able to gather views and opinions in a more informal way

Collect data on your social media channel’s usage e.g. Twitter’s Analytics feature (similar features are available on blogs and websites)

Talk to people and share their feedback Over coffee, by creating a more formal interview framework, by asking them to write things down

The Legal Bit… Data Protection and Consent

• It is extremely important that if you are collecting information from people (some of which may be sensitive and personal) that you are clear with them about;• What information is being collected• Why it is being collected• What will happen to it next

We give more detail on the next page

The Legal Bit… Data Protection and Consent

• Explain who is collecting the data and who this is going to be shared with• Tell people exactly what data you are collecting e.g. first and last name, email address, area• Explain why you would like to collect their data• Do not collect more data than you need• Consent is optional, it is their choice whether they would like to share their information or not and they

can make a decision at any time to no longer be contacted• You must never share people’s personal data unless you make this clear to them and they agree to this• If someone doesn’t want a quote attributed to them, you can agree to quote them anonymously• If you have filmed or taken photos, you should check with the people involved that they are happy with

the final result. If people are pleased with it, they are more likely to want to share it. If your film or photo includes children or vulnerable adults, first ask their parent or guardian to fill in a consent form.

Introducing the Community Choir…Let’s apply what we’ve learned so far to a completely different example!A local adult education college has setup a Community Choir to provide group singing classes. This project is part of a wider initiative by the College’s Community Development Team to get more people involved in adult education and life long learning. The College is hoping to attract people who would not normally consider the benefits of adult learning after compulsory education (outcome). Image from Skip The Budgie, Flickr

Introducing the Community Choir…How would you show the impact the singing classes (activity) are having on participants’ lives? Think about your answer for a few minutes before going any further…We can count how many people attended the singing classes or the number of people who came to the concerts, perhaps asking how many people each singer brought along (outputs). Let’s say one of the outcomes is ‘to involve more people in adult learning’ especially people who would not normally consider the benefits of adult learning after compulsory education. How would you measure this?

Image from Skip The Budgie, Flickr

Introducing the Community Choir…We could measure the percentage of participants who went on to take a class at the College within 1 term (indicator). We could collect this information with a follow-up survey.It would be quite easy to ask the community singers what they would be doing if they had not come to the Community Choir. Perhaps half would have stayed at home. You could also ask about changes people may have experienced, e.g. gaining confidence. You could create a simple questionnaire to measure confidence levels before and after the course.

Image from Skip The Budgie, Flickr

Step 3: Analysing itWhen you’ve collected your evidence you can then start to make sense of it all. Although it’s not always easy, try to come up with figures and statistics about your work; going back to our sports example, you might be able to say:• ‘x number of local young people attended our sports sessions in 2014, an increase of y

per cent on the previous year’• ‘z per cent went on to gain new qualifications within 6 months’• ‘x out of y said in our survey that the sessions helped them improve their confidence in

exploring other qualifications ’You can also try to decipher the general consensus of the community from your conversations with various people. If you can, pick out any key themes or points of interest.

What is the evidence telling us?When you’ve done all this, it’s time to sit down and decide what it means for the group. You can work out what went well and what you need to improve and then feed this back into your planning.It might be helpful to start with these questions:• What patterns can you see/what links can you make?• What was unexpected/surprising? Why?• What are you doing well? • What isn’t going well? Why? How could this be improved? • What would have happened if you had not run the project? It’s a good idea to ask someone else to independently check your findings to help make the best decision.

Step 4: Using itAt this point, we can return to why it’s important to show the impact of your project: To learn, improve, share your story and celebrate success.You can communicate your findings in every way you can, with everyone that you can. Your group members, the local community and funders will all be delighted to hear what you’re doing. You do some amazing things, so feel free to show off about it!

Telling people about your impactThere are many different ways you can reach people, so here are just a few ideas:• Traditional media (print, radio, TV): Media Trust have a guide on How to hit the headlines,

including hints on how to think like a journalist or editor. They also have a sample press release with some tips of what to include.

• Posters, leaflets, banners• Newsletters (printed or e-newsletters)• Email• Websites and blogs: Wordpress is one of most popular tools to build a website or blog. Just

Act has a simple guide for community groups to set up their own website with Wordpress. • Social media (Twitter, Facebook, YouTube): Locality has a social media toolkit to get you

started with Twitter and Facebook, including how to setup a profile and top tips.

Useful ResourcesJust Act’s Show the impact of your project has links to a wide range of resources and they have some great tips on showing how you’re making a difference.The Guardian’s Voluntary Sector Network has a short guide on how to measure your impact as a small charity and has collected some best bits of expert advice on measuring impact.The School for Social Entrepreneurs has 5 key lessons on measuring social impact.UnLtd’s has a comprehensive guide on Measuring, Evaluation and Social Impact Assessment .The Joseph Rowntree Foundation has also written a practical guide to evaluating community projects.Measuring Up! from Inspiring Impact is a step-by-step self-assessment tool that allows you to review and improve your organisation’s impact practice – that is, the way you plan, evidence, communicate and learn from the difference that your work makes.

And some further ideas for experimentation: SROI, CBA and the rest….One way of showing the difference you’re making is by doing a Social Return on Investment (SROI) calculation, which is a type of cost-benefit analysis (CBA). Don’t be put off by the title, it’s basically about giving a financial value to all of the things that you put into the project – your money, time, equipment – and assessing their relationship to any outcomes you achieve. Then you can say, for example, “for every £1 that we spent on our project we created £5 worth of value to the local community”. By attaching a financial value to outcomes such as ‘more local young people in employment’, you can give a monetary figure to your organisation’s social impact. Being able to show the social return generated from your investment can be a powerful way of showing your project’s impact. The key bit is finding a figure that roughly represents the value a person places on the change they experienced. You might find these questions helpful: How could you put a price on your outcome? or How much would a person be willing to pay?

SROI and similar types of analysis take time and resources to do well so it’s best to check if it’s right for you. You can do this by looking at Social Value UK’s guide, Starting Out on Social Return on Investment. They also have a more comprehensive guide on How to do an SROI analysis.

mycommunity.org.uk

@mycommunityhelp#mycommunityhelp

My Community Rights