p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTION
An Impact Evaluation
PROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
The Manchester Active Voices Essential Leadership Programme ran as a residential programme between 17th and 23rd July 2010
The programme was designed for 16-25 year and consisted of three main strands
Civic Participation
Empowerment
Leadership
The programme offers young people the opportunity to operate within established strategic frameworks and develop key transferable skills, necessary to become effective leaders within their schools, homes and communities.
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
AIMS AND OBJECTIVES
The aims of the programme are: To raise awareness of issues impacting on Young People within the UK
To provide for opportunities for YP to engage with the community
To help achieve these aims the programme had the following objectives:
To increase the levels of knowledge and understanding in
•Equality and Diversity
•Leadership
•Communication
•self and personality
•Social / Domestic Issues
To agree a series of actions or goals that work towards increasing community involvement, seeking education or employment, taking ownership and removal of barriers to personal growth
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
A CHAIN OF IMPACT
In order to effectively evaluate the impact of this programme, the aims and objectives were aligned with a ‘need’ for each of the 5 steps in the evaluation chain as shown below.
This will ultimately lead to the programme being able to evidence a Social Return on Investment
Chain of Impact
Participants find programme relevant / important
Participants increase KU of key skills / themes
Participants complete action plan and engage with community
Participant influences community policy
Programme OutcomeFor YP to make significant impact within their commu...
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
THE NEED
NEED OBJECTIVE
There is a need for YP to recognise the relevance and importance of an empowerment programme in addressing their personal growth needs as well as to reflect on their experiences during the programme
Attendees should rate the content of the programme as relevant and have indicated that their needs are fully or partially met.
YP should be able to demonstrate emotional intelligence through reflective journals
There is a need for YP to increase their levels of knowledge and understanding in the identified key issues
80 – 100% of attendees should indicate an increase in their KU of each of the key learning areas resulting in 70% of participants showing ‘some’ or ‘complete’ KU in the areas
There is a need for YP to apply and develop their skills within their own circumstances and implement changes towards achieving their aims
60-80% of attendees should show commitment to progressing and applying their skills through the development of an action plan.
40-60% of participants should evidence actions undertaking to achieve their goals up to 6 months following the programme
There is a need for achieved their aims in terms of improving their educational options, employability, community involvement and influencing community policy
30-50% of attendees should be able to evidence:•Improved attainment / employability•Increased involvement within community•Improved efficiencies within community policy / services
There is a need for the programme to demonstrate a return on investment through the measuring of YP’s aims and social benefit
To evidence a return on investment through cost to benefits ratio calculation and measure of intangible benefits
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
A PROFILE OF YOUNG PEOPLE
Ten Young People took part in the empowerment Programme. Of these ten the following information was captured:
4 Young People indicated that they were not in Education, Employment or Training (NEET)
6 indicated they were in involved in some level of employment, education or a combination of both
7 suggested they had previous involvement with at least one community group, this often included groups within education settings
4 indicated involvement with the shaping and influencing of local service delivery
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
Participants were asked to rate the relevance of the programme in addressing their personal growth needs using the below scale
•None of the content was appropriate / did not reflect need
•Little content was appropriate / needs were not met
•Some content was appropriate / needs partially met
•All content was appropriate / needs fully met
•100% rated the programme as ‘all content appropriate / needs fully met’
They were also asked to rate the importance (on a 4 point scale) of the programme in relation to community involvement being a key driver for change:
•100% rated the programme as ‘Very Important’
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
As well as rating the programme YP also had the opportunity to provide additional comments summarising their reaction….. Below is an example…..
Could not have found a better life changing opportunity …. every piece
of information gained…. has been inspirational and 100% effective.
Without a doubt, I am beyond pleased I partook in this course.
This has been an absolutely intense and enjoyable experience. I have
learned a lot about myself and others around me…… I started this course running with the world spinning too fast for me, and now I can walk with
the world following behind
The programme has been a great eye opener for me, and I look forward to implementing the
knowledge gained to improve society for everyone
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
As part of the programme Young People kept a daily reflective log to account for their thoughts and feelings as the programme progressed.
These logs have been reviewed by identifying the frequency of use of key descriptive words. Words such as ‘emotional’, ‘intense’, ‘enjoyable’, ‘great’. These were also categorised words relating to some measure of satisfaction or some measure of emotion. A proportion of frequency is demonstrated below
69%
31%
Emotion
Satisfaction
Proportion Words
Category
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
Further analysis of the frequency of word use suggests an emotional journey for the participants. The chart below shows the frequency of word use per day. Notably ‘satisfaction’ or enjoyment words feature high to begin with however emotionally descriptive words soon become more prevalent.
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
Saturday Sunday Monday Tuesday Wednesday Thursday
Emotion
Satisfaction
Proportion Words
Day
Category
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
…another quote
This programme has really helped me bring out my confidence and helped
me to communicate with others from a number of different backgrounds.
I feel like this programme has given me a vision of where I want to be in the future and the skills that I have
learnt will help me get to where I want to be.
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
An objective of the programme was to increase the levels of knowledge and understanding in several key areas:
Equality and diversity – to be completed
Leadership
•Theories and models of leadership
•Understanding of leadership, vision and strategic thinking
Communication
•Knowledge of effective communication, presentation, giving and receiving feedback
•Understanding of diversity and prejudice, how to tackle conflict
Self and personality
•Understanding of self and emotional intelligence
•Knowledge of personality types and what they mean
Social / Domestic Issues
•Understanding the effects of domestic abuse, the tactics and tell tale signs
•Knowledge of the barriers to help and what you can do about it
•Understanding of what makes a healthy relationship
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
Participants were asked to comparatively rate their levels of knowledge and understanding of each of the learning outcomes both prior to, and following the programme. This was measured on four point scale with 1 being no KU and 4 being complete KU.
100% of participants indicated an increase in their levels of knowledge and understanding in each of the outcomes
On average 78% of participants suggested they had little or no knowledge and understanding of all of the outcomes prior to the workshop
This shifted to 100% of participants indicating some or complete knowledge and understanding following the programme
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
The measured change for each participant was recorded as a score. These were totalled for each learning outcome to show the area of the highest increase; the results of which are below:
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
Further to this the charts below demonstrate the average shift of levels of knowledge and understanding (K/U) in the 5 key learning areas.
Leadership
78%
17% 6% 0%0% 0%39%
61%
0%
20%
40%
60%
80%
100%
No K/U Little K/U Some K/U Complete K/U
Pre Post
Self and Personality
56%
33%
11% 0%0% 6%
39%56%
0%
10%
20%
30%
40%
50%
60%
No K/U Little K/U Some K/U Complete K/U
Pre Post
Communication
22%
50%28%
0%0% 0%28%
72%
0%10%20%30%40%50%60%70%80%
No K/U Little K/U Some K/U Complete K/U
Pre Post
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
Further to learning, participants were also asked if they committed to making personal changes and the continued development of their skills.
All participants committed to development in the following areas:
Leadership skills (9)
Personal / social skills (9)
Community Involvement (8)
Home life improvements (8)
Education / Employment (9)
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
At the point of this evaluation 7* YP completed an action plan which set out a series of targets or goals for them to achieve over the coming months. These goals are designed to implement changes through the application of their skills and work towards achieving the programmes overall aims.
They generated goals in the following areas:
Involvement in community programmes / youth panels
Taking ownership over personal / community based circumstances
Seeking out educational / employment opportunities
Identifying and removing personal growth barriers
*please note that 2 YP are still expected to submit their action plans which will be incorporated into the final evaluation
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
The Goals
the table below represents a paraphrase of the goals set for each of the key areas:
involvement in community programmes / youth panels
• All of the returned action plans included goals of increasing involvement with community programmes. This included continued involvement with MAVYEP
• Two YP suggested developing workshops within MAVYEP
Taking ownership over personal or community based circumstances
•Three YP identified situations where they could share their knowledge and skills within community groups and their peers
•Three YP referred to taking responsibility for finding a job and working on their vision
Seeking out employment / education opportunities
• Four YP indicated that they intended to find employment or begin training to aid future employment
• Two YP suggested that they would be continuing into further education
Identifying a barrier to personal growth and removing it
• Three YP suggested lack of confidence was a barrier to growth and were working on removing that which included improving public speaking. A fourth identified a leadership programme to attend•Two YP referred to improving their personal skills through reflection and applying their learning into daily situations
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
The Goals
Each goal within the action plans were reviewed to identified running themes. The chart below demonstrates the frequency of themes for all of the key areas.
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
WHAT NEXT?
Manchester Active Voices Youth Empowerment Programme has clearly been highly successful in achieving its short term aims:
The programme engaged with young people that were identified as NEET, who were seeking employment and with a drive to be active in community involvement
Every Young Person recognised the relevance and importance of the programme in relation to their growth needs and to the fact that community involvement is a key driver in change
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
WHAT NEXT?
Every Young Person could evidence an increase in their levels of knowledge and Understanding of each key learning outcome
Through reflection Young People demonstrated an increase in their emotional intelligence and awareness
Through action planning Young People committed to developing their skills and reaching goals
Young People demonstrated a passion to share their skills and experience and make significant improvement s to their own lives and the community
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
WHAT NEXT?
Further Evaluation
Young People’s progress towards their identified goals will be monitored by young people being given the opportunity to reflect, evidence and assess their achievements
Qualitative and intangible measures will be evaluated to show the impact the programme has had in terms of confidence, improved community engagement and increased leadership
Quantitative measures will be identified and evaluate to show a financial benefit to the programme. Areas of benefit will include:
Gaining / improving employment
Delivering community programmes / workshops
Influencing community service delivery
p2e Driving Performance through Evaluation
Richard [email protected]
07737514762
PROGRAMME IMPACT REACTION REFLECTION LEARNING ACTIONPROGRAMME IMPACT REACTIONPROGRAMME IMPACT REFLECTIONREACTIONPROGRAMME IMPACT LEARNINGREFLECTIONREACTIONPROGRAMME IMPACT ACTIONLEARNINGREFLECTIONREACTIONPROGRAMME IMPACT
WHAT NEXT?
Evaluation Timeline
Task Aug Sep Oct Nov DecEvaluating Feedback and action plans from YEP residential
Preparing and delivering evaluation presentation
MAVYEP collection of ongoing action plan progress
Evaluation of action plan feedback
Telephone interview with YP
Evaluate cost to benefit ratio
Prepare and Deliver final ROI evaluation report