Embed Size (px)
Citation preview
RESILIENT FUTURES Evaluation Report, September
2018
Community Matters Pty Ltd
1
Table of Contents Executive Summary ................................................................................................................................. 6
1 Introduction .................................................................................................................................... 8
1.1 Background ............................................................................................................................. 8
1.2 Methodology ........................................................................................................................... 8
1.3 Methods .................................................................................................................................. 9
1.4 Programme Theory ................................................................................................................. 9
1.5 Summary of the Interim Report ............................................................................................ 14
1.6 Structure and limitations of the Report ................................................................................ 14
2 Outcomes for young people ......................................................................................................... 16
2.1 Introduction .......................................................................................................................... 16
2.2 Subjective wellbeing ............................................................................................................. 16
2.2.1 Self-awareness .............................................................................................................. 17
2.2.2 Self-regulation ............................................................................................................... 19
2.2.3 Confidence and positivity .............................................................................................. 19
2.3 Educational aspirations ......................................................................................................... 20
2.4 Passing it forward.................................................................................................................. 22
2.5 Sustainability of outcomes .................................................................................................... 23
2.6 For whom? ............................................................................................................................ 23
2.6.1 Young people in crisis or who have experienced trauma ............................................. 29
2.6.2 Summary ....................................................................................................................... 31
2.7 Generating outcomes: contexts and mechanisms ................................................................ 32
2.7.1 Authenticity ................................................................................................................... 32
2.7.2 Relationship .................................................................................................................. 34
2.7.3 Reinforcement .............................................................................................................. 37
2.7.4 Boundaries .................................................................................................................... 37
2.8 Conclusion ............................................................................................................................. 37
3 Quantitative Data .......................................................................................................................... 39
3.1 The Survey ............................................................................................................................. 39
3.2 Demographics ....................................................................................................................... 39
3.3 Change over time .................................................................................................................. 40
3.3.1 Scale scores ................................................................................................................... 40
3.3.2 PERMA Questions .......................................................................................................... 44
3.3.3 K10 Questions ............................................................................................................... 46
3.3.4 ONS Questions ............................................................................................................... 49
3.4 Differences between sub-groups at either T1 or T2 .............................................................. 50
2
3.4.1 Scales ............................................................................................................................. 50
3.4.2 Questions ...................................................................................................................... 51
3.5 Differences based on starting score ...................................................................................... 52
3.6 Conclusion ............................................................................................................................. 53
4 Project model and contributions .................................................................................................. 55
4.1 Modes of implementation and outcomes for young people ................................................ 55
4.1.1 Five skills or ten ............................................................................................................. 58
4.1.2 Implicit teaching ............................................................................................................ 58
4.1.3 Group work and individual work ................................................................................... 59
4.1.4 Mentoring ..................................................................................................................... 63
4.1.5 Trauma informed practice ............................................................................................ 67
4.1.6 Agency settings ............................................................................................................. 68
4.2 On-line resources .................................................................................................................. 69
4.3 Conclusion ............................................................................................................................. 70
5 Workers and agencies ................................................................................................................... 71
5.1 Impacts on agencies’ work .................................................................................................... 71
5.1.1 Cementing existing approaches .................................................................................... 71
5.1.2 New skills and approaches ............................................................................................ 72
5.1.3 Impacts for staff ............................................................................................................ 72
5.1.4 Strengthening teams ..................................................................................................... 73
5.1.5 Common language ........................................................................................................ 74
5.1.6 The value of Resilient Futures training for skilled practitioners ................................... 75
5.1.7 Diffusion ........................................................................................................................ 76
5.2 Linking project activities to agency outcomes ...................................................................... 77
5.2.1 Establishment phase ..................................................................................................... 77
5.2.2 Adaptability ................................................................................................................... 77
5.2.3 Training for agency staff ............................................................................................... 79
5.2.4 Agency support ............................................................................................................. 80
5.2.5 The role of the champion .............................................................................................. 81
5.2.6 Resources ...................................................................................................................... 82
5.2.7 Constraints .................................................................................................................... 83
5.2.8 Sustainability ................................................................................................................. 85
6 Contexts and Mechanisms ............................................................................................................ 88
6.1 CMO’s for young people ....................................................................................................... 88
6.1.1 Selective Targeting ........................................................................................................ 88
6.1.2 Reinforcement .............................................................................................................. 89
3
6.1.3 Credibility and trust ...................................................................................................... 90
6.1.4 Appropriateness ............................................................................................................ 90
6.2 CMO’s for workers ................................................................................................................ 91
6.2.1 Reinforcement .............................................................................................................. 91
6.2.2 Satellites ........................................................................................................................ 92
6.2.3 Appropriateness ............................................................................................................ 93
6.2.4 Other ............................................................................................................................. 93
6.3 CMO’s for agencies ............................................................................................................... 94
6.3.1 Reinforcement .............................................................................................................. 94
6.3.2 Other ............................................................................................................................. 95
6.4 Updated Theories of Change ................................................................................................ 95
7 Conclusion ................................................................................................................................... 101
7.1 Future Directions ................................................................................................................ 104
7.1.1 Working with families ................................................................................................. 104
7.1.2 Working with young people in crisis ........................................................................... 105
7.1.3 Working with younger young people.......................................................................... 105
7.1.4 Working with agencies to minimise disruption .......................................................... 105
7.1.5 Working with isolated staff ......................................................................................... 106
8 Appendices .................................................................................................................................. 107
8.1 Appendix 1 – SAHMRI Theories of Change 2018 ................................................................ 107
8.2 Appendix 2 – Survey ............................................................................................................ 109
8.3 Appendix 3 – Survey Data ................................................................................................... 109
4
5
Acknowledgements
Community Matters would like to acknowledge the assistance of David Kelly, Matthew Iasiello and
Adele Liddle, Dunja Nedic of SAHMRI, in coinvestigating throughout the evaluation of the Resilient
Futures Project, as well as contributing to the construction of this report.
We would also like to thank all of the workers, young people and agencies who took time out of their
schedules for interviews for this report, to administer surveys with young people and to do those
surveys.
Daniel Ball (Community Matters) and Matthew Iasiello (SAHMRI) have been responsible for the
quantitative component of this evaluation. Bronny Walsh, Vikki Booth and Daniel Ball (Community
Matters) have been responsible for the qualitative component. Dr Gill Westhorp (Community
Matters) has provided expert realist evaluation methodology advice, contributed to the writing of the
report and provided final comments on this report.
Research ethics approval was sought and gained from the Aboriginal Health Research Ethics
Committee (AHREC).
6
Executive Summary The Wellbeing and Resilience Centre (WRC) in the South Australian Health and Medical Research
Institute (SAHMRI) has, between 2015 and 2018, conducted a project entitled ‘Resilient Futures’. The
project aimed to improve wellbeing for vulnerable young people from disadvantaged communities by
delivering, through schools and youth sector agencies, resilience training and mentoring support for
young people.
This report summarises the evaluation of the project. It identifies outcomes for young people and
youth agencies; how outcomes were generated, and the circumstances in which the project is more
or less effective. This work was a joint venture between the staff at the Wellbeing and Resilience
Centre and Community Matters Pty Ltd.
Realist evaluation was selected as the methodology because it recognises that projects work
differently for different sub-groups and in different contexts. It provides a way of exploring and
explaining patterns of outcomes, and. the learning generated can support project refinement.
The project was initially designed as a structured learning programme in resilience skills, supported by
mentoring and on-line resources. Workers in schools and youth agencies would undertake the
resilience skills training themselves and then deliver a modified version of the programme to young
people. Due to feedback from the delivering agencies this was later adapted, through a process of
consultation, to a project that supported and resourced the delivery agencies to adapt and use core
materials in ways that were appropriate to their own settings and clients. Intentional Practice was
adopted as a means of ensuring fidelity to the evidence base and rigour of the resilience skills.
The change in structure was well received by the agencies running programmes.
The findings from the evaluation largely support the programme model and respondents reported an
array of outcomes which related to young people, workers and agencies.
Managers, workers and young people themselves all reported outcomes for young people including
improved self-regulation, greater positivity and using the skills in their life outside the setting in which
they were taught. Other outcomes reported for young people included greater self-awareness,
improved confidence and greater positivity.
These findings were also supported by the quantitative date which showed small, statistically
significant improvements on both the wellbeing scale (PERMA) and the psychological distress scale
(K10) which were included in the survey young people were asked to complete.
The analysis of the survey results suggest that the programme was more effective for males than
females and that it was more effective for young people with lower levels of wellbeing or higher levels
of psychological distress.
Interviewees also noted outcomes for workers which included learning new skills and concepts,
greater confidence in similar approaches they were already using and use of the skills in their own
lives which linked to improved relationships and an improved ability to teach the skills to young
people.
Potentially one of the most important outcomes reported was the development of common language
and approaches within teams which came about as a result of multiple staff being trained and mutual
reinforcement between staff members. Common language was seen as critical to providing consistent
7
messaging to young people, strengthening teams, reducing misunderstandings and aiding workers to
articulate what they were doing and why.
There were less outcomes reported at the agency level but the project provided confidence and
validation to managers in approaches that were already being used, more consistent approaches
across staff and stronger relationships between staff members resulting in a more positive work
environment.
Fading impacts over time were noted as a concern as additional, competing priorities were introduced
to agencies and staff turnover reduced the impact of the common language and understandings that
were developed during the project. Further training was suggested as a way of refreshing learning,
renewing enthusiasm and inducting new staff.
8
1 Introduction
1.1 Background The Wellbeing and Resilience Centre (WRC) in the South Australian Health and Medical Research
Institute (SAHMRI) has developed and piloted a project titled ‘Resilient Futures’. The project aimed
to improve wellbeing for vulnerable young people from disadvantaged communities by delivering,
through schools and youth sector agencies, resilience training and mentoring support for young
people.
The evaluation was intended to inform future decision-making about the Resilient Futures Project,
including whether it should be maintained, adapted or scaled out. To do so, the report looks at
whether the approach ‘worked’, for whom and in what contexts. In earlier reports, the evaluation
also aimed to inform programme improvement over time. The purposes of this evaluation were
therefore:
• to identify outcomes of the Resilient Futures Project for young people and youth agencies;
• to understand the circumstances in which resilience training for young people is more and less
effective, and why;
• to contribute to programme refinement and adaptation to different contexts, for work into
the future.
Community Matters Pty Ltd, an independent consultancy company, was contracted to work with
SAHMRI to undertake the evaluation.
Research ethics approval was sought and gained from the Aboriginal Health Research Ethics
Committee (AHREC) in South Australia.
1.2 Methodology SAMHRI had determined that realist evaluation methodology was appropriate for the evaluation of
Resilient Futures prior to contracting Community Matters. Realist evaluation was selected because:
a) It recognises that programmes work differently for different sub-groups and in different
contexts and provides a way of exploring and explaining those different outcomes;
b) It is a learning-oriented methodology which can support programme refinement.
Realist evaluation operates from different assumptions than most evaluation approaches. Firstly, it
assumes that programmes do not directly cause outcomes. Rather, they provide resources,
opportunities and constraints (‘resources’) which generate particular ‘reasoning’ (internal responses),
which in turn generate new decisions which generate new behaviours which generate new outcomes.
The interaction between reasoning and resources is known as a programme mechanism. Programmes
provide different opportunities and resources to stakeholders along the programme implementation
pathway, and so mechanisms ‘fire’ (operate) at all stages of the implementation pathway as well as
for programme participants.
New behaviours, internal states, and higher-level outcomes are all outcomes at different levels of a
hierarchy of outcomes.
All programmes have ‘intended mechanisms’ – that is, ways in which they are expected to work.
However, intended mechanisms only operate when the context is right, and programmes also cause
unintended mechanisms to operate. Realist evaluation therefore collects data about all three aspects
– contexts, mechanisms and outcomes – in order to build explanations of whether, for whom, how
and why programmes do and don’t work.
9
The overarching question for realist evaluations is not, therefore, “Does this work?” or “Did this
work?”, but “For whom does this work, in what contexts, in what respects, to what extent, and how?”
Understanding answers to these more detailed questions enables programme personnel to improve
the programme design, adapt the programme for different contexts or population groups, change
delivery modes while still enabling underlying mechanisms to operate, and so on.
Realist evaluation is intended to be iterative, gradually developing and refining programme theory
through recurrent rounds of evaluation. This evaluation design capitalised on that characteristic,
and used learning from earlier years of the Resilient Futures programme to refine the qualitative
data collection in subsequent years of the evaluation as new topics of investigation were added.
1.3 Methods For this final round of the evaluation, interviews were conducted with staff and managers from the
organisations involved in the project and, where possible, young people who had participated in the
programme. The organisations involved were:
• Service to Youth Council – Helping Young People Achieve (SYC) (2 sites);
• Red Cross;
• Wirreanda Secondary School WAVE (WAVE) ;
• Northern Adelaide Senior College (NASC);
• Adelaide Hills Vocational College(AHVC) :
• Junction Australia (JA); and
• Youth Education Centre Flexicentre.
In each agency one person in a managerial role was interviewed, along with varying numbers of staff.
Only young people in education settings were approved by the Research Ethics Committee to be
interviewed for the evaluation, so young people in youth detention and residential care homes were
not approached.
The interviews were taped and transcribed and then subjected to realist qualitative analysis.
Young people were also asked to complete an 18-question survey. The survey included three sub-
scales that were designed to measure mental wellbeing and psychological distress, notably anxiety
and depression. Responses were entered into SPSS and analysed. This was the second round of data
collection so change over time could be investigated. Data was collected in February 2018 and again
in July 2018.
1.4 Programme Theory In 2015 and early 2016, staff from SAHMRI and the evaluation team initially worked together to
produce programme theory models for the Resilient Futures Project, showing the anticipated
pathways to outcomes for young people and youth services. At that stage, it was anticipated that
agencies would deliver a common programme which SAHMRI staff had developed. It became clear,
during trialling of the project, that it was necessary to adapt it more substantially for different contexts
and target groups, and that workers and agencies needed to be able to undertake that adaptation.
Consequently, a new theory was developed for quality adaptation of the programme. The first
diagram below shows the programme theory for the programme overall, and the second shows the
theory for quality adaptation.
10
The first programme theory diagram should be read from bottom of the diagram to the top, and from
the left-hand side of the page to the right. The programme could only be implemented if youth
agencies agreed to do so. Consequently, the original programme theory for Resilient Futures began
with the selection of agencies and their agreement to be involved (the bottom of the left-hand-most
column of the model). That process included criteria for selection being determined and agencies
which met those criteria being identified. The evidence base for resilience training was presented to
those agencies and a shared vision was developed for the project structure and delivery methods.
Agency commitment to being involved then lead to the other four columns of the model.
The second column described the involvement of workers, beginning with them volunteering to
participate. (It was assumed that all participating staff would be volunteers and that this was
important to the effectiveness of the programme.) Staff were then provided training relevant to the
project. The anticipated outcomes were the development of a common language between workers,
the ability for workers to hold each other accountable for good practice and increased consistency in
agency practice. All three outcomes were predicated on multiple staff from an agency or site being
trained.
The third (middle) column described the process for recruitment of young people, with agencies
developing recruitment procedures and promoting Resilient Futures to potential participants. The
model suggested that both trust in the worker and valuing the intended programme outcomes would
operate as mechanisms contributing to young people’s voluntary choice to participate.
The fourth and fifth columns dealt with young people. Provision of resilience skills training for young
people was anticipated to contribute to increased self-awareness and to learning of the resilience
skills, which were the primary mechanisms for improved outcomes. Experiencing positive outcomes
as a result of practising the skills would then lead to continued use of the skills and incorporation of
skills in their day to day lives. Over time, this would contribute to increased resilience and persistence
– interim outcomes which also operate as mechanisms contributing to the intended overall outcomes,
which were increased retention in education and training and improved life outcomes.
Throughout the process workers were to not only train young people in the resilience skills but also
provide coaching and mentoring, which would support active reflection by the young people and
ongoing use of the skills. This would reinforce the outcomes from the skills development process and
thus contribute to overall outcomes.
It should be noted that while this initial theory of change did identify (albeit superficially) the
anticipated mechanisms to achieve intended outcomes, it did not specify the contexts in which, or
groups of young people for whom, outcomes were expected to be achieved.
SAHMRI staff updated these initial theories of change in 2017 and finally in 2018 in response to
changes in the programme, insights gained through their interaction with provider organisations and
the 2017 Resilient Futures evaluation report. The initial stage of agency engagement was deleted and
the remainder was split into two diagrams, with one for workers and one for young people. Each of
the new diagrams reflected adaptations to the programme, and provided somewhat greater detail
about anticipated strategies and outcomes. Enabling and disabling features of context were also
identified, but anticipated mechanisms were not identified. The revised theories are provided in
Appendix 1.
11
Figure 1 - Initial rough theory for Resilient Futures
12
The second major component of theory development was undertaken in response to the change from
a manualised programme to resourcing agencies to develop and implement their own resilience
training. Programme staff reported that this pivot was guided by the implementation science
literature, and underpinned by an intentional practice model and approach titled the Life Buoyancy
model.
Intentional practice seeks to bring ongoing mindful awareness to the intent of an intervention,
including the desired outcomes and the processes by which they are achieved. The intentional practice
approach can be operationalized through the following key questions:
1. What is the intent or purpose driving the intervention?
2. What outcome is the focus of the intervention?
3. How, or by which method or practice, is this outcome being achieved?
The intent was that agencies should be able to contextualise the materials while remaining faithful to
the intent of the original programme. The theory therefore addressed the factors that are necessary
to achieve quality adaptation. Six areas were hypothesised as being important. These were:
• Context characteristics,
• Implementing organisation characteristics,
• Relationships between SAHMRI and agencies,
• Staff characteristics,
• Time and resources, and
• Characteristics of the Resilient Futures programme itself.
The contextual characteristics that were considered important were organisational and peer support
for resilience. Departmental support for resilience approaches would trigger an imperative to act and
a sense of legitimacy while peer professional support would encourage workers to engage with the
project and adapt the programme to their situations.
The organisational characteristics that were deemed necessary were a formal commitment to
wellbeing, the nomination of a senior staff member to act as a driver and the investment of time and
resources to enable adaptation.
Relationships between SAHMRI and the agencies were considered important to enable SAHMRI to
provide resources and supports to reinforce anchor points and worker learning. Having highly skilled
staff at SAHMRI would generate respect, thus making it more likely that workers in other organisations
would engage with them and the materials they provided.
Provision of the TechWorks training and the IMPACT training would enable service providers to
develop a deep understanding of the resilience skills and anchor points which would, in turn, enable
them to teach the skills to young people. This was also linked to staff using it in their own lives, thereby
increasing their credibility with and ability to engage young people.
Time and resources to both adapt and implement the programme were also seen as important.
Having time and resources to adapt the programme for young people was linked to increasing its
credibility. Having time and resources to implement the programme was linked to reaching sufficient
intensity (or programme dose) to be effective.
The final of the six elements was characteristics of the Resilient Futures programme itself. Being
evidence based and delivery being negotiated in organisations meant that the programme could be
13
seen as credible. Resource materials being clear, easy to understand and comprehensive meant it
could be seen as viable. Both in turn linked to workers being able to understand, use and deliver the
anchor points (anchor points being the central ideas in the programme).
Diagram 2. Programme adaptation model.
Realist evaluation proceeds by testing programme theories to identify whether programs work as
anticipated, in what circumstances they do and do not, and the nature and extent of outcomes
generated. We return to these programme theories in Chapter 5 of this report (Contexts and
Mechanisms).
14
1.5 Summary of the Interim Report An interim report was produced in 2017. 18 interviews were conducted with staff from four
organisations. Young people were also asked to complete a short survey (the same as was used in this
evaluation). However, as the survey was only conducted once with those young people. it was not
possible to use the survey data to look at change over time or outcomes for young people.
The interim report focused largely on the change from the highly structured initial iteration of the
program to the more flexible approach that superceded it. The report also looks at outcomes for
young people, workers and agencies as perceived by the interviewees.
The findings from the interim report were largely positive with the change to a more flexible approach
being well received and respondents suggesting it enabled greater impacts with young people than
the previous programme. This was because the programme content could be adapted to the
particular needs, interests, and opportunities for learning of the young people involved.
A number of outcomes were described for young people including learning, retention and use of skills
and terminology from the Resilient Futures programme. Examples of changes in group dynamics and
the interaction of individuals within groups were also provided. It was not possible to determine the
proportion of participants for whom positive outcomes were achieved. No negative outcomes for
young people were reported.
Outcomes for some staff included use of resilience skills in their own lives and an associated
improvement in emotional state, improved knowledge and a wider range of tools to use with young
people.
At the agency level, the development of common language and shared understandings about
resilience skills was linked to improved communication between staff.
The interim report found that there were a number of key practices which impacted the effectiveness
of the programme. These included appropriate targeting and selection of young people to be involved
and providing opportunities for reinforcement of learning for both staff and young people.
Staff being seen as credible and young people trusting the staff were two of the key mechanisms that
enabled young people to engage with the programme. Respondents commonly reported that using
resilience skills in their own life was critical to being seen as credible as it allowed them to provide
personal, real-life examples.
Where multiple staff received Resilient Futures training, discussions between them served to reinforce
their learning and enabled the development of common language within teams and/or organisations.
1.6 Structure and limitations of the Report This final report provides findings from the final round of interviews and an analysis of the quantitative
data.
In the final round, interviews were conducted with 30 respondents representing four organisations
along with 3 respondents from the Wellness and Resilience Centre1. Eight young people were
interviewed from three agencies. Staff from seven agencies were interviewed, including seven
managers and fifteen other staff members including teachers, mentors and case managers. Additional
1 The key to the quotes throughout this report: YP – Young Person involved in the agencies; S – Staff and M – Managers, at agencies involved in the Resilient Futures Project; WRC – Wellness and Resilience Centre staff at SAHMRI.
15
interviews were also attempted but contacting the potential interviewees proved difficult. This
introduces a potential for bias as it is possible that people who had less positive experiences may have
been less inclined to respond to requests for interviews. Interviews were also conducted with three
staff from the Wellness and Resilience Centre at SAHMRI.
In the previous data collection period, towards the end of 2017, 18 interviews were conducted
covering four agencies and included a range of staff. This report concentrates particularly on the final
round of data but draws on earlier data as required.
The quantitative data comes predominantly from two organisations, with a small amount from
another two. Perhaps unsurprisingly, the quantitative data comes from schools involved in the project
rather than youth agencies. The schools deal with larger numbers of young people who are, for the
most part, more easily accessible, both to gain consent for their participation and to have them
complete the survey.
The quantitative data covers two points in time and represents the first opportunity to analyse change
over time for young people participating in the programme using quantitative data.
The first round of data collection was undertaken near the beginning of Term 1 2018. However, the
actual dates for data collection varied and it cannot actually be considered true pre-project data, but
rather, ‘early in the Project’ data. The second round of data collection was undertaken towards the
end of Term 2 2018. This means that change over the course of one semester can be assessed. The
differences in approach from organisation to organisation means that the young people may be
involved for longer than this and so it is possible that further, or different, changes may have occurred
after the second round of data collection.
Some of the young people were receiving information, training and/or counselling from multiple
sources. This, and the natural changes of young people over time, reduced the ability to attribute
outcomes to the project. One worker respondent to the 2017 round of interviews noted this
complexity.
…all of them have a lot going on in their lives and I think that’s the tricky part, to
know if anything’s actually changing for them or not… (Resilient Futures Evaluation
Report, 2017, Page 11)
The structure of the report is based around the initial key evaluation questions for the evaluation,
because some of the questions could not be answered in earlier rounds (in particular, relating to
change over time for young people). In addition, evidence has been extracted for questions that were
added in 2017 relating to the impact of the change from a highly structured delivery model to a more
adaptable model. In 2018 additional foci were added. These were to examine the value of the Resilient
Futures training to experienced, skilled practitioners, the value of implicit and explicit teaching
methods and the value of common language about resilience skills.
16
2 Outcomes for young people
2.1 Introduction There are three sources of data about outcomes for young people: interviews with young people
themselves, interviews with workers in service delivery agencies, and a pre-post questionnaire
conducted with young people.
Eight young people from three sites participated in the face-to-face interviews for this phase of the
evaluation. More interviews were scheduled but, on the day and for various reasons, the interviews
did not go ahead. The reasons for late cancellations included high anxiety and last minute ‘stress’
events in the young people’s lives.
All of these interviews were with young people in schools, rather than from juvenile detention or
residential homes settings. This was because the initial research ethics application specified target
organisations which were all schools and so did not cover the introduction of young people from
alternative youth settings.
It should be noted that young people who had had a positive experience in the Resilient Futures
Project may have been more likely to have been asked by agencies to be involved in the evaluation,
and/or more likely to have agreed to participate which could have introduced a positive bias amongst
respondents. Organisations were asked to try and include young people who had not had a positive
experience with the Resilient Futures Project, but none came forward during the interview phase and
whether they were asked and then refused was unknown.
Two of the young people interviewed were unable to remember participating in the programme.
Another did not recognise the name of the project but did recognise some of the skills that were
taught as part of it.
Agency staff reflected through their interviews about the impact of the Resilient Futures Project on
young people’s sense of subjective wellbeing and this chapter also incorporates their opinions,
observations and anecdotal evidence from feedback to them from young people.
The key evaluation questions relating to young people were:
• To what extent and in what ways does participation in the Resilient Futures program impact
young people's sense of subjective wellbeing, and how?
• For what sub-groups of young people is the Resilient Futures program effective, how and why?
• In what ways are young people’s educational aspirations and achievements affected, in what
contexts, how & why?
2.2 Subjective wellbeing The evaluation question for this component of the report was “To what extent and in what ways does
participation in Resilient Futures impact young people's sense of subjective wellbeing, and how?”
A number of subjective wellbeing outcomes were identified by and in relation to the young people.
These included higher levels of self-awareness, improved self-regulation and increased confidence.
This played out by demonstrations of learning how to manage their emotions; improved
communication skills; and how to manage their relationships with their friends and family.
17
Of the young people who could remember doing Resilient Futures work, all said that it had benefitted
them, particularly in the personal lives.
We’re really encouraged to do what we think will benefit us, rather than, like, ‘this
is what you’re being taught’. They encourage us to think a bit differently. ..... I’ve
struggled with mental illness for, like, 12 years.... I sort of need to learn to manage
it as it changes, if that makes sense and mindfulness definitely plays a big part in
that. Like, you’ve really got to focus on what’s going on. (YP1, page 2 and 5.2)
I was a little bit apprehensive of doing it (Resilient Futures course), because I
thought it would just kind of be…. like, garbage, you know, and they’re just going
to, like, feed me all this, like, this crock of shit, basically, and, like, I don’t want to
really hear it if it’s boring. I thought it was going to be, like, very simple, basically,
and I was just going to, like, completely tune out and be bored. But, um, no, it’s
definitely – the things that they talk about are very, um – they treat you like an
adult, you know. They’re not, like, molly-coddling or anything, they’re actually
giving you the information that you need. (YP4 Page 9.)
So, my twin brother’s quite excellent at learning an instrument, and I wanted to
learn an instrument as well, but I wasn’t as good as him, so I felt, like,
discouraged…. I felt like I wasn’t good enough to play music. But really, all you need
to do is just have a growth mindset and say, you know, well, maybe if I just practice
for 15 minutes a day I’ll get better, you know. I don’t have to be as good as other
people in order to, like, validate that. .......I’ve actually taken up playing bass guitar,
which is – well, something I wanted to do for a while. (YP4 Page 6.)
Workers also reported that young people were applying the skills outside of classroom settings.
So, I’ve got anecdotal evidence that young people have tried the things that we’ve
talked about in class because I think we have, you know, spent a good amount of
time deconstructing some ordinary, universal problems, and what you can do, and
how you can react to things. So, I think that’s great. (M2 Page 12.)
2.2.1 Self-awareness Improved self-awareness which resulted in more appropriate responses by young people was
commented on by both young people and workers.
And being able to actually know how to take my mind away from everything that’s
going on and be able to just focus on myself, focus on what’s going on, focus on
what I need to do for me, is huge. And it’s definitely helped outside of school as
well. (YP5 Page 4.)
I remember one of them… because he … was experiencing quite a lot of depression
and anxiety, and he said that he was using the different tools, especially, like, the
gratitude and grounding kind of stuff, just to help him keep perspective, and that
it was helping him with his, like, positive thinking, and, like, giving him more tools
2 Throughout this report, page numbers at the end of the quotes refer to the transcript of the interview.
18
to be able to deal with the intense emotions that he was feeling at the time. (S14
Page 6)
Also, there’s more self-awareness, you know, about how they’re experiencing
emotional states, where they’re experiencing it, what they can do about it. (M2
Page 12.)
There was some evidence of higher order thinking by some of the young people. This was important
as it demonstrated the ability to reflect and transfer learnings for use it at a later date, when needed.
There was also evidence of application of resilience skills in young people’s own lives which
demonstrated evidence of portable learning.
Another young person, he’s doing another course in which they’re also doing some
resilience skills, and he says 'oh, you know, you’re doing this for this…….', he's
putting things in together. They’re drawing consciously – they’re saying 'oh, you’re
teaching this because you want us to be able to do this and that'. It’s that higher
order thinking, isn’t it, where they they’re linking up things to their own life? (M2
Page 12.)
In some cases, self-awareness has been about learning about the brain:
What I’ve noticed in the young people, which I think is fantastic in my classes, is
that they’re actually developing an understanding of the brain, that basic
neuroscience around being in an emotional state and how you can’t learn in an
emotional state, whereas hooking in and using your frontal cortex, I’m actually
starting to notice the language starting to come up a bit more…they love to know
what’s going on in their own world, like, why they behave the way they do.......and
why there’s so many differences. (S5 Pages 20/21.)
Some staff reported that improvements in self-awareness were related to attitudinal change for some
young people in how they related to each other and to staff.
There was a lot more self-awareness. I think they were much more aware of each
other and the way they were treating each other, which was really nice. (S1 Page
6.)
They [the young people] have developed greater self-awareness. I think they’re
connected in some really positive relationships here, with each other and with staff.
(S6 Page 11)
There was also some evidence of a shift in how some young people understood their relationships as
a result of the project. They were able to understand more about the connection they had with their
friends and family members, and the role these different relationships played in their lives. The
project provided them with strategies for identifying and better managing them, in a way that was
positive for the young person.
…they’ve sort of thought about their relationships that they have ... (and) they’ve
actually started to be able to categorize some of the relationships that they have
with their friends. So, things like, you know, being able to recognise if a friend’s just
sort of a party friend, and a friend that’s actually got some depth and will be
supportive. And so, I guess learning how to manage those relationships. So not
19
necessarily breaking away from those relationships but learning how to manage
them, and who’s a really good influence for them, and – and who’s perhaps not so.
(S3 Page 13.)
2.2.2 Self-regulation Along with improvements in self-awareness some respondents reported that there had been
improvements in self-regulation. This is an important outcome as it suggest that at least some of the
young people involved were able to apply skills learned through the programme while in stressful
situations.
So, whenever a problem sort of happens I just think well what will happen if I react
badly? It sort of stops me, for the moment. (YP 8 Page 3.)
[Event Thought Response]: .... usually it’s when something bad happens, like maybe
I’ve gotten into an argument with my brother, and, like, I wanted to hit him, but I
didn’t. That’s, like, an event/thought/reaction. It’s kind of like, just, breathe. (YP 4
Page 13)
For some staff, self-regulation was the goal. Many of the young people engaged in this project lived in
difficult life settings and the staff focused the resilience work on improving their coping skills and their
ability to self-regulate.
It’s been my goal from the beginning to really help people with, like, managing
their emotions…So, we have quite a complex group of people here, and their family
lives are pretty, you know, complex, devastating sometimes. …So, I just want to
build capacity in the young people to be able to manage some things on their own.
I mean, obviously there’s crisis that they need help with, but you know, everyday
problems, everyday dramas, typical teenage sort of angst, that type of stuff…, I
think the young people are able to self-regulate a lot more now. (M2 Page 12.)
2.2.3 Confidence and positivity Particular skills taught through the project, such as gratitude, and particular approaches used by staff,
such as strengths-based approaches, were credited with helping young people to see the positives in
themselves and their situations.
I think that they’ve been able to see that, like, with gratitude...they’ve been able to
see that in amongst all of the doom and gloom, there’s actually some positivity,
and there’s a way out….. (S3 Page 13.)
We still work with a high-profile young man who’s looking at quite a lengthy
sentence for a high-profile crime in that group and, regardless of how long he’s
looking at, when you talk to him about redoing his strengths survey again he really
lights up around it. So he might be facing, you know, years and years and years,
and he’s quite affected by what’s happened, but as soon as you kind of mention
that there is this other side to him, that you do have some strengths and you
actually are – you know, you’re not this particular action, he still lights up at it, and
he still finds meaning in that, regardless of where he is now. (S7 page 11.)
20
Young people who became more confident also gained improved communication skills, which was
often demonstrated in class or group settings.
Another outcome is improved communication skills. I notice, because I’ve done this
two years now (sic), it’s my second year, they start off – the younger ones start off
very quiet in the beginning, and I’m doing a lot of the talking, but by term two they
can all contribute. You know, so I can see they’ve got increased ability to speak up
and say their own opinions, talk about stuff that’s meaningful to them in class, and
even they all read out loud in class. (M2 Page 14.)
However, one staff member noted that those attending specialised units because of anxiety and
depression found gratitude a difficult skill to master.
They do struggle with gratitude, that’s probably the one they struggle with most,
because they really can’t think of anything they’ve got to be grateful for, which is
really sad. (M1 Page 3.)
2.3 Educational aspirations The initial programme theory suggested that increased resilience would contribute to young people
staying in education and achieving better outcomes. The evaluation sought to investigate this, asking
“In what ways are young people’s educational aspirations and achievements affected, in what
contexts, how & why?”
This evaluation could not investigate differences in educational retention per se: to do so would
require a different design. However, throughout the interview process it became clear that for the
disengaged young people their educational aspirations and achievements were not necessarily all
about completing Year 12. It was suggested that in some cases, it was about just making it to school.
Two respondents explicitly linked Resilient Futures to improved education outcomes. One interviewee
reflected on how the Resilient Futures project led to a group of students persevering until they passed
and were awarded their learner drivers’ licences. Another reported that improved academic outcomes
flowed from positive relationships and a sense of safety (see also 2.3.2 below).
I feel like this has to be related, but.....when young people are in care a lot of them
don’t have the same opportunities for things as – as mainstream kids. But those
four young people, in that resilience – RF10 programme that we ran, they all got
their L’s, every single one of them. They all did it, they all got it. It might have taken
some of them, you know, twice, or three times, but they still did it. Which to me – I
feel like it has to be connected. I feel like it’s helped with their self-belief, their self-
talk, I feel like it’s helped the young people become more mindful and helped them
with their relationships around them and their ability to keep trying at those things.
(S11 Page 20.)
Yes, we have academic outcomes. Last year we had people passing PLP, SACE 1
maths ....... essential English and basic maths, we do all of that, but you won’t get
the academic stuff out of them unless you start with that safety and sense of
belonging, because they won’t try. These kids will not try because they have the
fear of failure, and then they cannot forgive themselves when they screw
something up. So, … we want to unpack that. (S8 Page 12.)
21
However, there was evidence about how young people could see a more positive future for
themselves. Staff emphasised perceived relevance for the young people, now and into the future. If
they felt that the topic was relevant, then they would engage more in it, leading to better outcomes.
Relevance is the key. Like any subject, it doesn’t matter what you’re teaching, if
they don’t see any relevance, how is this helping me, what am I going to get out of
this, it just doesn’t sink in, they just don’t absorb it and they’re just not interested.
Which I think again links in to why we delivered it the way we did. We always
……work in collaboration with the young people to develop programmes, like, what
do you want? The kids can see this just isn’t nonsense, there’s a purpose, and that’s
what worked really well with the Resilient Futures……. it was all relevant, real world
situations, which was good. (S1 Page 9.)
And so, kids understanding that positive emotions, the engagement, the meaning
of that stuff, and knowing that the accomplishment doesn’t need to be finishing
year 12, it could be you got out of bed today…… (S8 Page 8.)
Wellbeing is an outcome that’s maybe not even attainable at this stage. That’s a
long-term goal, and what sits behind that, we can’t even list how many things there
are there. You know, it might be emotional regulation, if you need it. It might be
rocking up to school, if you need that. It might be finishing your year 12, you know,
there’s such a broad spectrum, that we called it sort of BYO (Bring Your Own)
outcomes. (WRC 2 Page 9.)
The journey to outcomes in educational aspirations and achievements was seen as a long-term one
and it was highly likely that, after one semester of exposure to the programme, it was too early to say.
However, in keeping with the programme theory model, staff reported an increase in self-awareness,
of learning and practising resilience skills and young people experiencing some positive outcomes.
While there were few reported educational outcomes, several staff members had thoughts about the
capacities they would expect to see in the future. These were only theories and were not tested in
the evaluation.
I think they’ve all taken stuff on board, all of them. Even though they might not
necessarily be able to articulate it I think they’ve all got something out of it, and
they’re all aware, and they’re certainly aware of all the teachers’ opinions up here
about how they regard it, and because they have a good relationship with us that
is probably one of the major things they can take out of that. When they hear it
again outside of this context, they might actually think ‘hang on, I know someone
that I trusted who thought this was reasonably good, who thought this was a good
idea’, and so they might be ready to listen to it more. (S2 Page 16.)
We teach them all this stuff, but when they finally get into a situation where they
see the relevance and how they can use it, I think that’s where the outcome is.
That’s what the greatest outcome has been is that they can use it in a relevant way
in their lives… (S5 Page 9.)
Motivated to have a good life, and to want more for themselves……you know, we’re
talking about cross-generational poverty, and parents who don’t work, and have
been on Centrelink since they were born, and that’s all they know and all they’ve
22
seen. So probably having that drive and that motivation to want more, and know
that they can achieve it, because no one’s told them they can achieve it, ever, which
is really sad, but that’s the reality. (S1 Page 8.)
That’s something that’s going to stick with them for life. If they know how to focus
and become mindful in their work, they’re going to be able to take that, and that’s
something that employers really look for, especially with the mobile phone
age…….If you can’t stop yourself being distracted by your mobile phone, that is one
of the biggest things that all employers look at and go ‘he’s always on his phone’.
Boom, you’re out the door. And it’s like, if you can do it now in this sort of
environment, when you hit the workforce you’re so far ahead of the field that it’s
not funny. (S2 Page 6/7.)
For long term outcomes, ongoing exposure to and practice with the Resilient Futures principles was
seen as a vital element, and the need for a two-part process of explicit and implicit learning through
group settings as well as one-on-one embedding in their own life journey.
I think just delivering it in an eight-week group programme is not enough. It’s never
going to be enough. I mean, we’ve seen it with our staff, just going to the train-the-
trainer material is not just enough. There needs to be more of the ongoing learning,
there’s ongoing embedding, there’s ongoing trying, and you know, tweaking, and,
I think for those young people we delivered it to in the (organisation name) who
got maybe two or three sessions, they got the material, they understood it, but did
they continue to go and use it if they didn’t have the one-on-one mentoring?
Probably not. But those who get it every day in the classroom or who had a case
manager who’s continuing to use the language with them, they get it. So, it does
make a difference and so I guess the way that SAHMRI had initially envisaged the
programme, where.......the programme was to raise awareness, the mentoring
was to embed the material, I think that’s absolutely the case. We’ve certainly seen
that as the case. (M3 Page 14.)
While interviews did not provide direct evidence of a shift in the young people’s educational
aspirations and achievements, the longer-term outcome may be less about educational aspirations
and more about motivation and aspirations in a more general sense.
2.4 Passing it forward There was a little evidence that some young people were not only capable of making changes
themselves but also of teaching some aspects of the project to other young people who had not been
involved with the project.
And even with the other teenagers in my house, ‘cause we’re in a youth shelter, so
everyone in there is under 18 and there’s some pretty immature people in the house
as well that are very much the same. React in ways to silly little things that don’t
even need a reaction. And I … just say, “You know who you are. You know you’re
not whatever that person has said you are, or you know you’re not doing whatever
that person said you’re doing, so why let it bother you?” One of my closest friends
in the house he’s starting to really take note of it and not actually taking – there’s
still times where he’s struggling. ‘Cause you don’t learn things overnight. But he’s
seriously trying, and he’s seen a brilliant outcome from it as well. (YP5 Page 8.)
23
2.5 Sustainability of outcomes The extent to which these outcomes continued to play out for the young people after they had
completed their time in the educational and other settings is unknown, as no young people who had
exited the project were available for interview. However, a couple of examples of longer term
outcomes of the programme were provided through interviews with staff.
I caught up with (a former client) the other week, and we actually reflected on the
Resilient Futures project as well, and she talked about how it she learnt a lot from
it, and she’s doing quite well. She uses a lot of goal setting. (M4 Page 12.)
Another worker talked about an 18-year-old who had left residential care who had come back and
visited the service. While not reporting sustained use of learnings, the programme apparently had
positive associations.
He remembered the room, and associated the room and stuff with it, and I think
that’s really cool that he attaches that to a good thing. You wouldn’t think that
someone in care who had so much going on would do a programme like that
[Resilient Futures], and it be associated with a really positive memory. (M1 Page
12.)
2.6 For whom? Realist evaluation expects that programmes will be more effective for some people than others, and
seeks to explain how and why that is the case. The question for this component of the report is,
therefore, “For what sub-groups of young people is Resilient Futures effective, how and why?”
A number of participant sub-groups were identified through the interview process. These sub-
groups had different levels of engagement and outcomes in resilience skills. The main five sub-
groups of young people were:
• the Curious-and-Open-to-Learning Group
• the Positive Psychology Thinking is already Positively-Part-of-their-Life-and-Family Group
• the Previous-Positive-Interaction-with-Resilient-Training Group
• the Relatively-Settled Group, and
• the Currently-Ready-to-Change-Their-Lives Group.
The first group were those that were curious and interested in learning. In some cases, where the
young person had a level of self-awareness about their personal issues, they were more open to
learning resilience skills and applying it to their lives. For those with lower awareness, progress was
slower.
It works really well with the young people that…..are a bit open to it, are a bit more
self-aware of the fact that they have anxiety, or they have, you know, stress
triggers…..and it takes a much longer time for young people who are suffering, you
know, anxiety and depression, things like that, but aren’t actually fully aware of
that quite yet. (S5 Page 10.)
The second group consisted of those young people who came from a background where positive
psychology language was already part of their dialogue. The important factor here was the powerful
impact of positive reinforcement from other key players in the young person’s life.
24
My mum…..was into things like this before I started doing it so it helped a bit more
as well. (YP3 Page 9.)
I know when I first started at (name of agency) and they were introducing all this
stuff, I thought it was a bit silly, and kind of like kindergarten-ish, and…because it
was stuff I already kind of knew, so I was, like, I already know this, why are they
trying to teach it? But once they started explaining it more, and you’re kind of at
school with it, so you’re kind of stuck with it, and you’ve just got to accept it a bit,
I was kind of like oh yeah, this isn’t that bad, actually, and I could learn some stuff.
(YP3 Page 12.)
And then when I talk to families, because I work with families and the parents, not
just the kids……I can say that to them, you know, this is what I think works for me
at home, I do this, and I share because that works too, is it’s a sense of belonging.
We’re all in this together. I’m a parent, I know how hard it is, try this, you know,
and they’re more inclined to do it, give it a crack. (S8 Page 23.)
The third sub-group was the previous-positive-interaction-with-resilience-training group. These were
young people who had experienced resilience work before and in a positive setting. Upon being
introduced to the themes in the project, positive memories were triggered, and they engaged.
The fourth sub-group was the relatively-settled group. Young people from less traumatised
backgrounds tended to have more support from home and community and were more able to absorb
the project material.
They could have more support at home......they could also be more engaged in the
community.....and, if they’re engaging with people in the community and they’re
......building positive relationships and positive emotion, (then) I think they’re in a
better place to absorb what it is. They’re in perhaps a more positive frame of mind
to absorb what it is that we’re saying. (S6 Page 12.)
This group included young people who were more settled in life, attending school, and were readier
and more open to accept the project.
If they’re actually still going to school and doing a lot of things, they might be a bit
more up here, ready to accept that stuff, but if they’ve come in and they’re really
been moved around, don’t know what’s going on, there’s other kind of things going
on in their life, it’s probably not the best time to do it. (S10 Page 19.)
So, what we found with delivering the material in the (agency name) is that
actually when we had young people who had all their needs met by being in the
(agency name), they were actually usually willing to take on some part of the
material or hear some part of it and, in particular, the part that we found they liked
the most was things like the character strengths..... because, I think, it’s such a nice
conversation to be able to have with them…(M3 Page 10.)
The fifth sub-group of young people included those who were ready to engage in order to change their
life’s trajectory.
25
I think it was identified that children who were moving from residential care into
independent living were sort of in a space where they could take this sort of stuff
on-board in that capacity. (S9 Page 2.)
While evidence was found to support all of these groups, there was also evidence to demonstrate
the opposite; groups of young people for whom Resilient Futures did not work. These included
young people with a fixed mindset who were not open to change; those who came from a family
that did not know about nor use Resilient Futures language, or actively opposed it; those who had
experienced negative previous interactions with Resilient Futures language; young people in crisis;
and young people not ready to engage in order to change their life trajectory.
The other side of the first subgroup of Curious and Open to Learning were those young people with
a fixed mindset.
They just don’t think that learning these skills will help them, because they have a
fixed mindset, they just believe that what they have…… is what they have, they
can’t see that there’s more learning involved, more discovering yourself in that
situation. So, I just believe most people just tend go to the negative based on good
experiences, bad experiences, and just stay the same, don’t want to grow as a
person or as an individual. (YP2 Page 6.)
The second subgroup for whom it didn’t work came from families that didn’t have those skills and
abilities. One staff member voiced their hypothesis about how this could work for those families not
supportive of the project.
It’s almost as if we need to be working with the parents to reinforce it. ...I think too,
you know, there would be some that [we] would … potentially talk about it and just
be talked down. ‘Oh, that’s rubbish’, you know, so it undoes what we’re trying to
do here. (S6 Page 17.)
The third sub-group for whom it didn’t work came from a background where the young people had
previously learnt inaccuracies about resilience skills, or associated a negative experience with
resiliency approaches. The negative impression of resilience skills led the young person to have no
faith in it and shut down when it was mentioned.
Like, growth mindset tends to be, ‘oh, I’m going to succeed, I’m just going to be
awesome at everything’, that tends to be what they come out with, not the fact
they’ve got to try, that effort is what you’re really looking at, and then you’ll grow
from effort……..with these kids here, once they have a bad relationship with a
teacher, they’ll already understand what’s going on, and it’s reinforced by external
views from home, maybe, as well... Like ‘all this positive education is just think
happy thoughts’. And you get that everywhere, like I don’t just get that through
necessarily these kids, students, I also get it through people that I meet outside of
here that I see, ‘oh, this positive education, it’s all just about, you know, giving them
an A when they don’t deserve it’, and all this type of stuff. And that’s the view
they’ve got, and that has nothing to do with necessarily what they’ve read or what
they’ve heard, but it’s got to do with the slant they’ve put on it. (S2 Page 12.)
When you people had people had previously been exposed to resilience language but attached it to a
negative memory, the staff found that they were starting “below zero” with trying to engage young
26
people. One staff member likened it to trying to get children to eat vegetables by making cauliflower
rice and passing it off as rice.
I think the minute you try to introduce – these students have heard this stuff fifty
thousand times prior to coming through the door, and I think…because you’re
working with a highly disengaged group, they’ve been to counsellors, they’ve been
to psychologists…they’ve been put into alternative learning projects where this is
always a key focus, and I think when they hear it again they’re, like, ‘we’re not
doing it, we don’t want to do it, we’ve done it, we don’t get it, we don’t like it’. And
that’s the honest truth........Whereas if you sort of – I said it’s sort of like smooshing
[sic] vegetables into food, do you know what I mean? (Like Cauliflower Rice). (S1
Page 4.)
A lot of the kids don’t even want to talk to a psychologist. They don’t want to talk
to a counsellor. They’ve talked about their feelings and their stories so many times.
Some young people like talking about their story over and over and over again, but
once you mention the word psychologist or counsellor, they’re out, they don’t want
to do it, because it brings up so much stuff for them, and so I think that these skills
are a good way of, you know, helping their wellbeing and their emotional
regulation without them realising actually what you’re doing. (S11 Page 14.)
When they’re coming here with multiple barriers, they’ve been through CAMHS,
some of them are now with Headspace, they’ve been through psychologists,
psychiatrists, doctors, all these people have wanted to work with them, and they
just get turned off, you know, ‘I’m not doing this again’. Like, as soon as it’s
anything to do with mental health they’re just not ready to do it again. I’m not
saying it wasn’t valuable, but just that concept to them was overkill. (M1 Page 2)
For students with these sorts of responses to the programme content, staff found other ways to work:
these are discussed further in section 4 below.
An important reflection on the evaluation was that one of the Resilient Future surveys included a well-
used depression and anxiety scale (the K10 scale) which some of the young people recognised. The
teacher had been unaware that the survey included this scale.
We did a survey that SAHMRI initially gave us as well, which I didn’t realise, but for
a lot of my kids it was a diagnostic tool that psychiatrists use. That turned them off
as well, because they recognised that that was what it was. (M1 Page 2.)
The fourth group included young people involved in the Resilient Futures Project who were in crisis
and where there was the need for staff to adapt to ‘whatever walked through the door that morning’.
This group is discussed further in section 2.6.1.
The fifth group contained young people who were just not ready or open to changing their lives.
For those who live in the moment, so a lot of the boys who are in (agency name),
they aren’t forward planners, it doesn’t work so well, because they’re not
interested in changing yet, or not interested in doing any hard work or growing in
that way yet, because they’re just not – their brains aren’t mature enough. (S7
Page 14.)
27
We’ve got kids who come in at 16, so it’s very hard to sort of get that independent
living skills and positive psychology when somebody probably has never had to
brush their teeth before, or something like that. (S10 Page 18.)
I’m going to be completely honest with you here, I’ve been working with this young
person for two years, there’s – there’s nothing you can do. You can’t force them to
do something that they don’t want to do. (S11 Page 18.)
There are some young people we knew just were not going to be even interested in
having these conversations, and we were fine with that. So that was usually young
people who were maybe into really full-on offending, with chaotic lifestyles, not
actively engaged with our services, not really looking to be actually engaged with
any service, just not ready, I guess. (M3 Page 10.)
There is a young man for whom it isn't working ......he’s kicking back because it’s
hard to hear. It’s hard to hear that this is something you can control – that’s the
thing, is switching from an external locus of control to an internal locus of control,
because you need to understand the things that you can control .......otherwise you
will never ever actually be happy or have any kind of wellbeing. S8 Page 13.
In the residential care area, however, two further groups were identified. On the one hand there were
those young people about to transition from Residential Care to Independent Living for whom there
was a sense that life was about to get ‘real’, and that they needed to learn skills, and quickly.
…generally, 16, 17 is when they start thinking 'right, I need to do something for
myself, I need to start learning a lot of life skills, because essentially, I’m going to
be on my own in a short period of time', because they are in a space where they
can take on that new learning. (S9 Page 2.)
However, another staff member talked about how there was no incentive for young people in
residential care to learn life skills:
That’s always been a big sticking point for the kids who are 16, 17, ready to move
on, but don’t want to engage in any of the independent living skills. And why would
you, if someone’s going to cook and drive you, well I’ll probably take that as well,
if that’s what’s happening. So, then they get to 18 and we’re, like, see you later,
and relationship ends, and they don’t know what to do anymore. They didn’t go to
school, so they don’t usually have a good friendship group, they haven’t got a job
because they haven’t finished school, like, there’s all these things, and then they’re
there by themselves as well. (S10 Page 13.)
This group of young people would then be at risk in terms of maintaining their accommodation:
And all the literature says that’s actually the most critical time of all, just those first
six months out (from residential care), that’s when kids lose their home… (S9 Page
13.)
These comments appear to identify two sub-groups of young people leaving residential care: one
group of young people who were more independent, positive and confident and another group who
appeared more dependent and less confident.
28
Another group for whom the Resilient Futures Project was not seen as effective was those young in
age and/or young developmentally:
I think you’d have to sit down and modify the project for the real little ones as
well......because there was a lot of not ‘adult’ concepts, but higher-level thinking.
And I’ve got young ones – not only are they young, but they’ve also got
developmental delays, intellectual disabilities, and that goes for older kids as well,
that they’re not always a 16-year-old, chronologically, but probably not
developmentally. (S10 Page 20.)
Some of the skills are really focused, which is tricky for some of our kids because of
their developmental trauma, and the developmental ages as well…..(So, the staff
selected some of the easier to grasp Resilient Futures skills)…So, then we kind of
thought these are really easy for them to understand and grasp, and we could see
them wanting to learn more about it, and using it, so we’ve taken with those to
implement just some day-to-day stuff. So, we have a lot of gratitude journals…and
the fixed and growth mindset, that’s something that’s really widely known now
because of the way that it’s gone through schools. (M4 Page 7.)
We always knew going into it that there were going to be a group of young people
who cognitively just were not ready to take on the material, and we were happy to
accept that. It’s the same with any of our work. (M3 Page 10.)
One respondent suggested that they had found classroom settings did not work as well for the
younger students as the older students and that the opposite was true for case management.
When we were doing it in its original format or even its first adaption of the format,
pretty much if you’re 16 or lower we’ve seen a real difference they were just getting
out their phones and rah, rah, rah. … So I can definitely answer that because they
didn’t have the maturity to look at it from that open mindset. Doing it in one-to-
one, we’re seeing a bit of a flip actually, we’re seeing, getting some of the most
benefits with some of the younger students but because they’re able to do it at
their level and their time, and their way at their intensity… (M5 Page 22.)
Staff in youth services also discussed the difficulties of introducing the Resilient Futures Project to
younger teenagers, especially those with high levels of risk-taking behaviour.
High school is where things do get pretty wobbly for our children, so if they had
those skills prior to that wobbly phase, I think that would be ideal, but it might not
be the ideal time to be adding more to their load. I think that’s sort of where they
are sort of learning their space in the world, developing a sense of identity, there’s
loads of risk-taking behaviours, quite often they’re not at the (residential) house as
much as we would like them to be, absconding, which makes it really difficult to
put these sorts of interventions in place. Um, and during that time there’s just a lot
of other pulls that are a lot stronger than us, and once they’ve hit about 16 is when
they start going okay, you guys aren’t so bad, and I have got a lot that I need to
learn from you guys. (S9 Page 3.)
I am working with a group of boys and there’s a lot of absconding, so they’re not
at the house a lot. When they are they’re under the influence of drugs and all sorts.
29
They’re quite dysregulated a lot of the time, so unable to sit and tolerate
conversation for any more than 10 minutes, and generally when I’ve got that 10-
minute window I’m talking about court appointments, or anything like that. (S9
Page 6.)
Some of the findings from this section suggest opportunities for further work in developing resilience
training for young people. It suggests that introducing resilience skills at a younger age (say, 10 to 13-
year olds, before the early to mid-teenage years), might provide them with skills to better handle
those years. However, those younger people would, because of their stage of development require
new, less ‘cognitively abstract’ ways of working to develop the capacities of resilience.
Some workers suggested that the project was more likely to work for females than males.
…girls are heaps more open to it. It makes sense. And boys are more into......you
know, thinking about their next bong, or you know, driving the car on the weekend
........ boys aren’t as into learning as girls are… (M7 Page 17.)
This contradicted the quantitative data which suggested that the project had had a stronger impact for males than for females. There are a number of potential reasons for such a contradiction between the surveys completed by the young people and the reflections from staff. Girls, for example, may be more articulate, so that staff think they’ve moved further along in the resilience work, or it may be that the quantitative data is incomplete and therefore unrepresentative.
There were suggestions in multiple interviews that the project was more effective for more mature, generally older young people; and there was a trend in some of the quantitative data (excluding impacts on nervousness) for the programme to be less effective for the 16-year-olds. However, the quantitative data showed few statistically significant differences based on the young people’s ages and there was no clear pattern of greater change for different age groups. It may be that the real issue is not age but developmental maturity.
2.6.1 Young people in crisis or who have experienced trauma As noted earlier, young people who either were in crisis or had experienced trauma required workers
to adapt how they provided resilience skills development with young people. The adaptations ranged
from not teaching the class at all and responding with something else, through to using a relevant
Resilient Futures skill to respond to that particular crisis. It is important to note here that there is a
difference between trying to teach new skills to someone in crisis and using skills which have
previously been taught to help manage a crisis: if skills had already been learned, they could be used
(albeit with support); if not, a crisis was an inappropriate time to introduce new material.
I find that every time I’ve planned a lesson, especially around something like that
(Resilient Futures) as well, that I have to chuck it out the window pretty quickly
because it depends on what happens the night before (to the young person). …..if
there’s something going on, like, the lesson structure has to adapt very quickly. So,
to follow those steps, procedural steps becomes a bit harder. (S2 Page 15.)
I guess the thing with our students here is they’ve had so many barriers in their
lives that we have to be a little bit strategic in how we teach them to be positive
about their lives, because they don’t necessarily see that positivity straight away.
And so, what we did with the ETR, we found that we had to actually teach that on
the fly. So, when a student was actually having an issue straight away, that’s when
we were able to actually put it into practice. So, in that regard, it was really
authentic … .but we found that difficult to teach to the masses. (S3 Page 9.)
30
This sub group also included the most seriously disadvantaged young people, facing issues such as
homelessness or lack of food, or those from traumatic backgrounds with ongoing stress. These could
be contrasted with those in school settings still living at home but suffering with anxiety and
depression. Some respondents suggested that the Resilient Futures Project was not as effective for
the groups of young people from traumatic backgrounds, and introduced Maslow’s Hierarchy3 in order
to explain why the Resilient Futures Project had not worked with this sub-group.
So I would probably say those who come from really traumatic backgrounds it
potentially doesn’t work as well.......whereas some of those who don’t, you know,
who may have anxiety, who may have depression, who may have been bullied and
don’t have that historical trauma, that perhaps doesn’t allow them to engage with
it as much as those people who don’t.......perhaps they have some interference, you
know, in their learning, they have some developmental delays potentially, because
of the trauma, and so they’re not able to engage with it at the level that we’re
pitching it, potentially. (S6 Page 12.)
Maslow’s hierarchy - because if those needs aren’t met first, you know, you’ve got
no hope with the others, because they’re just the priority, survival. (S6 Page 15.)
One of the areas we didn’t try it in, that we really would have liked to, was in our
homelessness services. The staff were keen to hear about it, but it’s such a crisis
driven space that ……when you have young people coming in who could be, you
know, experiencing psychosis, or, just looking for somewhere safe to stay that
night, it was a really hard space to start thinking about getting staff to bring in
some of this material……..I think if you had staff who really understood the
material, they would find opportunities to bring it into their daily work., (but) we
just didn’t have the opportunity or the time to go down that path, unfortunately.
(M3 Page 10-11.)
I think that maybe some of the young people who have got more extreme
circumstances, I think it might be hard for them. There’s the operating out of the
amygdala………so that’s what we learn from the respond rather than react training,
that people have got to feel safe in their environment before they can engage
frontal lobe. So, I think some of those young people have such a lot of crisis, and
instability in their life, that they just can’t focus on this. Amygdala: that sort of
emotional centre of the brain where the flight or fight area is, and all that. So,
because of trauma, they’re not able to operate from the frontal lobe. (M2 Page 15.)
Amongst all the interviewees there was one teacher who stood out against all of the others and was
very vocal in saying that young people in absolute crisis stage could still benefit from resilience skills
education. She did not agree that basic needs had to be met before these skills could be introduced
to the young person.
3 Maslow's hierarchy of needs is a motivational theory in psychology comprising a five-tier model of human needs, often depicted as hierarchical levels within a pyramid. Needs lower down in the hierarchy must be satisfied before individuals can attend to needs higher up. From the bottom of the hierarchy upwards, the needs are: physiological (food, water, warmth, rest), safety and security, love and belonging (intimate relationships and friends), esteem and self-actualization (achieving one’s full potential, including creative activities). Ref: https://www.simplypsychology.org/maslow.html
31
Bullshit. It’s about your communication. You cannot say this isn’t going to work for
a particular type of person. I get what they’re saying because, like, you know, CBT
and trauma-based doesn’t work, because you’re blaming the person’s behaviour
on their feelings, and that their feelings aren’t valid, and you should change them,
right. I get that. We’re not doing that here. What we’re saying is ‘here’s your brain
development, here’s what’s going on, …. what can you control, what would you like
to change, what can we worry about?’ Let’s narrow that shit down. If you’ve got
someone who’s in a homelessness situation, sit there with the stress bucket, write
it all out, right. Write everything down and then sort it into groups of who’s going
to help? ‘Can you sort this out, can you do that? Your brain is here, it’s still
developing. Who have you got in your life that’s got a full frontal lobe? Who have
you got that you can go to?’ Here’s a list. Writing a plan out. What do you do when
someone comes to you with suicidal ideation? You write a safety plan out. You
discuss it. You can discuss that with this underpinning, abso-freaking-lutely. (S8
Page 26.)
One WRC staff member suggested that it was necessary to distinguish between trying to teach new
skills during a crisis, and encouraging young people who are experiencing a crisis to draw on what they
already know to assist them to deal with it. The distinction probably also relates to different styles of
enabling learning. High levels of anxiety interfere with learning, and so traditional teaching is much
less likely to be effective when anxiety levels are high. However, supportive counselling methods can
be used during times of high anxiety to support development of a plan to address a crisis. This, along
with reflection on personal strengths, can contribute to reduction of anxiety and support learning of
skills to apply in future. This appears to be consistent with the second quote above, which refers to
using a resilience based approach as an underpinning – as distinct from overtly setting out to teach
resilience skills.
What was obvious throughout the interviews was the importance of the teaching style: teaching
traumatised young people had to be approached in a very different style than mainstream school
teaching.
I’ve got a young man who I’ve been working with for three and a half years, and I
try and use ETR [Event Thought Response] with him all the time. I had to wait for
his brain to catch up. … ‘Oh yeah, go teach this person ETR because they need some
self-regulation and some self-reflection around their behaviour’. Well, it’s not, you
know, not the most palatable message if you go down that route, whereas if you
talk to them about what are your goals, what is it that you want to achieve and
what’s standing in the way, that’s a different kind of conversation, I think... (S7
Page 5.) I just don’t think it’s as simple as here’s this skill, which is a lot easier to
explicitly teach in a school, but with our cohort we have to understand that they’re
in the classroom of life, and they’re probably more mature in those experiences
than some of the other people who might have been at school. (S7 Page 9.)
2.6.2 Summary This chapter started off listing five different contextual sub-groups of young people involved in this
project. These sub-groups included those young people who were ready and curious to learn; those
that already used and were supported in positive psychology by others, such as family members; those
who had a previous positive interaction with positive psychology; the relatively settled group; and the
32
ready to change their life group. Negative experiences within each of the contextual sub-groups were
also explored.
For many young people, a few of these contextual factors were applicable to them. So, for example,
there were young people who were relatively settled, ready to move on with their lives in a positive
way and who had also been exposed to positive psychology in a meaningful way earlier on in their
lives. And there were young people who were currently traumatised and who had had negative
experiences with positive psychology.
Synthesising across the qualitative and quantitative data, it appears that there may three main groups
of young people in terms of the extent to which project outcomes are achieved. The first group is
those more positive young people, who are widely supported to take resilience skills on board and
have the highest level of outcomes. This is followed by young people living with a level of depression
and anxiety, who still benefit but may be slower to reach intended outcomes and may need more
support. The third group are those young people who are more traumatised and/or less mature, less
confident and who are not ready nor able to connect with the work and do not achieve the resilience
skills outcomes. These are groups for whom new implementation strategies are required. Future
iterations of programme development could usefully address these groups. We return to this issue in
the Future Directions section of this report.
2.7 Generating outcomes: contexts and mechanisms In the next section of this chapter, we turn the focus to the implementation strategies, contexts and
mechanisms which were reported to contribute to generate positive outcomes for young people.
2.7.1 Authenticity Perhaps the most frequently mentioned mechanism for engaging young people with the material was
authenticity on the part of the teacher or youth worker. For many respondents, authenticity was
linked to whether the worker concerned used the resilience skills in their own lives.
It was suggested that many young people could ‘see through’ the staff member if they didn’t apply
positive psychology in their personal lives. When staff believed in the material they were teaching,
particularly when demonstrated through personal experiences, this increased credibility for the
young people, which increased their engagement with the programme being taught.
She [teacher] was really engaged in it because she believes it actually works,
personally. I reckon she strongly believes in it, like, one of those person (sic) that
really believes that doing the training is one thing, but, like, participating in it,
wanting to learn more about it is showing initiative, showing that she wants to
teach others because she believes in it herself, and she wants others to believe in
it, that works. (YP1 talking about a teacher. Page 1.)
I mean, we’re teenagers. A lot of them don’t know who they are, what they have
and haven’t done that is good and bad, and where they can improve on their lives
and I believe that having that familiar, comforting – ‘this is what is was like for me’,
it encourages children to open up a little bit more and be willing to accept these
different challenges that the programme provides. (YP5 talking about the use of
examples by case manager. Page 10.)
[Staff using their own life examples] 'that’s really good. That’s kind of what made
me think it’s actually genuine…...They need, like, passion behind it.' (YP3 Page 6.)
33
We need to be able to believe what they’re telling us. (YP3 Page 13.)
…it goes to show that everyone is human and even he's been through similar things
that I have. It just goes to show that you can come out better from it. (YP8 Page
7.)
Where staff had not applied the Resilient Futures teaching to their own lives it affected their ability to
teach it with meaning and for the young people to engage with it. Both young people and workers
were quick to explain that young people were able to spot authenticity and that they didn’t trust the
Resilient Futures process if a staff member didn’t live the Resilient Futures skills explicitly themselves.
Like, I’m not going to believe anyone who’s not doing the same thing as what
they’re trying to get me to do, you know, I’m not going to do that. (YP4 Page 16.)
Everyone uses it here, everyone has understanding of it, and once you understand
it you can then go and use it by yourself in any situation. It does take commitment,
I guess, like, you’ve got to be able to engage in it, and not just understand it but
use it as well, like, in any – in certain situations. (YP1 Page 3.)
Staff identified authenticity both in its presence and its absence.
And I think a lot of times there’s nothing for them to relate to, connect with, if we
just tell them this is how you can do this. When we talk to them about how I applied
it in my life, they also see that we are human, and we have failures, and how we
overcame our failures. So yeah, it becomes more real to them… (S15 Page 6.)
They can see that I truly believe it, and I’m so genuine and authentic about it. I feel
like these young people can see straight through you if you’re not. (S4 Page 14.)
I think you have to understand it completely first, and the way you understand it is
probably by doing – practising it yourself, so you’ve had a lot of experience with
that. Then you can teach it to someone else, … I think that definitely think that
helps, because I think otherwise you’ll be like school, which they don’t like, handing
out the sheet, and you do your work kind of thing, and they’ll just be, like – whereas
if you were actually doing it, showing them how it’s done, supporting them,
praising them, all that, they’ll – they’ll start doing it as well. Um, I’d like to think I
do some of this stuff naturally anyway, I hope so. (S10 Page 16.)
It is important that whoever’s talking about this stuff with the young people
actually practises it themselves, because that was one of the cornerstones of our
training, was to find examples in our life that we could share with whoever we were
teaching to, or working with, as a case manager. So, they called it small A – small
adversity, small A examples. So, you don’t talk about your wife of 30 years has left
you, but you might talk about you’ve had a little fight with your teenage daughter
on the way to school, or something. (M2 Page 20.)
One respondent noted that staff who did not practice it themselves also did not engage with teaching
the skills to the same extent.
(About other staff) They haven’t applied the material to their own lives, because
I’m a big believer (in it). They’re actually not engaging in the teaching part of it to
the same degree (S7 Page 13.)
34
Another respondent noted that some staff members who undertook the training but did not apply it
with young people had wanted to be provided with a more structured approach, similar to the first
iteration of the project. They also noted that where staff were not highly engaged in the project the
effectiveness of the project was stunted.
So, they did the training, they sat in the room, they were excited about it, walked
out and didn’t apply it. And then when we discussed it in team meetings around
applying it they wanted really stepped-out instructions of how to apply it. It doesn’t
work like that. So I can put together worksheets and lesson plans, which I do and I
am doing now more so because we want to be able to put together a programme
that we can say this is what we do.......I can do that, but unless you’ve got a staff
member that can look at that and take that on and then deliver that with their own
personality, you’re not going to get outcomes and you’re not going to get what you
need. End of story. (S8 Page 2.)
This same respondent clearly linked this shortcoming to staff not having volunteered to run the
programme.
And the staff that we ended up with were put in the programme because the
programme that they were in needed to get – like, cut some fat, and because they
were permanent staff, they couldn’t dismiss them, so they had to offer them
another role. So that’s really key, because you want people that want to be in these
roles, you want people that want to, like, spread this, like, you know, wildfire, so I
think that was the problem. (S8 Page 2.)
2.7.2 Relationship As well as workers’ commitment to use the Resilience Futures skills in their personal lives, the
relationship with, and the sincerity of, the staff was vital to the young people. A strong relationship
was important for the staff to start to elicit curiosity to learn from the young people.
A big part of our job is building trusting, safe relationships, and some of these kids
don’t have that, and I think that is key to actually getting in, so to speak. (S6 Page
16.)
I think for them to be able to connect with us in the first place, that sort of shows
that the relationship’s been strengthened initially, and then once we were able to
support them and give them strategies and techniques, it actually affects their life
more positively, so then they can come back to us then and go ‘that really worked,
what’s next? What else can I do?’ And so, once they start to have a few wins, then
they can actually – they’re interested to find out what else is out there. What other
ways can, you know, I guess help them make better choices. (S3 Page 26.)
It was also noted that relationships were not only important to building trust but also to increasing
young people’s curiosity and desire to learn. In settings where young people felt safe and supported
by the group, they felt more confident to ‘have a go’ and were less concerned about failure, and this
helped to bring out the person they wanted to be and improved academic outcomes.
All the teaching we do here, it’s the relationships. So, because we have students
who’ve been called disengaged, so they, for some reason they’ve disengaged from
their learning, for a variety of reasons. So, while they’re disengaged they’re not
switched on to learn. So, to get outcomes, you have to build the trust up, but also
35
try and develop curiosity again, so that they are curious to learn. It actually is not
that hard. It’s quite easy to do that. (M2 Page 16.)
These are kids that if I’d asked them when they first started with me to tell me, you
know, something about yourself, they’re blank, they can’t reflect, they don’t know
– they don’t have the language, they don’t have the skills, they don’t have the
confidence or the safety to be able to do that. So, I think that’s the outcome, is that
that safety brings out who they really want to be, and when you feel safe and you
feel like you’re part of a family, and we call it a (agency name) family ……That’s the
outcome… (S8, Page 12)
Knowing the students as well as believing the material enabled teachers to work effectively.
If I hadn’t had all that training and background, in the beginning of the year when
I sit there and everyone’s quiet in class, I might think ‘oh, this is not working, this is
a lead balloon, why am I flogging this? It’s not for the kids.’ But, because I had the
belief in it, I could get past that initial stage of where they might feel they’re not
sure what’s happening, or they feel shy to speak up or contribute, or to even think
about themselves and their emotions. It’s knowing the young person, believing in
it, setting it up in a way that supports them but not overdoing it. (M2 Page 17.)
Having the right kind of relationship meant that authenticity could also be demonstrated through role
modelling.
We just do it in our day-to-day interactions with the kids……we do model behaviour.
We try to show them respect and……. understanding and – and help them to see
that there are things to be grateful for in their daily life, even if they’re tiny and
trivial, and we’re constantly building those relationships and having those
conversations. And I think that’s probably the thing we do best. It – it still came
through in the learning programmes but reinforced through that relationship that
we had with the students. (S1 Page 6.)
One respondent indicated that they had a dual role as both a teacher and a youth worker and that
they worked differently in the two roles. Others blended the roles and used role modelling in both.
I had a split role …. modelling what Resilient Futures is all about (as a youth worker)
as opposed to just delivering the content of the programme (as a teacher). (S1 Page
1.)
The importance of modelling was also noted from other circumstances in which it had not been
demonstrated.
Over at the school we deliver a lot of positive education programmes, and it’s
compulsory. Positive education is just something they deliver weekly to students.
Some teachers think it is a completely load of BS, and it fails, because they’re not
demonstrating the behaviour, they’re not interacting with the students in a positive
way, or staff in a positive way, so how can the kids be expected to do that? (S1
Page 6.)
All of the agencies participating in the Resilient Futures Project focused on engagement of the young
people, where the building of safe and trusting relationships was of vital importance. Of the young
36
people who attended a unit within a school4, only one was unable to separate Resilient Futures from
the normal way that their school unit operated. However, the relationship-based approach and the
understanding it generated was (whether Resilient Futures or not) linked to higher attendance at
school.
They really seem to care a lot more than a lot of other teachers do. (Facilitator:
How does that make you feel?) Oh, so accepted, so accepted, and welcome, I think
is a good word. It’s uplifting, it’s encouraging, it’s encouraging, really encouraging.
I’m here more often because of it, which is always a start. I know that in coming
here I’m not going to be judged if it’s kind of obvious that I’m not in a great mood,
or anything like that. They’ll work with me, they’ll just do the decent thing, you
know. At other schools that I’ve been to, it’s very much just, you know, ‘leave that
at home, you’re here to do your work’, but they understand that it’s not as easy as
that. (YP1 Page 7.)
One agency manager also commented that they could not clearly distinguish Resilient Futures from
other work.
I don’t think I can separate what we do and the Resilient Futures Programme,
because I think they’re part of what we do, so I can’t therefore say. (M1 Page 7.)
Positive relationships were important for creating a safe environment for young people.
A lot of people come here with anxiety, and ... .after, like, going through the
process, and seeing how friendly it is, and everyone’s very welcoming here, it’s like
a very tight-knit community, you know, it’s almost like a family.…It’s a bit of like a
kinship thing, you know, like, everyone needs to have human contact and feel, like,
welcomed, and have your opinions be heard, and, like, feel like they’re valid, you
know. Um, like, people just don’t dismiss you. (YP4 Pages 1 and 4.)
In the school settings when young people felt that they had people that cared for them, and this had
been demonstrated through positive relationships, this triggered a sense of responsibility and led to
better behaviour, at least within the school grounds.
Our kids are very self-centred, often, and it’s all about me, and I am so hard done
by. And in reality, yeah, they are, but it’s that when you can get them to
understand that there is stuff to be grateful for, and there are people in their lives
that actually care, that’s probably the a-ha moment. But that’s all about that
positive relationships stuff as well, you know, it’s about when they get that people
are actually caring about them, and with that comes almost a sense of
responsibility about how you’re going to behave, or what you’re going to do. (M1
Page 9.)
4 The term school is used in this report, as distinct from other agencies that were involved in the Resilient Futures Project, such as youth detention centres, residential homes and other youth settings, but in all cases, these were small educational units within a campus of a larger school, specifically for disengaged young people.
37
2.7.3 Reinforcement As discussed in section 6.2.1, reinforcement was identified as being important to generating outcomes
for young people and this included receiving the same messages from multiple staff and repetition of
ideas and concepts.
…we knew from the beginning that you can teach these skills but if they’re not
reinforced it’s going to have minimal impact… (M2 Page 2.)
So we do that, and we do it over and over and over again, and we have
predictability and consistency. (S8 Page 15.)
2.7.4 Boundaries Clear boundaries were important in school units because young people were free to come and go and
the learning style was based on an adult education approach, which was different from the
mainstream school setting. In a few examples, at three different school settings, the teachers
reported that young people would sometimes arrive at school in a distressed state and these teachers
would change their planned curriculum and introduce a relevant resilience skill to respond to the
issue. Boundaries had to be set, however, to ensure that the curriculum was returned to and the work
was completed.
We’ve created an environment where they know we’re very patient and very
understanding, and really, really flexible, but if they push it, it’s all over, basically,
and they’re aware of that. (S1 Page 13.)
One young person commented on how this approach really helped her to study, as in mainstream
school her emotional state would have to be put to one side and the work done as planned, even
though she was too stressed to take anything in and learn.
2.8 Conclusion A number of outcomes for young people were identified in this chapter. Improvements in self-
awareness, self-regulation, confidence and positivity were all reported. These outcomes also show
that at least some of the young people involved in the programme were learning and implementing
skills encountered in the programme.
There was a lack of evidence to support the hypothesised outcome of improved educational
aspirations but there was some evidence suggesting that there were improved educational
outcomes, rather than educational aspirations, for some young people.
There are a number of key messages from this chapter. The first relates to the nature of the
relationship between the worker and the young people involved. Trust and authenticity were, as was
initially hypothesised, found to be critical to effectiveness.
The second relates to the expertise of staff in incorporating resilience skills in different ways, using
different language, for different young people and in different situations. As well as expertise,
however, support from management was required. In particular, provision of time for individuals
and groups to work together on developing and implementing programmes was necessary.
The third relates to the sub-groups of young people for whom the project, as currently conducted, is
more and less successful. Five sub-groups of young people were identified for whom the programme
was more likely to be effective. These were:
• the Curious-and-Open-to-Learning Group,
38
• the Positive Psychology Thinking is already Positively-Part-of-their-Life-and-Family Group,
• the Previous-Positive-Interaction-with-Resilient-Training Group,
• the Relatively-Settled Group, and
• the Currently-Ready-to-Change-Their-Lives Group.
The groups for whom the program was least likely to be successful, at least in its current form, were:
• those experiencing current crises;
• those with previous negative associations with positive psychology and its language;
• those who were younger or at an earlier stage of development, including those with more
concrete learning styles.
The quantitative data (see Chapter 3) suggested that the programme was most successful for those
who started from less positive emotional wellbeing positions. The qualitative data, however,
suggested that the programme was least successful for those who are most traumatised. It is
entirely possible that the most traumatised participants did not complete surveys, and that this
explains the difference. It is also possible that they are in fact different groups: those with ‘less
positive’ emotional wellbeing may be somewhere further along a continuum than those who are
‘most traumatised’. It is also possible, however, that workers in agencies have some misconceptions
about how and for whom the programme is working. Qualitative data suggested that basic needs
should be met, and young people should have established at least a basic sense of security before
resilience skills development is likely to be useful.
39
3 Quantitative Data
3.1 The Survey The survey included three sets of questions which used different answer scales, drawn from three
existing instruments.
PERMA stands for Positive Emotion, Engagement, Relationships, Meaning, Accomplishments, a
framework created by Martin Seligman. There are a number of PERMA based surveys. The PERMA
questions used in this survey were taken from the PERMA Profiler developed by Julie Butler and
Margaret Kern of the University of Pennsylvania. Five PERMA questions, relating to positive emotion,
meaning and accomplishment were used. These questions used an answer scale from 0-5 where 0
indicated ‘never’ and 5 indicated ‘always’.
The Kessler 10 (K10) survey is a 10-question survey designed to measure psychological distress. The
K10 questions consisted of 10 questions with an answer scale of 1 (none of the time) to 5 (all of the
time).
Three questions relating to life satisfaction from a United Kingdom Office for National Statistics (ONS)
survey were included and used an 11-point answer scale, from 0 (not at all) to 10 (completely).
It is important to note that the for the PERMA and ONS questions, which ask about ‘positive’ factors
such as happiness and satisfaction with personal relationships, a higher score would be considered
better. However, for the K10 instrument, which asks about ‘negative’ factors such as nervousness and
depression, lower scores would be considered better.
To form scale scores, each of the three sets of questions (PERMA, K10 and ONS) were averaged by
adding the scores for each question within the set and then dividing that by the number of questions
in the set. Any respondent who did not answer one of the questions was excluded from the analyses
of any scale for which they did not answer a question (but not from the analysis of individual
questions).
Another important issue in relation to the survey was the timing of collection. The first round of
surveys (Time 1, T1) was completed near the beginning of term 1 (February 2018) and the second
round of data (Time 2, T2) was completed toward the end of term 2 (June/July 2018). As the timing
and length of programme delivery was different in different sites and, in some sites, varied from
facilitator to facilitator, this cannot be construed as ‘pure’ pre-post data. Rather, it is indicative of
change over a period of approximately one semester of delivery.
3.2 Demographics In all 330 individuals responded to the survey in 2018. Of those, 117 responded only at T1 and 52
responded only at T2. There were 161 respondents for whom data was available at T1 and T2. In the
analyses that follow, only the data for those that responded at both time intervals was used. This
meant there was no concern as to whether apparent changes were due to different cohorts providing
data at different times.
The table below shows the number of respondents, the gender of the respondents and the average
age of the respondents. Respondents were also asked whether they were a parent and, if so, how
many children they had. Due to the very small number of respondents who were parents (n=2) no
further analysis was undertaken related to this.
Note that the total number of respondents does not add up to the total for male, female and other as
not all respondents answered the question about their sex.
40
In the analyses below, only three sites are represented. T1 data was available for a fourth site but no
T2 data was available for that site. All the sites for which there was quantitative data available were
educational settings. There were also four other sites, three linked to youth justice and one to
residential care, which were running the Project but from which data was not collected.
The four sites from which data was collected were listed in the research ethics application as part of
the Phase 1 roll out while the other four were not. Sites 4 (the site for which there was only pre-
programme data available) was running the programme for the first time in 2018 while the other three
had run it previously.
Table 1. Demographic Data
N. Male Female Other Age (Years)
Site 1 117 51 (45.1%) 60 (53.1%) 2 (1.8%) 17.8
Site 2 32 19 (61.3%) 11 (35.5%) 1 (3.2%) 17.0
Site 3 12 7 (63.6%) 4 (34.6%) 0 (0.0%) 15.2
Total 161 77 (49.7%) 75 (48.4%) 3 (1.9%) 17.5
3.3 Change over time
3.3.1 Scale scores Paired sample t-tests were conducted to determine whether there were any statistically significant
changes between T1 and T2.
There was a small, statistically significant increase in the PERMA scale score and a small, statistically
significant decrease in the K10 scale score (both of which would be considered positive results). When
t-tests were run by site, there was a similar, small statistically significant increase on the PERMA scale
score for Site 1 but not for Sites 2 or 3. As almost three quarters of respondents were from Site 1, this
is not surprising, as the larger number of respondents there makes it easier to achieve statistical
significance.
Even where statistical significance was not achieved, there was a positive trend for all sites using all
instruments. That is, there were higher scores at T2 than T1 in all sites for the PERMA and ONS scales.
There were lower scores at T2 than T1 for the K10 scale.
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
4.00
Entire Sample High initial PERMA Low initial PERMA
Change in Wellbeing (PERMA)
T1 T2
*
*
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
Entire Sample High initial K10 Low initial K10
Change in Psychological Distress (K10)
T1 T2
*
*
41
The sizes of the change were quite similar for the PERMA (which used a six-point scale) and K10 (which
used a five-point scale).
In the graphs below, columns shaded in grey indicate that there was not a statistically significant
difference for that group (site, age or gender as appropriate).
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Site 1 Site 2 Site 3 Total
Mea
n S
cale
Sco
re
PERMA Scale Scores by Site
T1
T2
1
1.5
2
2.5
3
3.5
4
4.5
5
Site 1 Site 2 Site 3 Total
Mea
n S
cale
Sco
re
K10 Scale Scores by Site
T1
T2
42
These results show that across the whole group there was a small improvement in both positive
emotions and a small decrease in anxiety and depression. There was, however, no statistically
significant difference on the ONS scale which also relates to positive emotions.
Paired sample t-tests were also run to see whether there were statistically significant differences when
the group was broken down by age or gender. For the purposes of these analyses, the respondents’
ages were grouped into four groups as shown in Table 5.
Table 5. Age Groups
Age N
16 years or less 43
17 years 42
18 years 33
19 years or more 37
There was one age group, 17 year-olds, for which there was a statistically significant differences over
time and one scale, PERMA, for which this was the case. The increase was larger than that for the
whole group or any of the other ages. There was no obvious pattern of change in relation to age as
the next largest change was for respondents who were 19 years of age or more.
The very small amount of change for the youngest group does seem to support the views from
interviewees about the Project being less effective for the youngest participants (see Chapter 2).
0
1
2
3
4
5
6
7
8
9
10
Site 1 Site 2 Site 3 Total
Mea
n S
cale
Sco
re
ONS Scale Score by Site
T1
T2
43
When broken down by gender, there were statistically significant changes for males on both the
PERMA and K10 scales. As with the whole group both changes were positive. There were no
statistically significant differences on scale scores for females.
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
16 or less 17 years 18 years 19 or more
Mea
n S
cale
Sco
re
PERMA Scale Score by Age
T1
T2
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Male Female
Mea
n S
cale
Sco
re
PERMA Scale Scores by Gender
T1
T2
1
1.5
2
2.5
3
3.5
4
4.5
5
Male Female
Mea
n S
cale
Sco
re
K10 Scale Scores by Gender
T1
T2
44
For both scales, the male students’ initial scores were more positive (higher for PERMA, lower for K10)
than for females. Further, males’ scores improved by more over the course of the Project. These
results suggest either that the Project was more effective for male students than female, or that it was
more effective for students who were starting from a more positive point.
In an attempt to see whether the Project had greater impact for students with more positive starting
points, the T1 PERMA and K10 scale scores were divided approximately into quarters. (Exact quarters
were not possible due to groups of students having the same scale score. Had the group been divided
into exact quarters, some students with the same scale scores would end up in different quarters.)
For the PERMA Scale, the most improvement was shown by the students with the worst starting
scores. This was followed by the students with the second worst starting scores. The students with
the second-best starting scores showed very little change while the students with the best starting
scores scored worse at T2. Without additional evidence, it is not possible to explain why this may be
the case. It may reflect regression to the mean (where those with unusual scores at one point in time
simply revert to more ‘normal’ scores at a second point in time).
The pattern for the K10 Scale was very similar. Students who started with the highest (least positive)
scores improved the most and the students who started with the lowest (most positive) starting points
scoring worse at T2.
3.3.1.1 Summary
There were statistically significant improvements from T1 to T2 for the whole group for both the
PERMA and K10 scale scores.
There was no clear pattern of change based on the age of the respondents but the results do suggest
that the Project was more effective for male students than female students. Overall, students with
less positive initial scores improved more than students with less positive initial scores. However,
male students had, on average, more positive starting scores than female students and also, on
average, improved more than female students. This suggests that there was a difference in how
effective the project was based on gender as the overall pattern suggests that female students, who
started with less positive initial scores than male students, should have improved by more.
A number of factors, other than varying effectiveness of the Project for different levels of wellbeing,
could be at play in these results. Students having a particularly good or bad day at either time interval
would likely impact their results significantly, as all scales within the survey specifically ask about
factors such as satisfaction, sense of worthiness and happiness which are likely to be impacted by
current mood. There is also the simple fact that on any scale, the worse the starting point, the more
room there is for improvement.
3.3.2 PERMA Questions The PERMA Scale consisted of five questions. Each question used an answer scale of 0 (Never) to 5
(Always) and higher scores are considered better. The questions were:
1. How often do you feel joyful?
2. How often do you feel excited and interested?
3. How often do you feel satisfied with personal relationships?
4. How often do you feel you lead a purposeful and meaningful life?
5. How often do you feel you are making progress towards achieving your goals?
45
There were only two questions for which there was statistically significant changes, regardless of the
type of analysis undertaken. These were how often the respondents felt joyful and how often they
felt satisfied with their personal relationships.
For the whole group, there was a small, statistically significant increase in how often respondents felt
joyful. There was a slightly larger increase for how often they felt satisfied with personal relationships.
When broken down by site, there was a statistically significant increase for how often respondents
felt satisfied with personal relationships in Sites 1 and 2, but not in Site 3 (with the smallest number
of respondents). There were no individual sites where increases in joyfulness reached statistical
significance.
When broken down by gender, there were statistically significant increases for males for both
questions. There was a statistically significant increase for females for how often they felt satisfied
with personal relationships. This suggests that the programme was more effective at improving mood
for males than for females, but was similarly successful for both groups at increasing their satisfaction
with personal relationships.
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Joyful Satisfied with personal relationships
Mea
n S
core
PERMA Questions
T1
T2
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Site 1 Site 2 Site 3
Satisfied with personal relationships
Mea
n S
core
PERMA Questions by Site
T1
T2
46
There were differences for the same two questions when broken down by age group. For 17-year-
olds, there was a statistically significant increase for both questions. For respondents who were 19 or
more years of age, there was only a statistically significant increase for how often they felt satisfied
with personal relationships. However, there was no clear gradient over age for either question. The
changes were stronger for 17- and 19-year-olds, and weaker for 16- and 18-year-olds.
3.3.2.1 Summary
In summary, the results suggest that the improvement in PERMA scores related to two areas:
improvement in mood (particularly, feeling joyful) and improvement in satisfaction with personal
relationships. Males showed improvements in both areas, and females only for relationships. There
were stronger outcomes for 17-year-olds than for any other age. The lack of a clear pattern in
outcomes by age group suggests that the latter may have been due to the specific cohort of students:
the same pattern may not be repeated in future years.
3.3.3 K10 Questions The K10 scale consisted of 10 questions using an answer scale from 1 to 5. As the questions relate to
negative factors, lower scores are considered better. The questions were:
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Male Female Male Female
Joyful Satisfied with personal relationships
Mea
n S
core
PERMA Questions by Gender
T1
T2
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
16 or less 17 years 18 years 19 or more 16 or less 17 years 18 years 19 or more
Joyful Satisfied with personal relationships
Mea
n S
core
PERMA Questions by Age
T1
T2
47
1. About how often did you feel tired out for no good reason?
2. About how often did you feel nervous?
3. About how often did you feel so nervous that nothing could calm you down?
4. About how often did you feel hopeless?
5. About how often did you feel restless or fidgety?
6. About how often did you feel so restless you could not sit still?
7. About how often did you feel depressed?
8. About how often did you feel that everything was an effort?
9. About how often did you feel so sad that nothing could cheer you up?
10. About how often did you feel worthless?
There were statistically significant differences across the whole group for two questions. These were
how often respondents felt nervous and how often they felt so nervous that nothing could calm them
down. For both questions the scores improved (lowered) from T1 to T2. This suggests that the
programme was more impactful in addressing anxiety than depression.
There were statistically significant changes for the same two questions at Site 1, but neither question
showed statistically significant change in either of the other sites. It is very likely that this is likely a
function of the higher number of respondents in Site 1 making it easier for the test to reach statistical
significance, because there was a larger change in nervousness for Site 2, and a larger change for
inability to calm down in Site 3.
None of the other questions showed statistically significant change in any of the sites.
1
1.5
2
2.5
3
3.5
4
4.5
5
Nervous So nervous could not calm down
Mea
n S
core
K10 Questions
T1
T2
48
When broken down by gender, there were statistically significant decreases for both males and
females for how often the respondents felt nervous. As with the whole group, this suggests that the
Project had stronger impacts on anxiety compared to depression. For males, there was also a
statistically significant decrease in how often they felt so nervous they could not calm down. This
suggests that the programme was more effective at impacting high levels of nervousness in males
than in females.
Respondents who were 16 years of age or less showed statistically significant improvement for the
same two questions. Respondents who were 18 years of age showed statistically significant
improvement in relation to how often they felt nervous. Changes for 17- and 19-yearolds did not
reach statistical significance. While the age pattern is not clear, it may be that the programme tends
to have greater impacts on anxiety for younger participants.
1
1.5
2
2.5
3
3.5
4
4.5
5
Site 1 Site 2 Site 3 Site 1 Site 2 Site 3
Nervous So nervous could not calm down
Mea
n S
core
K10 Questions by Site
T1
T2
1
1.5
2
2.5
3
3.5
4
4.5
5
Male Female Male Female
Nervous So nervous could not calm down
Mea
n S
core
K10 Questions by Gender
T1
T2
49
3.3.3.1 Summary
Across the whole group and for each of the sub-groups there were only statistically significant changes
for the two questions relating to nervousness. As noted, this suggests that the Project was more
effective at reducing levels of anxiety compared to levels of depression.
Again, there were more impacts for male respondents than for females with both groups improving
on how often they felt nervous but only males having a statistically significant improvement in how
often they felt so nervous they could not sit still.
There were also differences in impacts based on the age of the respondents with the strongest impact
being for respondents who were 16 years or less.
3.3.4 ONS Questions The ONS Scale questions used an 11-point scale from 0-10 and asked respondents to rate themselves
for the following three questions:
1. Overall, how satisfied are you with your life nowadays?
2. Overall, to what extent do you feel that the things you do in your life are worthwhile?
3. Overall, how happy did you feel yesterday?
There were no questions for which there were statistically significant differences between T1 and T2
for the whole group, nor were there any questions for which there were statistically significant
differences for sub-groups when broken down by age or gender.
There was one question, how happy respondents felt, for which there was a statistically significant
improvement at Site 1. The fact that there was a larger, negative change in Site 2 that was not
statistically significant again suggests that reaching significance at Site 1 was likely a result of the larger
number of respondents. The results across the three sites were very different, with an increase in Site
1, a similarly sized decrease in Site 2 and no change in Site 3: however, these results cannot be
accepted as reliable given the lack of statistical significance.
1
1.5
2
2.5
3
3.5
4
4.5
5
16 or less 17 years 18 years 19 or more 16 or less 17 years 18 years 19 or more
Nervous So nervous could not calm down
Mea
n S
core
K10 Questions by Age
T1
T2
50
3.4 Differences between sub-groups at either T1 or T2
3.4.1 Scales The paired sample t-tests above showed that there were some questions for which there were
statistically significant changes over time for some questions for some sub-groups but not others. This
section examines whether there were statistically significant differences between sub-groups for any
of the questions at either time interval – that is, when the first round of data was collected toward the
stat of the year or when the second round of data was collected in the middle of the year. This is done
to assess whether differences between groups increased or decreased. The sub-groups used for these
analyses are site, gender (excluding ‘other’ due to the low number of respondents) and age (recoded
into the same four groups described earlier).
For the overall scale scores there were no statistically significant differences between respondents
based on either the site they attended or their age. There were statistically significant differences at
both T1 and T2 for the K10 scale based on gender. At both time intervals males scored lower (more
positively) than females and the gap between the two widened slightly. This supports the previous
suggestion that the project was more effective for males than females, at least in relation to the
anxiety and depression.
0
1
2
3
4
5
6
7
8
9
10
Site 1 Site 2 Site 3
Happy
Mea
n S
core
ONS Questions by Site
T1
T2
1
1.5
2
2.5
3
3.5
4
4.5
5
T1 T2
Mea
n S
core
Mean Scores by Gender, K10 Scale
Male
Female
51
3.4.2 Questions When looking at the individual questions at the different time periods, there were no statistically
significant differences by age or site for any of the questions across the three scales.
There were also no questions from the ONS scale for which there were statistically significant
differences by gender. There was one question, how often the respondent felt joyful, from the PERMA
scale for which there was a statistically significant difference at either time interval. The difference
between males and females was not statistically significant at T1 but the gap increased with males
both starting more positive and improving by more, so that by T2, the difference was statistically
significant.
There were statistically significant differences between genders at T1, T2 or both for eight of the ten
questions which make up the K10 scale. The graph below shows the mean scores for males and
females at both time intervals. In all cases, males scored more positively than females at both times.
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
T1 T2
Mea
n S
core
How often joyful
Mean Scores by Gender, Individual PERMA Questions
Male
Female
1
1.5
2
2.5
3
3.5
4
4.5
5
T1 T2 T1 T2 T1 T2 T1 T2 T1 T2 T1 T2 T1 T2 T1 T2
About howoften didyou feeltired out
for no goodreason?
About howoften didyou feel
nervous?
About howoften did
you feel sonervous
thatnothing
could calmyou down?
About howoften didyou feel
hopeless?
About howoften didyou feel
depressed?
About howoften didyou feel
thateverything
was aneffort?
About howoften did
you feel sosad thatnothing
could cheeryou up?
About howoften didyou feel
worthless?
Mea
n S
core
Mean Scores by Gender, Individual K10 Questions
Male
Female
52
The chart below shows the differences in scores between males and females at the first and second
rounds of data collection. As can be seen in the chart below, for six of these eight questions the
difference between males and females increased. For two questions (how often they felt nervous and
how often they felt worthless), the gap decreased. There was two questions, (how often they felt
hopeless and how often they felt everything was an effort) where the widening of the gap meant that
the difference between males and females was statistically significant at T2 when it had not been at
T1. There was also one question (how often they felt nervous) for which the reduction in the gap
meant that there was no longer a statistically significant difference at T2.
3.4.2.1 Summary
The analysis of the difference between sub-groups at each time interval showed that there were no
statistically significant differences for any of the three scales or individual questions based on age or
site. There were, however, numerous differences based on gender.
Male respondents had more positive starting scores than female respondents for the K10 scale and
improved by more. The same was also true for one question from the PERMA scale and six questions
from the K10 scale. There were also two questions from the K10 scale for which males scored better
than females at both time intervals but females improved by more.
These results suggest, in line with the previous analysis of differences between sub-groups, that the
project was more effective at improving male respondents’ sense of wellbeing and psychological
distress compared to female respondents.
3.5 Differences based on starting score Starting scale scores were also used to look at whether the project was more effective for those that
had more or less positive scale scores. For these analyses the respondents were divided into two
groups, those who had an initial score higher than the median score for that scale and those that had
a score lower than the median score.
For the PERMA scale there was a slight, but not statistically significant different, decrease in the
PERMA scale scores of the respondents who started with higher wellbeing scores and there was a
larger, statistically significant increase for the respondents with lower initial wellbeing scores.
0
0.2
0.4
0.6
0.8
1
About howoften didyou feeltired out
for no goodreason?
About howoften didyou feel
nervous?
About howoften did
you feel sonervous
thatnothing
could calmyou down?
About howoften didyou feel
hopeless?
About howoften didyou feel
depressed?
About howoften didyou feel
thateverything
was aneffort?
About howoften did
you feel sosad thatnothing
could cheeryou up?
About howoften didyou feel
worthless?
Dif
f. in
Mea
n
Difference in Mean Score of Males and Females, K10 Questions
T1
T2
53
A similar pattern was seen for the difference in change for respondents with higher and lower levels
of psychological distress (higher K10 scale scores indicate a higher level of psychological distress).
Respondents with higher levels of initial distress has a small, statistically significant decrease in the
mean scale score while respondents with lower levels of initial distress had almost no change.
As noted earlier in this chapter this suggests that the project was more effective for respondents with
less positive initial scores. That is, it made more difference for respondents who initially reported
lower levels of wellbeing and higher levels of psychological distress.
3.6 Conclusion While modest, the overall results from the survey data were positive with statistically significant
improvements on both the PERMA and K10 scale scores and statistically significant improvements for
4 of the 18 individual questions. In support of the fundamental dictum that programmes should do
no harm, there were no questions or scales where the scores statistically significantly worsened from
T1 to T2.
When interpreting these results, it is important to note that the surveys were collected roughly at the
beginning of term 1 and the end of term 2. This means that this change represents the change within
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Higher wellbeing Lower wellbeing
Mea
n S
cale
Sco
re
PERMA Scale Score by Initial Positivity
T1
T2
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Higher distress Lower distress
Mea
n S
cale
Sco
re
K10 Scale Score by Initial Psychological Distress
T1
T2
54
one semester. Differences in the way the project is provided means that this is not necessarily the full
length of time that respondents were, or will be, exposed to the Project and there could be more, or
different, changes over a full year.
There were some differences in whether outcomes reached statistical significance based on site, but
this most likely reflects different numbers of respondents at those sites. No conclusions about
different effectiveness of the sites should therefore be drawn.
There were some differences in outcomes based on age, but patterns were unclear, and it is quite
possible that these reflect the particular cohort of participants.
There were, however, a number of differences based on gender. Because of the small number of
respondents indicating ‘other’ gender (N=3), they were removed from the gender analysis. The
analysis therefore only included male and female respondents. For all questions, and both time
intervals, where there were statistically significant differences between the sexes, males scored more
positively than females. Further investigation of patterns of effectiveness for males and females
seems warranted.
The quantitative data suggests that the programme had greater impact for male students than for
female students. It also suggests that the programme was more effective for respondents who started
from less positive positions than those who were more positive.
55
4 Project model and contributions
4.1 Modes of implementation and outcomes for young people As the importance of adapting content and delivery methods to young people became clearer, interest
in the effects of the mode of delivery on outcomes for young people increased. This was reflected in
the evaluation question: “Do different modes of implementation of the project skills (e.g. modular,
project-based learning) influence the nature of outcomes for young people, and if so how?”
In the first iteration of the Programme Theory model for Resilient Futures Project, ‘high fidelity’
implementation was encouraged (that is, using the training programme in exactly the way it was
presented, with a prescribed number of sessions in a prescribed order).
Service delivery agencies described their responses to the initial structured programme. At least one
knew ‘in advance’ that the structured approach wouldn’t work; others tried to implement it but failed.
When it first was introduced it was quite a structured programme, and there were
some really key, you know, points that we had to tick off and kind of work through,
and with our demographic of student, they’re all playing around outside, it wasn’t
going to work. It just – that structure of it wasn’t going to work. (S1 Page 2.)
The first year didn’t go so well because we had the Resilient Futures pack, and it
was teach this, and it didn’t work. Our kids have been through a whole lot of
counselling and a whole lot of everything, so they’re sick of anything to do with
wellbeing, so they’ll just turn off. So that was pretty clear, that first year, when we
couldn’t get them to do it. So then we decided we’d actually map it across the
curriculum and we did that quite effectively, so within PLP we’d map certain things,
and we worked it out across everything, and when it was hidden the kids were quite
happy to do it, or when it became an English task they were quite – so if we were
looking at gratitude in English they were quite happy to – if I’d called it something
different they were quite happy to still do the same skills and the same knowledge.
So that’s how we ultimately ran the entire package through. (M1 Page 1.)
Because it’s an adult environment, we have students that might be working, or they
might do their class, they might be enrolled in a class but do it on a different day,
and things like that. And so, because it was something that was quite heavily
scripted, I guess for our students, like, being in an environment like this, it was
probably a little bit too traditional schooly [sic] for them. (S3 Page 3.)
One WRC staff member who had been involved since the beginning of the project described some of
the early experiences in implementation.
I think it’s fair to say at the start of the project we probably didn’t know our punters
well enough, we didn’t know exactly how they rolled out any of the things that they
do, and even I think that changes on a class-to-class basis. (WRC2 Page 3.)
We went to one place and we said here’s your manual, and here’s the participant
guide ... and they said if we give anything to our kids on a worksheet they’ll rip it
up … that’s just not how they work, right? So, we were so dejected. Then the next
day we went to the next partner and they said we love it. If it’s not on a worksheet,
they won’t do it. And it’s like – honestly, this was like Monday and Tuesday, back-
to-back. (WRC2 Page 4.)
56
And a bunch of these kids in the first prototype saw juggling balls, said, in their own
language, I’m not interested in that, and they just didn’t come back. And it’s, like –
again, that was a big wake-up call of this is the real deal, like, this is not the rich
schools where the kids are there, and they rock up and they say please and thank
you. Like, not that we were going in naively, but probably more naively than we
should have been, maybe, or could have been. (WRC2 Page 5.)
Other WRC staff agreed that their understanding of the effects of context had deepened during the
early stages of the project.
We might have been quicker to move to a sort of a blended, implicit/explicit model,
rather than believing that the rhetoric of the (Training) organisation, which was
here are these skills, just deliver them as we’ve told you, it’s like slip, slop, slap,
everybody goes through the sheep dip, we wouldn’t have bought into that. Because
what I always knew, but what I now know very differently, is that context is
everything, and that you have to adapt and evolve to enable people to adapt to
their unique circumstances. So, it took us the best part of a year to work that out,
really. Um, and maybe if I’d been smarter I would have known that, but maybe not.
(WRC3 Page 12.)
What I think we tried to convey was here’s something to run with, you know,
roughly. Ah, but a lot of them heard we are scientists and we need you to do it
exactly like this. And of course, they resented that........or some resented it, but
some just said we can’t do that, you know. ... (some would say) this is not a lab,
this is the real world. (WRC2 Page 3.)
The second iteration of the programme intended that agencies and workers should adapt the content
to their own settings and clients. The intent of the Resilient Futures project remained the same, but
the process of delivery changed significantly.
My understanding is that the intent of the programme is to build the resilience and
wellbeing in young people who are disadvantaged and disengaged from education
and the community, and the purpose of the programme is to deliver a skills-based
intervention to that cohort which helps build their resilience and wellbeing. The
programme was intended initially to be a modular group programme across eight
weeks with 10 skills taught and that’s obviously changed over time, and it’s now
it’s a much more fluid mode of delivery. But really it was just about trying to look
for those explicit opportunities to deliver these skills to young people really to help
build their implicit capacity to use these skills in their day-to-day lives with the
intention of bringing about greater wellbeing for them. (M3 Page 1.)
This involved a lot more work in identifying and understanding the context and working with the key
players to develop the best fit for Resilient Futures Project for the young people.
Giving them much more respect for their own knowledge and wisdom, and ‘how
do you bring that into the programme?’, rather than kind of naming a prescribed
set of skills that you have to follow in a particular way. I think there was a sense, …
then there was almost this, um, maybe teaching teachers to suck eggs because (we
said) ‘you’re going teach a skill in this way’. And teachers know their context, they
know what works and what doesn’t, and they – they’ve got broader knowledge to
57
think about this than perhaps what we have. Um, so yeah, just using their
knowledge I think has been a big shift. (WRC1 Page 1.)
The second iteration of the Resilient Futures Project enabled the skills to be implemented in different
ways to suit the learning style of the young person and the agency involved.
It was just about using those cornerstones, those pillars, and introducing them in a
way that felt a lot more, I guess organic, and suited our setting, and it worked really
well. Obviously, we’re working with disengaged young people who a lot of the
time may not have attended school in potentially two, three years, and then they’re
coming back, or … coming in with really, really complex, complex backgrounds. So,
you haven’t got that set structured class where you can deliver content and the
students are just there. You’re sort of always trying to be flexible and trying to
create programmes that they could potentially do at home, but you’re not able to
just deliver to a class like you can (to a mainstream class) (S1 Page 2/3.)
… that made it easier…to deliver the material, because then people could be much
more flexible and fluent, obviously, in how they delivered it, while still maintaining
their anchor points. That was probably the best thing. And then after that I think
people just ran with it and were able to make sense of it. And, I think having [WRC
staff members] available was really valuable as well, to be able to run things past
them, you know, like, what do you think about this, is this skill relevant? (M3 Page
4.)
Intentional Practice was incorporated into the Resilient Futures delivery model as a method of trying
to balance the need for flexibility to deal with different contexts with fidelity to the programme goals.
It was described briefly in Section 1.4 above. Providing ‘anchor points’ (critical aspects of the resilience
skills that should be taught) alongside ’mindful attention’ to the intent of the teaching was seen as a
strategy to “enhance program fidelity, or consistency of the Resilient Futures Program, across multiple
and diverse practice settings”. (Underpinning Theory of Resilient Futures, Page 3) .
WRC staff suggested that the change seemed to affect agencies’ level of engagement with the
programme.
I think we’ve seen a much more positive and engaged response from the agencies
as a result of the sort of changes that they were asking for that we were then able
to facilitate. So for I think most of our partners the shift away from a really
prescriptive, explicit model meant that ....... they believed that we were listening to
them, that we were understanding and respecting their strengths and their skills,
that we were trusting their creativity, that we were trusting their willingness to still
deliver the content with fidelity and respond. And I think as a result of that the
response we had when we did come in with strategies that were about ensuring
the fidelity and integrity of the model, were much more engaged. (WRC3 Page 1.)
However, It also appeared that some agencies had not been clearly informed of the change from
‘structured’ to ‘adaptive’ implementation, or had not grasped the significance of the change.
I think from SAHMRI’s point of view they concentrated a lot of efforts on schools, which kind of left us
out a little bit, and so I was persisting for a really long time with, you know, the way that they’d given
it to us in the book, getting very frustrated about it not working. And it wasn’t until I sort of said this
isn’t working, and they said you can use it off the book, and I was, like, why did you tell me that? Oh,
58
yeah, (Organisation) are doing that, and these people are doing that. It’s, like, well, how come you
didn’t tell us this before? (S7 Page 7.)
4.1.1 Five skills or ten The change underpinned development of a set of different modes of implementation by different
agencies. Some decided to focus on only some of the key skills of greatest perceived relevance for the
young people. This was negotiated with WRC and led to the selection of five key skills that could be
delivered.
Last year what we did was we went through all of the skills with them, and we
found that going through all of the skills, and not actually spending time really
exploring each skill in-depth ..... felt like..... there was a lot more of just lip-service
and then not actually being able to apply them effectively in their lives. So, we
picked out the ones that they........ really connected with, that they really found
relevant to them, and then we thought okay, we’re going to focus on these this
year, and really try and get them to apply those skills in their lives, instead of just
sort of talking about it in class and then having them walk away and not really
connecting with the skill. We’re seeing them actually being used in their vocabulary
and just watching how effective they are working this year then we can – we’ll
introduce other ones probably later on, if we’ve got time. (S5 Page 4.)
4.1.2 Implicit teaching Implicit teaching is a teaching strategy in which what is to be learned is not made explicit. It is often
undertaken through the provision of multiple examples through which the learner is expected to
deduce patterns or rules.
Because of the nature of target groups and some resistance to key concepts from some students
(discussed in section 2.6 above), the Resilient Futures work had to be merged into, and even hidden
in, more interesting sounding electives, such as a Sci-Fi elective at one school, or a Fitness elective in
another.
I find I have to be a bit surreptitious with this type of stuff, because as soon as you
use any of the terms mindfulness and that, it seems that a lot of the students come
with a bit of baggage about what these things mean. (S2 Page 3.)
They’re hearing it a lot and in hearing it a lot I guess they desensitise to it. So, in
some respects, you know, we have to be clever with how we sell it to them. It’s still
mindfulness and it’s still gratitude……but it’s not this thing that they’re hearing over
and over and over. I think positive psychology has become very popular so…..
whether they’ve heard it in school and it’s been drummed into them and drummed
into them enough that they want to tune out of it, so it’s about being creative in
how we bring that back to them, hence the electives. (S6 Page 7.)
In one school, there were electives that had Resilient Futures themes woven through them so that, by
the end, every young person there had had exposure to the Resilient Futures Project at some level.
They’re actually compulsory, and what we’ve said is that students at (agency) have
to be involved in electives because that is where we run the RF10. It’s more
59
informal. It’s not that formal teaching environment, you know, where they may
tune out, whereas in electives it’s their interest area, you’ve got them. (S6 Page 5.)
However, it was also evident that merging 15 minutes of Resilient Futures skills into a lesson on a
different topic significantly impacted the time constraints in the teacher and length of time students
spent on Resilient Futures sills. The non-voluntary nature of the merged lessons in regards to the
resilience aspects of the lessons may also have impacted how receptive the young people were.
(The teacher) introduces it as part of her cooking programme, but I think
sometimes that would be harder, because kids are there to do cooking, and they
might not be open to talking about RF10 skills, so I think they’ve got a tougher gig,
actually, the ones that are trying to, out of their hour and 15-minute lesson, they’re
supposed to be devoting 15 minutes to RF10. So I think they’ve got a tougher gig,
because kids haven’t chosen to do RF10, whereas they all have chosen for me. (M2
Page 24.)
Implicit teaching could also be undertaken in individual work, through the relationship between the
staff member and the young person.
Well, it doesn’t feel like high school to them, or primary school, it’s not like a
teacher ‘sit down’ thing, ‘oh, great, you know, (Staff name) is trying to teach me
this skill’, it’s just me dropping going ‘oh, you know, do you know what gratitude
is’, and its conversation, and then maybe an ad hoc activity, but they don’t even
know what I’m doing at that point in time. And then I might bring it up again, and
be like ‘oh, well, you know, how did you go with the gratitude? I bought you a
gratitude diary’, and they’re like ‘oh, really? Thank you’. And they think it’s like a
gift, as opposed to if you’re doing it in a group and you go ‘oh, okay, we’re going
to talk about gratitude today……and here’s your thing, and you have to write, like,
three things a day’, they just go ‘oh’. It’s (one-on-one) very different. (S11 Page
13.)
When asked why the approach worked with those young people, the interviewee suggested that it
was due to the young people having a different perception of what was happening compared to their
previous experiences.
Because the young people don’t feel like we’re trying to fix them, probably. That’s
all they’ve ever had in their life is people trying to fix them and guide them and get
them on the right track, and when you’re a mentor and you’re doing it one-on-one,
you have to be very therapeutic, and you have to be very creative in the ways that
you’re trying to guide them and help them. Like, I honestly think that they don’t
even know what you’re doing at times, but they love the relationship that’s built
up and they actually love the activities that you do, but if they were to read the
case notes, which they don’t……unless they ask when they turn 18, they’d read
back, and they’d go ‘oh, wow, like, I didn’t actually realise what my mentor was
doing at that point in time’. (S11 Page 13.)
4.1.3 Group work and individual work In multiple organisations, it was suggested that a two-part process worked best with young people –
class work followed by one-on-one supportive, reflective work. Consistent, reinforced messages was
60
seen as one of the most important aspects which made the process valuable for young people. This,
in turn, supported the importance of common language and practice within organisations.
…class work followed up by one-on-one … so those that probably had the best
uptake were the ones who were familiar with the material, so really opportunities
to build awareness, maybe had done the group programme once or twice in the
(agency name) or in the whole programme every day or had a case manager who
was able to constantly use the narratives with them and pushed that kind of thing,
is where we found it worked best. (M3 Page 10.)
We knew from the beginning that you can teach these skills but if they’re not
reinforced it’s going to have minimal impact. (M2 Page 2.)
Another respondent suggested that for younger students one on one had worked better than class
settings while the opposite was true for older students (see section 2.6). This suggests that different
delivery models are more appropriate for different ages, although further research would be
necessary to confirm this hypothesis. They also noted that they had been able to see a difference in
uptake between two of their classes, one which received the project through a class setting only and
one which received the project through both a class setting and case management. This fits with the
the programme’s hypothesis that young people should be exposed to the skills first, in this case
through the group setting, and then those skills are reinforced through one-to-one work.
I have a class that isn’t case managed and are [school] students and a class that
are case managed and I see the difference between how quickly the uptake of
things in that difference too because one’s obviously becoming more absorbed in
it, even having that one-to-one and they can bring back what they’ve learnt from
the group and if they’ve had any concerns and that can get unpacked and so, yeah,
where we see the absolute optimal benefit which, I think, is what the original model
was meant to be doing anyway… (M5 Page 24.)
The difference in purpose of classroom delivery to one-on-one delivery (often undertaken as part of
mentoring) was also echoed by staff in organisations in their discussion of what the strengths and
weaknesses of the modes of delivery were.
I think in a class setting it’s good to really overview the tools. But at a unique,
individual level you're going to miss students completely because again, like we
said, it’s custom made. Students are just going to be taking away from it what's
relevant to them. Sometimes, if they don’t know it thoroughly, they're going to
miss that relevance in a greater presentation. I feel on a one-to-one basis you can
cut through the fat and go directly to the source. In a presentation, you're really
getting a lot of noise in the room. For a lot of our students here, presentations and
classrooms and that bigger approach are triggers for a lot of anxiety, for a lot of
misunderstanding… (S12 Page 19.)
I think a coupling of the both I think could be really helpful. I think actually
transferring information is better in a formal kind of setting – still playful and
integrated and you know, done well. Um, whereas if you try … and then maybe
unpacking that more on a one-on-one level… (S14 Page 18)
61
Well there’s so much going on, like, there’s so many different personalities, and
there’s so many different complex issues. You know, you’re managing behaviours
but you’re also managing whatever comes up for those young people, trying to
keep them on track and redirect them. I personally think it’s a lot easier to do these
skills one-on-one. (S11 Page 5.) … And then they get uncomfortable because you’re,
you know, you’re doing it – they – they don’t want to respond in front of other
people, and you know, a lot of these young people are very private. (S11 Page 7.)
Group work was seen to be valuable in some contexts.
I just like the brainstorming and how different people think, and being able to
bounce ideas off of one another, and, maybe it’s not as intense, it’s making it a bit
more relaxed, which can be good. (S11 page 6.)
… the strength is really positive group discussions and feeling like they’re part of
something, I think feeling like they’re moving on together as a group in regards to
using the terminology and the skills, but the weaknesses are that some students
don’t connect. You know, there are some that probably do feel left out because
they don’t see the relevance of that skill to them. (S5 Page 19.)
However, group settings also had risks in other contexts. Building a trusting and caring group structure
with the young people, which takes time, before introducing the Resilient Futures material helped
strengthen the outcomes for young people. Trust between young person and facilitator was viewed
as critical to success (see section 2.8.2 above). However, in group delivery settings, trust and
relationships between participants were also be important.
The strengths (of a group setting) would be if you could get the actual the vibe
going, it’s self-sustaining, like, it’s self-building. It would have a positive feedback
effect, so it would just build on itself. And certainly, ... that’s what’s so good about
group training, is that because people are so keen on it, it just goes, it just builds
on itself. But the weaknesses are here. If you’ve got a whole lot of people coming
in here with lots of issues……it’s too easy to derail, because those people will derail
it, and then if they’re the stronger personalities in the group... (S2 Page 23.)
For the agencies involved in implementing the Resilient Futures Project with children in care, the risks
of trying to do it in a group setting often outweighed the benefits, and the agencies remained at the
level of implementing Resilient Futures at a one-on-one level in an implicit, as needs and time arose,
non-structured way.
They’re all in care, all these kids are all in care, it’s not just, like, mainstream kids
that go home to their parents. Every single one of them has been removed from
their home and has been affected by, you know, different things. So, lots of
different trauma, you sit those people in a room, and if they don’t get along things
can kick off so you’ve got to be really intentional with which young people you put
in the room to then be able to teach them a programme. Is it going to work, is it
not? That may be another reason why we haven’t run it yet, because we haven’t
figured out what – what that group is going to be. … Just like so much going on for
them, they’re not in their pre-frontal cortex, they don’t want to take it in, then
they’re not going to be able to absorb the information, they’re not going to be able
to commit to coming every week, they’re not going to be consistent, ...... basically
62
their adversities in life are way more important than trying to get themselves out
of it, and that’s where the one-on-one comes in. (S11 Page 17 and 18)
One interviewee suggested that group work was more likely to work better in a formal school setting,
where the young people were more settled, as compared to working individually with young people
experiencing trauma:
The good part of it being a in a group is it’s more structured. Like, it’s an actual
structured thing, you know, they’ve got the skills there one after another so, it’s
like fresh in their brain. The good thing about doing it one-on-one is that it’s really
casual, and they don’t know what you’re doing which, I think with the kids we work
with, is better. (S11 Page 12.)
Groups were also reported to be less appropriate for some young people with high levels of anxiety.
I think there’s that kind of obvious limitation which is, in a group setting, some
students may empower conversation, or, it also might make others, you know,
we’ve got a highly anxious cohort, social anxiety and all that can kind of inhibit
their voice a little bit, they’re just a bit nervous to speak up in a group'. (S4 Page
19.)
Many factors were reported as required for Resilient Futures to work in a group setting. The numbers
attending, the mix of young people and whether the group is voluntary or mandatory, as well as the
number and skill set of the facilitators were all reported as impacting effectiveness.
I still feel, very much, individual is best for me, because there’s so many energies in
a group, there’s so many roles that are played, and you’re trying to work people
out in such a big group in such a short amount of time, that it’s quite overwhelming
in some ways. (S7 Page 21.)
I guess you’d have to be really careful about which children you include at those
sessions. You might have to have more group sessions than initially thought
because there’s all different sort of dynamics, and different issues between
friendship groups, and risks involved in having some people within the same room.
(S9 Page 12.)
I think it’s that show-off factor, or shame factor, that if you’re going to talk about
this, other kids will be saying things, doing things, to either get your attention or
try and upset the person you’re with. So, anytime I had to do an important
conversation I’d usually try and go for a walk or go for a drive with that person
anyway, because it wasn’t a good idea when other kids were around, even if it’s
only two kids around. You have to have someone who’s ready and receptive to it. I
think if they’re sort of all together, and they’re kind of, like, wanting to show off, or
if they’re drug affected, or anything like that, it makes it a bit difficult as well, and
when some of the kids have been out all night, they come back, they want to sleep,
they don’t want to talk about gratitude and things like that. (S10 Page 11
While there were differences identified between one-on-one modes of delivery and group work
modes of delivery, it was also noted that there was a difference in the nature of group settings based
on the size of the group. Larger class sizes were seen as detrimental to implementation.
63
Teaching this to 30, like in a classroom, won’t be the same experience as doing it
with 10, 10 is very intimate, you know, and everyone gets airspace. 30 is too many
a cohort to teach it to. I think there needs to – probably groups of 10 to 15 would
be ideal. (M2 Page 22.)
Even in a mainstream school you couldn’t do one on six, like we do. I couldn’t
imagine it working as well with your 28/30 kids in mainstream. I think that would
be a nightmare. (M7 Page 24.)
Other interviewees said that Resilient Futures skills were better suited to individual learning
opportunities.
Individually I think people are much more likely to be encouraged to be vulnerable,
and I think that’s what the material – a lot of the material is actually about. (S7
Page 19.)
Many interviewees discussed the importance of ‘seizing the moment’ for the Resilient Futures skills.
Staff having the ability to discuss resilience when the opportunity was right, whether that was in a
group session or in mentoring session, was reported to be an important skill. Staff needed to be able
to read the situation and be able to respond with resilience skills in whatever topic was resonating
with the young person or people at the time. This suggests that staff need to have a high level of skill
to be able to assess the relevance of a particular resilience skill to a particular context, sometimes in
uncharted waters.
And you test the waters a bit with what you’re saying, and if they start lapping it
up then you just keep going with it. (S2 Page 23.)
If the kids were going to attend every lesson they’d be in mainstream. They’re not,
so it’s about grabbing them when you can and working on it. (M1 Page 15.)
While the differences in delivery modes could be clearly identified and described – the number of
skills, the strategies for delivery and so on – there was insufficient quantitative data to test whether
or not these differences impacted on the nature or extent of outcomes for young people.
4.1.4 Mentoring WRC had anticipated that mentoring and one-on-one work between workers and young people would
play a different role to class or group-based learning. Classrooms and group work settings were
expected to provide an environment for more general learning about the skills while the one-on-one
interaction would provide an opportunity for workers and young people to discuss ways in which the
skills could be applied to specific situations.
The evaluation therefore sought to investigate: “In what circumstances does mentoring contribute to
outcomes for young people, how and why?”
Staff from SAHMRI described different ways in which this might occur.
I think we always thought that mentoring would be the heavy lifter in this; the
biggest driver of change. The idea was roughly that you would learn about the
stuff, the skills in class, or whatever, in a group forum, and then you would then be
able to do that reflection in the safety of the one-on-one relationship. That was
that idea, because obviously, you know, you have to reflect on this stuff to make it
work and work out how it can apply to your life. We just wanted awareness from
64
the classrooms and sort of skill use, skill practice, rehearsing, coaching from the
mentors. (WRC2 Page 15.)
Through the mentoring. So, providing clear, private ways to practice and reflect on
this stuff, which for the average Geelong Grammar student, or most school
students, they probably don’t need that extra opportunity is, I guess, what I’m
saying. (WRC2 Page 21.)
Just, at least, to provide some sort of offline care. (WRC2 Page 30.)
Strengths of one on one mentoring were reported as being able to provide greater relevance to the
young person; tailor conversations specifically to that individual’s needs, situation and capabilities;
and thereby increase the chances of the young person embedding the learning of resilience skills into
their own lives. It was also seen as a method which promoted greater understanding the young person
and their circumstances.
(You teach) in a context, with real life experiences, and seeing how they can apply
this to their actual life, as opposed to just being sort of seen as maybe a text book
type of thing. And so that’s where I think the mentoring and the, you know, on-the-
fly discussions that you have with the student when they’re going through an issue,
I think that’s where they become really beneficial to the student, because it’s
something that they can draw upon immediately, and trial immediately. (S3 Page
27.)
Relevance, one-on-one, purely relevant to the young person, purely adapted to
whatever’s happening for them at that time. (S4 Page 19.)
When you’re a mentor you get to know that young person really well, and what
they will respond to in terms of cognition, and what they won’t. So say if they’ve
got, like, a really fixed mindset on something, then growth mindset would be a
good one to bring up, hey, like, let’s talk about how you talk to yourself, your self-
talk, and you can tie that in. Whereas if I was to try and do sort of gratitude in that
moment it probably wasn’t going to work. … it’s really based on their personality
and their interests as well......it is based on the depth of the relationship and your
understanding of the key relationships they’ve got around them. So, it is really
understanding that young person, the complex trauma that they’ve been through,
the developmental age that you know that they’ve got, and what they can sort of
understand, their ability to understand. (S11 Page 3.)
The programme theory, and some respondents, suggested that a mixture of group and one-to-one
settings would provide the strongest outcomes. Group settings were seen as appropriate for
introducing the skills and then one-to-one settings would allow opportunities to practice and reinforce
those skills. One respondent also suggested that there could be benefits to the effectiveness of group
work as a result of the young people having received one-to-one mentoring. It seems likely that the
two approaches can be mutually supportive.
I think mentoring one-to-one has massive value, massive value. I think that’s
probably the most important part, because I think with that ongoing case
management mentoring of these skills, these young people connect more when it
comes to group chats about the skills in a classroom environment. And I think, you
65
know, the mentoring from case managers, they’re the people that these young
people go to in confidence to talk about, you know, certain issues that come up in
their lives, so they’re the ones that come at the forefront of being able to say ‘okay,
well we could use this skill in this scenario’, or you know, if they’re thinking in a
certain way the case managers are then able to pull out the terminology around a
certain skill to use when things don’t go so well in a young person’s life. (S5 Page
20.)
I think what worked really well with our group and with our young people is that
they’re doing it with their mentor, someone that they have a really good
relationship with, and I just think it makes all the difference. (M4 Page 13.)
As discussed at multiple stages through this report, trust was seen as critical to the effectiveness of
mentoring as it allowed young people to be more open with the mentor and broach sensitive subjects.
It’s ......... probably because they don’t have an elder male role model in their
lives…especially one that might – they can take the piss out of a bit as well, and not
– they won’t get upset with them, and they can have that sort of casual banter, as
well as having some deeper conversations. So, I don’t treat them like children, I try
and treat them as people, and I think they really respond to that. And so that’s part
of that building of trust. And once they trust, once you build that relationship of
trust, you can broach all manner of topics with them.......and the strength is that
you can have a really powerful input. (S2 Page 22/3.)
But now it’s all, like, thinking about how they think, what they’ve been through,
what’s best for them, what are their values, what’s their purpose ....... it’s
(mentoring) a lot more complex than it used to be. (S11 Page 23.)
In some organisations the case managers were filling the role of mentors. However, where the roles
of case manager and mentor were separated, mentors had more time reflect with the young people
and work with resilience skills. In these situations, case managers’ roles revolved more around
organising the lives for the young people, in terms of court appearances, or medical appointments, or
meetings with schools and less about teaching of resilience skills.
To be honest, I would say probably the people in my kind of role (examples of other
case managers) I haven’t heard them saying ‘oh, we’re using this’, or ‘we’re doing
that’. So, I think it’s hard in our roles. The mentors, it sounds like they’ve done a bit
more of it. (S10 Page 4.) …And I know it’s (staff member name)'s passion, kind of
thing, as well, that kind of stuff, so I can see her rolling that more with the mentors.
(S10 Page 5.)
So – and also therefore the strength of doing in mentoring with case managers is
that – that – that case management mentoring sort of can pick up those ones that
feel would be left out of that group setting in regards to skills. (S5 Page 19.)
One of the elements of mentoring that was seen as useful was the ability to talk to a young person
across a variety of topics even when the mentor had not been the one to deliver training in the
resilience skills to the young person.
And it (mentoring) occurs across subject bands. So, it depends on who you form a
relationship with. So even though we might not be – like, for instance, this morning
66
I was sitting with a student who was doing English, and I don’t teach English, but
because we have that relationship I can talk to him about that. And so that happens
with all our mentoring, and all that sort of stuff. Like I know that he did one of the
more explicit versions of mindfulness, like the – the anchor points. I think (teacher's
name) went through and did a wellbeing course with him. But......I could talk to him
about that and bring up stuff about that on that one-to-one level, because I’ve got
that relationship with him, and can talk to him about it. So… (S2 Page 21.)
At least one worker identified that young people practicing the skills and coming back to talk with
them about it during mentoring sessions served to reinforce learning.
…and then it’s that kind of repetition of that, that they can go out, they use it and
they do it, and they come back and tell you how it went, and kind of flows on that
way. (S10 Page 16.)
Staff also needed to be able to read the young person’s current situation, determine the necessary
approach and to identify which skills may be useful to the young person at that time. This also linked
back to the idea of ‘seizing the moment’ as discussed in the previous section.
You’ve got to know when to go hard or when to be gentle. Knowing their regulation
circle, so whether they’re at baseline or whether they’re at escalated. Knowing, you
know, whether – is it tough love they need at that time, or is it that gentle reminder
and caring gestures that they might need, you know, like do you need to put your
hand on their shoulder and say, like, you know, come on, let’s try and do some
mindfulness breathing, let’s try and ground you, or is it, like, directive statements
that they need and you need to try and use a break-state to snap them out of that?
(S11 Page 23.)
It was also reported that some staff had improved at identifying which skills to use with young people
during one-on-one mentoring sessions and that was linked to more efficient use of mentoring time.
The case managers come and talk to me about it. They said 'oh, I’ve just used such-
and-such skill with this person, and I’ve taught them this', and they are getting
more skilled at choosing when a young person comes to you and they go “blah”, …
‘oh yes, they need, you know, it’s ETR they need’. Or ‘it’s, you know, it’s gratitude
they need’. So, I think the staff, the one-to-one mentors are getting more skilled at
being able to pick it. And then so instead of spending the whole session with them
hearing them whinge about their parents who blah, blah, blah or whatever, then
they can say ‘well actually one of the skills that help resilience is gratitude, and if
we think about your parents and this and this and this, you know, it might help you
to understand their perspective’. (M2 Page 23.)
4.1.4.1 Models of mentoring or case management
During the establishment phase of the project, it became clear that different agencies took different
approaches to provision of individual support. The evaluation question to investigate this issue was
“What impacts do different agencies' models of coaching, mentoring or case management have for
which sub-groups of young people, in what contexts, how and why?”
This is a very fine-grained level of analysis, and a combination of factors resulted in this question not
being answerable in this evaluation. Most significantly, research ethics approval limited the range of
settings from which data could be collected from young people, and approaches to mentoring or case
67
management tended to vary by type of agency. Secondly, all the quantitative data from young people
came from educational settings, where there was a similarity in the manner in which the one-on-one
delivery was handled. Thirdly, changes to the evaluation design over time meant that quantitative
data was not available until the final round of the evaluation, limiting the amount of data available.
Qualitative data was adequate to identify that workers attributed impacts to individual support, but
not to distinguish between the types of outcomes for different types of individual support.
While there were differences in the delivery of the programmes within the settings that provided data,
there were not statistically significant differences in outcomes for young people between the sites.
The site which included both classroom and one-on-one instruction did show more statistically
significant differences over time than the other sites, however, that is likely to be a statistical artefact:
there were higher number of respondents in that site, making it easier to reach statistical significance,
but the sizes of the changes were similar to other agencies.
4.1.5 Trauma informed practice For the purposes of this programme trauma is defined as “any experience, stressor or event that
significantly overwhelms an individual’s capacity to respond, deal with or process that event”
(Underpinning Theories of Resilient Futures, p. 8). Trauma can negatively impact young people’s
emotional state and cognitive development and is linked to a variety of mental health issues.
The Underpinning Theories of Resilient Futures document also provided a list of features that
consitituted trauma-informed practice. These were:
• Adults presenting as calm and safe, and responding and not reacting to youth
behaviour.
• Adults and agencies offering young people hope and supporting them reconnect with
and understanding their world by engaging with them in relationships and creating
safety.
• An understanding that teaching and learning occurs in the context of relationships,
and supporting adults can build relationships that are respectful, compassionate,
consistent and sustained over time.
• Adults seek to look beneath the youth’s behaviour and understand and respond to
the unmet developmental and healing needs.
• Adults have a key role to support young people with unmet trauma needs to identify
and manage their emotions.
• Adults can help young people to connect to their peers and understand their world.
• Youth agencies can create a place of calm for young people with unmet trauma needs
in their otherwise chaotic world.
• Youth agencies can create predictability and a sense of belonging for young people.
(p.9)
In some settings, staff also required knowledge of trauma informed practice, as teaching
disadvantaged young people with multiple complicating factors has to be done differently from
mainstream school. Many interviewees discussed that trauma informed practice needed to underpin
every aspect of their work.
…consistency, discussion, repetition, visual cues. So, most of our kids are
kinaesthetic and interpersonal learners, so it means they have to do it and talk
about it. They’re the two things they don’t let you do at high school. You’re not
allowed to talk about it, and you’re not allowed to get up and do anything, you’re
68
in big frigging trouble. So, we do that, and we do it over and over and over again,
and we have predictability and consistency. (S8 Page 15.)
Everything we do is safe, reliable, predictable, and discussed to death sometimes.
But always discussed, because that’s trauma informed practice. That is 100 per
cent safety, predictability, and giving them a sense of ownership and belonging.
Once you’ve got that from kids, they’re unstoppable. (S8 Page 19.)
Being able to provide examples of having used the skills in their own lives was also reported as
important in trauma informed work. In addition to helping to create trust and a sense of authenticity
as discussed in Section 3, it helped to normalise the experience that young people were going through
and had the potential to reduce self-blame.
…it shows that I’m not an expert, and I’ve stuffed up, and I actually need to use
these skills as well because I don’t have all the answers. And I think sometimes
people that are working with these young people don’t tell that to young people,
and so they think they’re the only ones screwing up, which is just not true. (S14
Page 24)
While references to ‘trauma informed practice’ were relatively rare in the interviews, it could be
argued that many aspects of implementation in several settings were trauma-informed. Individually
tailored work, delivered through one-to-one service delivery including mentoring, in relationship-
based approaches, with attention paid to trust (and so on) are all relevant to trauma-informed
practice.
4.1.6 Agency settings WRC staff suggested that the Resilient Futures Project had been more successful in schools as opposed
to youth service settings. The key factors here may have been that the young people in the schools
were more settled, that they had chosen to attend school, and had chosen to be involved in the
project. The young people experiencing greater vulnerability may have been exposed to the Resilient
Futures themes under less voluntary terms, in settings such as in juvenile detention or residential care,
and at a time when their life was too chaotic and traumatic to be able to focus and benefit from it.
I think there’s no doubt in my mind that this programme seems to work best in an
educational context. There is something about that semi-structured environment,
and it is only barely semi-structured for some of our partners, it’s still pretty flexible,
but there is something about the fact that young people are choosing to be there,
because most of the young people we’re working with are over 16 so they don’t
necessarily have to be there, on some level they are making some degree of
commitment to be there. … And there is something about that level of commitment
that’s meant that it’s been more successful in those contexts than in homelessness
programmes, for example, where you’ve got young people who are just beginning
to move out of homelessness into a journey of stability, and it’s just too fractious
and too uncertain for them. So, we haven’t been able to make those programmes
as effective, I don’t think. (WRC3 Page 12.)
There were some issues between external agencies where the Resilient Futures Project was
implemented with agencies working together in order to develop the project. One issue raised was
having multiple facilitators with different skills and experience levels working together and that having
the disparity between staff from different agencies made it harder to address.
69
… if we were thinking of having some cross-agency work again, I think we’d also
need to provide support around what that looks like to get rid of some of the pain
for them. … I’m talking probably about four practitioners, but I think it was painful
for them, and we could have thought about how our design could have helped with
some of that. ... So I think a new practitioner within Resilient Futures will perhaps
want to give the skills a bit more by-the-book, they’ll want to be that very explicit,
I’ve learnt about growth mindset, I’ve learnt about the science, this is how we do
it, … whereas I think a more experienced practitioner will understand that you don’t
have to always to be so explicit, and you can layer it, and it’s about drawing out
responses from the young people as opposed to talking at them. … just having to
negotiate between different levels of practitioners, when you’re two separate
agencies … if I’m more of an expert but I’m working with you from another agency
to say I’m more of an expert here, they didn’t feel like they could. … therefore there
was frustrations with going actually, I am more of an expert here, but we’re both
partners here, in some ways they couldn’t assert their knowledge. (WRC 1 Page
17.)
Another issue that was raised was organisations having different goals and different ideas of the best
way to deliver the project. Differences in teaching philosophies and the personal motivations of staff
for involvement were also suggested to influence the effectiveness of agencies working together to
provide Resilient Futures programmes.
4.2 On-line resources In the early stages of the project, SAHMRI tried to develop on-line resources to support young people’s
use of resiliency skills. The question for investigation of this strategy was “In what ways do on-line
resources contribute to outcomes for young people, how and why?”
The strategy proved less successful than anticipated.
One of the things we’ve tried was to introduce an online tool, for example. We
worked with a small enterprise here to put some of the knowledge on an online
platform that young people could download, or the idea that maybe if you were
having a mental conversation you might use an online resource to look something
up. But that didn’t work very well, and it didn’t work very well because the young
people weren’t very interested in… looking at the content. So, we did a few live
tests like that. We did a few things where we left it with our partners to try and talk
with young people over time with it. But it was quite clunky, our end as well, like,
in terms of a technology tool, it wasn’t easy to use....... we’ve got a set of
assumptions about why it didn’t work, and some of them are around the
application, but some of them are around just the whole experience of looking at
knowledge around positive psychology........ we probably didn’t focus on people’s
needs as much. (WRC1 Page 14.)
So, when the programme was originally dreamt up they said there was going to
be three parts to it, there will be the teaching, the mentoring, and the online
support. And we ....... figured out pretty quickly that the online support is very, very
complicated, and we did a bit of a token attempt to do that, mostly to satisfy the
obligation to the funders, I think. Whereas I think what we would have said is ‘let’s
not worry about that at all, because we can focus our attention on more things, the
more important things, possibly’. I think again it was one of those things where it
70
was worth trying anyway, just to learn, because you don’t know what you don’t
know sort of thing. (WRC2 Page 33.)
It was, therefore, not possible to evaluate impacts of on-line resources.
4.3 Conclusion The initial, highly prescriptive, model for delivery was changed in response to a far more flexible
approach during the early stages of programme delivery.
Agencies made a series of different decisions about ways in which to implement Resilient Futures.
This included the delivery of five skills rather than ten in some sites, a change which was made in
conjunction with WRC staff. However, there was insufficient quantitative data to directly test whether
or not these strategies made a difference to outcomes for young people.
WRC staff anticipated different roles for classroom and larger group based teaching compared to one-
on-one work with young people. Classroom teaching was seen as a way to introduce concepts and
skills while one-on-one delivery was more likely to provide young people with the chance reflect and
to practice the skills in relation to their own lives. This was largely supported by the staff in agencies
delivering programmes.
Being able to use implicit teaching strategies and ‘hide’ the resilience concepts were important to
acceptance by some young people, especially those who held negative views towards positive
psychology due to previous, negative, exposure.
WRC staff also suggested that the programme was more successful in school settings. This may be
due to the young people involved being more settled in their lives than those who were accessed
through other agencies and that they had chosen to re-engage with learning. Due to quantitative data
only being collected from educational settings it was not possible to test this hypothesis.
71
5 Workers and agencies
5.1 Impacts on agencies’ work As has been described earlier, the programme theory suggested that the Resilient Futures project
would improve outcomes for young people by changing the ways in which agencies worked with them.
The evaluation question for this element of programme theory was: “What impacts does participation
in the Resilient Futures Project have for agencies' work with young people, in what circumstances?”
5.1.1 Cementing existing approaches The Resilient Futures Project helped to cement approaches that units for disengaged young people in
schools were, in some cases, already using. It provided a sense of legitimacy and professional
alignment to their approach, especially when comparing to the mainstream school to which they were
attached.
In one school unit which worked with disengaged young people, the main school organisation had a
focus on positive psychology and was already talking some of the language. So, when the Resilient
Futures Project started this helped to reinforce what positive psychology might look like but also
strengthened the delivery of the project.
[SAHMRI gave] us more of a focus on how to deliver it....and so it was more about
having that structure behind us so that we knew what we wanted the students to
get out of it, what we wanted – because I guess a lot can get lost. If you don’t have
some sort of curriculum to sort of deliver, it can just get lost as a bit wishy-washy,
I guess, but because we had key elements to focus on, that really helped steer the
positive education stuff. (S3 Page 7.)
So, as an organisation more broadly it’s given us a greater, or a stronger framework
that now has positive psychology embedded in it. (M3 Page 4.)
Many of the schools said that they were doing it already but without the name and set process, and
the Resilient Futures Project had given them a renewed vigour and confidence for what they did.
Resilient Futures just reaffirmed a lot of stuff that we did…...and that was really
nice, because sometimes when you’re in this sort of situation you kind of question
what you do and how you do it because not everything is successful. Like, we fail
at a lot of stuff and it’s about how resilient you are as well, as a staff member,
because you’ve got to be able to take that on board and go ‘you know what? It’s
not me, necessarily, it’s just the way I did it’. .......that affirmation of what we do is
really important, or just raising some questions about well, how can we incorporate
that better, or maybe we haven’t considered that that anchor point was
particularly important, and how can we then build that into what we do as well?
So, it was interesting that someone else sort of came up with those anchor points
and looked at what they wanted to get out of the programme. (M1 Page 13/14.)
A lot of the stuff that is incorporated in Resilient Futures we do as a daily practice
anyhow, because our school is based around relationships. So, we talk to kids, we
know them really well, there’s a lot of that positive self-talk, there’s a lot of being
grateful. We’re trying to build those skills into everything we do, so if we did it that
72
way, they just went with the flow, because that’s kind of what we do anyhow, so it
worked in that respect. (M1 Page 3.)
(Resilient Futures Project) made us think more about how we do it and what we
could do more effectively. I would like to think that we’ve had a fairly big impact,
but I can’t break it down to separate the two. (M1 Page 7.)
One outcome which was reported by WRC staff but not discussed by the interviewees from the partner
agencies was the introduction of new policies and a wellbeing and resilience management role. It was
clear that some managers in participating agencies did play active roles in supporting and championing
implementation of the program. This is discussed further in Sections 5.2.4 and 5.2.5 below.
5.1.2 New skills and approaches While reinforcement, and sometimes refinement, of existing approaches was more commonly
reported, there were some workers for whom the Resilient Futures project introduced explicit
teaching of resilience skills.
I think we had the relationship.........the relationship was already there, but we
didn’t teach things like gratitude or mindfulness or how to value self. We valued
them as people, but we didn’t really teach them how to value themselves. And so,
we thought that, you know, we were doing a good job, and – and we were doing a
good job, but once we had the Resilient Futures and the anchor points in front of
us, then we could actually think about and talk about how to teach this to the
students so the students .....(know) how to be able to use this in their lives, which
is what we weren’t doing before. (S3 Page 16.)
For others, the change related to building young people’s capacity, rather than rescuing them.
…before we did this programme I could see the staff coming in as rescuers…I’ve
really been pretty keen on that not happening, so I’ve addressed that when I’ve
seen it, and we’ve tried to skill ourselves up in being aware that the young people
have got their own resources, and that, you know, we can teach them how to
manage things. (M2 Page 12.)
5.1.3 Impacts for staff There was much evidence of staff using Resilient Futures skills themselves, both in home and work
life. This had enhanced their confidence in it and in their use of it.
Experiencing it yourself first...And being able to share your own story around it. I
don’t know how many times I’ve said to a young person this programme has been
life changing for me. I can’t speak for them, but it certainly has for me, and I think
that, you know, word of mouth is the best advertisement. (S6 Page 18.)
I think since I’ve done it all.......... I’m a lot less stressed. It’s impacted my
personality, I think, and so the way that I execute myself here is really positive… (S3
Page 13.)
I’m a lot more grounded........the way I see it is, if I don’t work on myself I’m not
going to be any good for those kids, so I do the work on myself. I’ll sit, I’ll go back
home, and I’ll journal, I’ll write gratitude points if I had a bad day, I’ll do reflective
practice and how I could have done that better, or I’ll debrief with my partner or
73
my boss on the way home on the phone. I know what my values are now, and I
regularly do a test to see what they are and how that might shift so I’m not wasting
my time doing stuff that’s not important to me. Like, it’s massive, it’s massive, the
young people see that you’re doing the work so they’re going to think that it’s more
credible, they’re going to take on board what you’re saying. They’re not just going
to be like oh, they’re trying to teach me something that they’re not doing
themselves. (S11 Page 27.)
Learning the Resilient Futures principles resulted in some staff being more aware of their own and
their students’ life journeys. That increase in awareness enabled positive discussion between staff
and young people. It also impacted staff perception of young people.
It did improve my awareness and helped me remember where these students come
from, and that all of them have a story. That really opened up a lot of dialogue
around that, and when they’re not behaving in a way that we would like, 99 per
cent of the time there’s a reason for it, and it just helped us all see students, I think,
a bit differently, in a different light, which was really nice, through that process.
(S1 Page 7.)
Staff who became highly competent found that they were able to integrate the teaching of the
resilience skills into their own programmes more easily than previously.
I think that we, as staff, all got a lot out of it as well, and we actually were able to
recognise what we were doing well, but also improve what we were doing......it
improve(d) our practice. And now we, even without even having to try, we weave
that into all of our subjects now. Whereas before when we were first learning about
it, it was something that was really intentional that we had to say ‘okay, how can
we weave this into....’, and now it just naturally just weaves into our subject areas.
So, yeah, that was really – that was really good. (S3 Page 23.)
It was also noted that some staff had taken it upon themselves to undertake further learning about
resilience.
So, the people who just ran with it really took it on and they found a real symmetry
with the material both in their work and in their own lives. So that idea that SAHMRI
talk about around 'you need to learn it and live it so you properly can teach it', what
we found is that that was actually the case with lots of our facilitators, and those
here that you could really see had taken it on, and had done more research into it
and had really thought about how it fit into their life and the impact it has had on
their life, were then much more likely to run with it in their own work, because it
became part of their personal narrative and their personal framework. (M3 Page
3.)
5.1.4 Strengthening teams The connection between staff, and the consistent messages from staff in the small, contained units,
when they had all undergone the Resilient Futures training helped to strengthen teams. Positive
interaction between staff members was noted as positively impacting young people and assisted in
the development of a sense of community.
Knowing that you’re supported by other staff – like I know I’ve always got someone
to back me up if I need it, and that’s everything. That means so much to me. So, we
74
are really modelling that behaviour. The students then see how we engage and
interact with each other, which can be really, really lovely, and it becomes a bit
more of a, yeah, a little community. It’s nice. (S1 Page 13.)
I think doing Resilient Futures, it allowed us to sort of open up dialogue a bit more
about what’s going to work, what doesn’t work, what do you think of this, how
could we improve it, that sort of thing. So, I think in terms of collaboration in
developing learning programmes, and having similar expectations for students
across the board, that was really helpful, definitely. (S1 Page 15.)
We always share and talk about things, and ....... it’s just constant, we’re always
reinforcing each other. (S2 Page 7.)
5.1.5 Common language The development of common language was reported as an important outcome for organisations as it
provided a way to minimise misunderstandings, through having specific language for skills and
concepts, and increased people’s ability to articulate what they were doing.
I think common language is a really important factor in any organisation and, I
think, the more we can be utilising it in our communities at least, you know, a
similar language that is interpreted and, I think, that’s a really good skill. (M5 Page
29.)
I think, like I've said before in discussions around this as well, it’s nothing really
different in its approach. But the language used is different. I think the simplicity
there really transcends guesswork or different labels that other people can
misinterpret. I think the simplicity of language - we know what growth mindset is.
It’s a really big term. It’s overarching. But it has meaning to people. So if we’re
working on growth mindset, gratitude - so you start to learn a term and then that
overarching term is better than using eight different words that can be
misconstrued, and then people have different reactions and use it differently. I'm
not saying that these can't be used differently. The language used is really -
everyone has their different approach. But the language is important, so it feels
like everyone is on the same page, at least. (S12 Page 6.)
I think on a whole, at an organisational level, you need that camaraderie and you
need that interconnection between people. Or else you're going to miss marks
because people won't know how to articulate what they’ve been doing. But if you
have a set language, it gives people words that they can use to articulate what
they’ve been doing. (S12 Page 6.)
Common language was also seen as important as it allowed young people to interact with a variety of
staff members with clarity.
Because in this space, we have students that will - even though they're on my case
load, they’ll see a different case manager if I'm not there. Just as that added layer
of support. So if we’re using the same language when we chat to a student, it
makes it more consistent for them, and trustworthy, and it just feels like we’re a
bigger team. Then the students feel okay, this is consistent; I'm getting consistency.
75
Because if you didn’t have the same language and you're still getting a different
approach, it could feel quite alien for a person. (S12 Page 6.)
Having a common language also made it easier to identify when young people were learning and/or
using the skills.
We had some wording behind it, and we had some process behind, you know, the
anchor points, it was like you were able to then see that the students were taking
that and running with it and learning the skill. (S3 Page 20.)
5.1.6 The value of Resilient Futures training for skilled practitioners During early implementation of the project, it became clear that many workers in education and youth
work settings already had high levels of skill in supporting young people to become more resilient.
While not one of the original questions in the evaluation design, the effects of the programme for this
sub-group of skilled workers emerged as an area of interest for the project team. Consequently, it was
included as a question in the 2018 round of interviews with managers and staff. As well as asking
directly about the issue in 2018, a review of relevant findings from 2017 was also conducted.
Both rounds of interviews (2017 and 2018) supported the finding that some staff already understood
many of the skills being taught as part of the Project. Nevertheless, a number of benefits were
described by respondents including:
• refreshing the practitioner’s practice:
…none of these skills were new to me but, I think, having them placed in a way, I
think, that can almost become a barrier sometimes when you’re so used to working
with skills on a certain level that for people that have no appreciation of those skills
in the first instance, to be able to take your mind back to those days can be quite
difficult and, I think, the training really helps to ground you back into the mindset
of the clientele… (M5 Page 29)
…it’s good, because you already have a good understanding of, you know, trauma,
and resilience, and the background of young people, um, and it’s easier to just
relate it, and know how you’re going to put into practice. (M6 Page 10)
It’s definitely heightened my awareness of it, and, um, things like ACR, the active
constructive responding, um, being more mindful in my relationships about the
impact that that can have, and stuff like that. And it’s all stuff that, you know, was
sitting in the periphery of my mind before, but I think, um, having some actual
concrete reflection time on that, and actually learning about it in some kind of
concrete way has definitely heightened my awareness of it. (S14 Page 22)
• simplifying the language used:
I think that was beneficial because you can really get - even if you're quite
advanced, you can still fall prey to too much language use. It can get quite
complicated without it needing to be. So I think that simplicity was good for both
parties, to join in the middle. (S12 Page 17)
• developing common language across agencies, irrespective of initial skill and knowledge level
of workers:
76
I think common language is a really important factor in any organisation and, I
think, the more we can be utilising it in our communities at least, you know, a
similar language that is interpreted and, I think, that’s a really good skill. (M5 Page
29)
There was also new information for some practitioners who were already familiar with the majority
of the skills.
I didn’t know the impact of gratitude on the brain, but I think everything else, like,
I was already practising mindfulness, and… ETR and balance your thinking is very
much like CBT, like, cognitive behavioural therapy… (S14 Page 22)
5.1.7 Diffusion In some agencies, there was evidence of diffusion of resources and approaches with staff as well as
young people.
….I was facilitating resilience training last week and we had an existing staff
member who somehow slipped through the cracks and had never really done it
and, on the first day we get them to do the survey, the character strengths survey,
and then the following day we go through that skill. And she goes 'oh, I’ve already
done them'. And I said 'oh, okay, you know, what made you do that?' And she goes
'oh well, my house supervisor got us all to do it, and then we all talked about it in
our team meeting, and we’ve got a big chart on the wall in the house, so we know
each other’s strengths'. And that’s the sort of stuff that you can only hope for –
like, you can suggest it, but the house supervisor had gone to one of (name's)
workshops where she’d taken that away and gone 'I want to use that with my
team'. (M4 Page 15.)
Using Resilient Futures in the interactions between staff and agencies, not just between the staff and
the young people was a significant outcome for at least one agency. Resilient Futures provided a
framework to resolve issues between agencies. This included issues such as agreeing on different
approaches towards the same outcomes for young people, and needing to work together in a shared
space.
I know for a fact between staff members, they’ve used the skills to resolve conflicts
amongst them, or not necessarily a conflict, but you know, some kind of a
discrepancy type thing, and we can label it. We’ll voice it in a staff meeting. For
example, we had a situation, and we used win-win communication, and we feel
very okay with each other, and it’s all sorted, and we feel much better. It’s really
powerful and it’s really awesome to hear those examples. (S4 Page 15.)
We tend to try and utilise the Resilient Futures skills between us, if there’s any
tension between staff we’ll go to win-win communication. (S6 Page 4). … We’ve
just actually been through a very difficult period. Two organisations existing in the
same space with different policies........you get all the normal stuff around teens,
personalities, people getting along, not getting along........(and) we’ve been able to
actually utilise the framework, the Resilient Futures framework to, I suppose,
improve staff cohesion. So, we’ve actually used the skills ourselves to bring the
team together, to unite the team. (S6 Page 24.)
77
Staff that have really taken on all these skills work better together, because we all
have different roles, it’s a very complex environment where we’re working with
different purposes to achieve a similar goal - achieve the same goal for the young
person, but we’re coming at it from different avenues. So therefore, there’s always
the possibility of conflict in those scenarios. So, I think that (Resilient Futures
training) actually has reduced any issues when it comes to that kind of thing. (S5
Page 17.)
5.2 Linking project activities to agency outcomes Understanding how the Resilient Futures project had its effects on worker and agency practice was
important to WRC because it could inform how future work should be undertaken. The question for
this component of the evaluation was “What aspects of the 'Resilient Futures' Project influence which
aspects of youth agency practice, in what ways, and how?”
5.2.1 Establishment phase Work that SAHMRI had undertaken prior to rolling out the project was important to increasing
credibility, as it provided research evidence to support its content. Having support from an agency
with a science and research background was also seen as advantageous in getting support to run
Resilient Futures skills training.
Having [WRC staff members] on board to be able to support that process was really
useful. Not only did it give credibility to the material but it also supported ... our
thinking [around planning] for the material and making sure that we maintained
the integrity of it. There’s a process we have to go through before we can deliver
a programme at [organisation] and before that there needs to be an application
process (to the Programme Review Panel) and you need to show that the material
has a theoretical underpinning, it’s going to be relevant to the cohort. And having
the designers of the material who could talk very clearly about the theoretical
underpinnings and why they thought it would match well with the cohort gave the
material real integrity for all the (organisation’s) staff. (M3 Page 5.)
5.2.2 Adaptability SAHMRI used a flexible, two-way project development dialogue to work with agencies in introducing
and implementing the programme, and in modifying it to suit agency needs. That approach sat well
with confident staff who understood the process. For younger staff, or staff with no experience in
collaborative programme development, this came across as too fluid and with shifting boundaries.
So, on the one hand:
I think the people at SAHMRI were amazing, in terms of they were always
communicating, they were always open to new ideas, they were always open to
getting our feedback. And, they knew we were on the ground, and I think they
respected that they knew we were going to work. (S1 Page 4.)
I think it just makes you feel supported, and, you know, understood, because I think
a lot of people don’t really get what we do, and you’re sort of fighting this battle
around justifying what you’re doing, and justifying the programmes, and it was
just, they completely got it. It’s like as soon as we met them it was no, we
understand this, and we’re going to work around what you need, and that was
lovely. I don’t think I’ve seen that before, which was great. (S1 Page 5.)
78
They (SAHMRI) worked really hard to modify the programme. So they kept going
with it, and.....we had regular contact with them all the time, like, about, you know,
what would work, and – so that was really good to be involved in that, because we
could actually, like, I guess tailor it a little bit to our needs, and I guess that
authenticity kind of came through as it got modified, rather than being a
scripted.......it was quite different from how it ended up to how it started off at the
beginning. (S3 Page 3.)
WRC staff also provided additional one-on-one support that helped to change at least one staff
member’s view on the value of the action research approach grounded in the new, more flexible
model. That encouraged the staff member to deliver the programme differently than it had originally
been envisaged.
[SAHMRI staff member] has been a really big mentor for me and I think that that is
because she – I think I thought research was quite stuffy before I met her. I was
never really interested in it that much. I’ve been very much groundwork and it
wasn’t until I met [name]that she ......... flipped it on its head .......and, you know,
it’s not just about numbers and data, it’s actually about stories, and all the things
that you would hope it would be, but perhaps never gets to be because it’s harder
to measure. She was really impacting, I think, in encouraging me to use it in a really
authentic way, as opposed to the way that perhaps it was expected that it would
run in the first place. (S7 Page 4.)
However, some workers, perhaps not so familiar with the ‘adapt to your students’ approach, were
slightly confused about the programme and how to use SAHMRI-produced products. There were
differences in how the material was received and used by staff. In at least one instance, supervision
of workers became a significant pathway for implementation.
I think with this material everyone has taken it on in their own way, and it’s difficult
sometimes to get... the anchor points are the same, but how you get there looks
different, and there was a lot of people in that PD [professional development
provided through the Resilient Futures project], so I don’t think it was run as
successfully as it could have been. But I do clinical supervision with a couple of
people so I’m always using that kind of language, and I’ve always kind of used that
kind of language since it’s came around, because it fits really well with my values,
and so people in the team kind of pick up, you know, have you tried doing the
strengths survey with them, have you tried talking to them about this? I think that’s
kind of – in the immediate team, that’s kind of how it’s filtered out. (S7 Page 6.)
It appears that the change from a structured programme to a much more adaptable approach and the
lack of specific foci reduced the clarity for agencies regarding their participation. This may well have
been at least partially due to a lack of internal clarity within the WRC team about specific goals for the
programme.
I think probably (there has been) a lack of clarity sometimes in terms of what we’re
doing, what we’re trying to do. Like, our aims are quite broad. You know, it’s like
build wellbeing and resilience for – but actually, maybe as a team we needed to be
a bit more concrete in terms of right, for this next six months we want to look at
this, this and this. (WRC1 Page 14.)
79
5.2.3 Training for agency staff Two types of training were provided through the Resilient Futures project, resilience skills training
(TechWerks), and intentional practice (IMPACT) training. These were reported to have increased staff
knowledge and skill-sets, and were greatly valued.
(Initial 5-day training) - the best outcome was the staff training, that we had access
to all these thousands of dollars’ worth of training. .....there is a greater knowledge
and a greater skill-set amongst the staff to teach the skills (M2 Page 17.)
I do think it’s definitely making it a more informed part of their practice, a conscious
part of their practice and they’re perhaps looking at different ways to do it as we
bring and discussing different models… (M5 Page 9.)
…it did give us a new direction in case management. So that was really where the
intentional practice, … he put the RF10 skills into his intentional practice model,
that gave us, – or gave me a lot more confidence that the work that was happening
with the case managers would be beneficial for the young person… (M2 Page 19.)
The training was generally well regarded, but it was also widely suggested that the effects wore off
over time. Some agencies tried to continue the work of the Resilient Futures by organising and running
their own Resilient Futures in-house workshops for new staff or existing staff who had not been
exposed to the SAHMRI 5-day intensive training options. The results appeared to be some use of the
Resilient Futures skills but with lower levels of intentionality and intensity than graduates from the
original programme.
She (the team leader) started with the RF10 skills, and at the time I didn’t really get
it. Like, she might have given me a handout, I didn’t get it. Even after doing the
two-day thing, I sort of got it, like, some of the skills, the ones that resonated with
me, like the gratitude, the mindfulness, the ones that I was already embedding in
my life. (S11 Page 9.)
At one stage there were a few people that didn’t have the training, and they were
more the (support staff) and we offered an abridged form of training for them, but
they just didn’t have the full experience. They had less belief in the skills than we
did. And I remember at one stage trying to get a young person to use one of the
skills, and the (support staff) person said 'oh, you know, you know, this doesn’t
work'. Straight like that. But you know what, I could say ‘well, actually, there’s
scientific research says it does work’. So, that’s a good thing, because I’d already
done the university course and I’d done the Resilient Futures 105 and all this is
based on research....... So, I could squash that negativity. (M2 Page 10.)
In youth services where new staff were introduced to the Resilient Futures skills through internal
workshops, understanding and application of the Resilient Futures skills varied according to the type
of training attended (that is, those who received the SAHMRI training compared to those that received
shorter in-house training). One case manager spoke of her other colleagues who had received the
shorter training.
5 Resilient Futures 10 refers to the original package of 10 skills. Many agencies reduced the skills to 5.
80
I know that they’re using some of – probably one third of the skills, but not all of
them. And I know that the other amount of them, they probably would be using
them without realising, but it wouldn’t be intentional teaching, it wouldn’t be
intentional activities based on that skill, if that makes sense. (S11 Page 11.)
5.2.4 Agency support Being based in supportive organisations encouraged staff to practice what they preached, and some
agencies provided structures that reinforced staff learning.
We have Thursday morning meetings, and we’ll have prompts around the different
skills. This morning we were talking about emotion regulation, improving that in
terms of the students. We were talking about gratitude, we were talking about
mindfulness. So, it’s in our language and we’re talking about it all the time. So, I
guess that’s the sort of incidental support that I get. (S6 Page 4.)
So, the one-to-one is not every staff member has the same focus on it. So, some,
even though they have the exact same training as me, you know, we both had the
same five days training to start with, they haven’t really applied it as much. So,
they haven’t, for whatever reason, connected. So, then I have to remind them to
go back to it a lot, whereas others just run with it, and then they can – so it’s
individual differences in case managers, I suppose. (M2 Page 24.)
Management support was the most frequently identified aspect of agency support. In some youth
settings, staff have been relocated from the team that had undertaken the Resilient Futures initial
training. As a result, they were now working under new managers who had not been exposed to
Resilient Futures. The staff did not feel as encouraged and supported to continue with the work.
…If somebody (in this context: their manager) hasn’t done something (the training)
and they don’t really understand it, they’ll rarely encourage people to do something
they don’t understand, because they’ll just be like, well, I don’t know if you’re doing
it or not then. (S10 Page 6.)
However, for staff who had managers that came from a reflective practice background, this worked
as reinforcement.
I found that those people also had the greater buy-in. And I think it was the
capacity to talk about it, to reinforce, to build it, the shared language that we were
able to talk about so that really helped to embed it in their lives and in their
practice much more......and also that they themselves take it on board and apply
it personally and see the benefits for their own personal live, which then reinforces
the message. (M3 Page 3.)
Management who came from different professional backgrounds did not offer reinforcement in
Resilient Futures practice to their staff to the same extent, and this, coupled with staff who were not
as reflective in their own work practice, resulted in less investment in practising Resilient Futures in
their life and work.
Those who didn’t take it on, some of them I think probably aren’t as driven in their
own practice with people, so they’re not really reflective…...or don’t have perhaps
the structural support around them to be really reflective practitioners. So, they
might have a manager who’s not, a reflective sort of practitioner, but might have
81
some management, or have some experience in delivering the programme, but
might not be like a social worker, or a psychologist, or one of the professions where
there is really explicit reflection as part of the profession, if that makes sense. (M3
Page 3.)
Another respondent felt that there had not been much change in their organisation because the
project had not been actively supported by higher level management.
I would love to say, yes, but I don’t think it’s probably been embraced at the top
level … but they’re like, “yeah, it’s all great, do what you want to do with it”. (M5
Page 27.)
5.2.5 The role of the champion The importance of the role of the champion for Resilient Futures work became evident early on in this
evaluation (Resilient Futures Evaluation Report, 2017). While a formal definition of a champion was
not developed, champions were those people who actively supported the adoption of the Reslient
Futures programme and encouraged other staff members to be involved. Over time, however, the
agency role of some of the early identified champions changed and they no longer had the time to
support the work of Resilient Futures, as much as they had done during the early phase of the project.
The champions themselves, and the other staff, clearly identified the importance of a champion to the
ongoing success of this work. When the champion was also in a management position they were also
able to provide direction and support to staff who were not highly self-motivated in relation to
delivering Resilient Futures with young people. However, in areas where the champion’s role had been
reduced, this led to a reduction in motivation amongst the team.
I was able to focus quite a bit of time on the delivery, and supporting staff, and
really working on the why is this not working here, and let’s start here, and let’s
get people together and make sure we’re delivering it well, and supporting them
to make sure that they had the material as it should be, (but) that changed when I
took on a new role and some of the others aren’t self-motivated to be delivering
the material. (M3 Page 7.)
WRC staff who had observed the programme agreed about the need for champions to push Resilient
Futures within sites.
I think the agencies that were really successful were characterised by having some,
not all, really experienced, passionate staff. So, one of the things that’s been really
important to this programme is being able to identify where there was great
practice – examples of great practice happening that were championed by great
practitioners, who were then able to share that amongst the less experienced
colleagues and were in many instances able to leverage that great practice into
systems change. So, I think that’s been a real feature of the model, and that’s been
a pathway of the model. I’d say for those agencies that weren’t maybe quite so
successful, they didn’t have those practice champions. They didn’t have great
practitioners who cared about this, who understood it, and were able to lead
innovation and adaptation around it. (WRC3 Page 3 & 4.)
…having leadership who really care about the field of positive psychology, and have
seen results in it, so I think ...... we’ve had some good leaders. ..... I’m not
necessarily talking about the leaders at the top of the agencies…more about the
82
team of people who, yeah, what does their leadership look like, and how have they
supported, I think supported, but also given permission to try things out in this
space and get things wrong. So ...... there’s been some significant changes where
people have been allowed to fail, but the conversation has happened to go ‘oh,
being really explicit there about growth mindset didn’t work, let’s try something
else’. So just that ability to keep trying. (WRC1 Page 6.)
We looked at the champions first, rather than looking at the people who were less,
.... who were more resistant to this as an offer. We kind of went ‘oh, let’s focus on
who really – this really gels with to see if we can amplify what they’re doing, as
opposed to bring the others up’. So, I think there’d be more strategies we could do
in the future. (WRC1 Page 8.)
While there were examples of systems changes in these interviews large systemic changes
were not commonly reported.
WRC staff also noted that in one site, lack of support from a leader and a lack of commitment
to the project had inhibited implementation.
Then, the anti-champion. It will depend on the context, but perhaps it’s someone
who ... they’re not convinced about the value of positive psychology, or of these
kinds of skills as being useful for them in their practice. Or ...... perhaps they’re
someone who’s not been confident or ...... can’t see a way forward in terms of what
we’ve done with them after they’ve been to the two separate trainings. (WRC1
Page 12.)
5.2.6 Resources As well as working with agencies about the programme overall, the WRC developed a set of written
resources to support practice. In a school setting, the agencies were more likely to use the SAHMRI
resources if they were teaching an explicit Resilient Futures class.
We were provided with resources from a youth friendly version of a PowerPoint,
and some other resources to use, so I did rely on those a fair bit. But, it’s also, like,
sort of individualising it, so you know, you can teach it in a way that you feel
confident and happy about promoting it. (M2 Page 4.)
The need to use the resources depended on the confidence of the group and the ability of the teacher
to encourage and manage reflective practice in a group setting.
Sometimes the actual resources provided by SAHMRI, which was the youth friendly
version…they tried to make it fun by adding lots of video clips, but I certainly didn’t
stick to that. I might have chosen one or two things from it, so when they’re
introducing a skill, they might have had five video clips, I might do one or two. I
didn’t stick to that either because this group, I don’t have to entertain them, they
can talk like – they can talk for an hour about this stuff. (M2 Page 9.)
Some other, more confident teachers did not use the SAHMRI resources, but discussed with students
what commercial films or video clips would best trigger conversation.
83
5.2.7 Constraints Agencies identified a range of constraints that affected their ability to implement the flexible model,
to scale it out, and to sustain it in future.
5.2.7.1 Time
The first constraint was time. The less structured model required more time for organisations to
develop materials into a programme suitable for the needs of the organisation and the organisation’s
clients.
I did the training last year some time. I think it’s been really hard. It’s not something
we seem to have had time to then develop into an actual programme. I think the
intention was to bring some of it to team meetings. I think it’s something because
of our caseloads and the complexity, it just really hasn’t happened. Like, we try and
add bits and pieces. I did a presentation only the other day with (Manager), that
was going out to more of the (Agency) staff, though, not the young people, and I
think that’s the issue, I don’t think we’ve had time to really sit as a group and say
what are we doing with this? (S10 Page 1.)
At the moment we are in, I would say, a staff in crisis, where we’ve got staff who
are working excessive hours at the moment. So, when we’re talking about
work/life balance, and promoting that, and promoting healthy habits and stuff
with the children, we need to be practising what we’re preaching as well. So, if staff
were in a space where they could do that I think it would be easier......and we know
when staff are stressed and tired they’re probably not responding in the best way
they potentially could be, they’re not drawing on all their training and knowledge,
and everything… (S9 Page 9.)
A passionate champion in one organisation had no time allocated to offer training to other staff in the
agency. She could see all of the lost opportunities and was frustrated by them.
I believe it’s great what I’m doing (RF10), and I see it working, but I’ve got 22 kids
in my class. I want to do it bigger than that. I want to be able to do it not just with
the young people but with the staff that are touching the young people’s lives every
day, and I get approached quite often to be able to do that, but I don’t have
capacity. (S8 Page 6.)
For some workers a lack of time allocated to the merging of Resilient Futures work into their normal
workload was frustrating.
We’ve had to make it work with everything else that we do. We don’t get extra
time, unfortunately.....if you were able to have more time to be more.....(S7 Page
4.)
I think that if my case load were reduced and I had the time and space to do it, I
would absolutely love to. It’s not that I don’t believe in it, it’s just the time and
space. (S9 Page 7.)
I think that was what we were hoping, is that we’d get together, we’d work on how
we were going to deliver this, and then start delivering, I suppose. But we’ve never
really had the chance because there’s been so many changes. And even moving
office, and people leaving, and – I’ve had to do shifts in the houses because they’ve
84
been so short staffed, so it’s all these other things that have come into it, that it’s
sort of been put on the back-burner a little bit, I think. It’s like, it’s never really been
discussed, and because of the time pressures it’s – we just haven’t really had a time
to sit and work it out. (S10 Page 2.)
These groups also struggled to run intentional group sessions with young people, that would support
the one-to-one sessions:
…it’s not that we didn’t want to do it in a group, it’s just that we haven’t been able
to run another programme, because I mentor nine kids a week, so that’s 27 hours
out of my 38. (S11 Page 11.)
5.2.7.2 Costs of training
The second constraint was money. Intensive training for entire groups of staff, as well as on-going
support provided by SAHMRI during the project, was too expensive for some agencies. The main cost
issue was the cost of sending staff to Resilient Futures training, coupled with the cost of back-filling
staff while others were at the training. Some agencies also faced difficulties in backfilling staff
positions when workers attending training.
The way it’s worked here is - like, we had that – six of us had the five-day training.
So, that’s expensive. How can they – there’s a hundred teachers up there (In the
main school). So how are they going to (do it)? If you don’t have that intense
training, I think even people who had one or two days’ training probably wouldn’t
run with it as much. Because it’s one of those personal growth things. You go, you
go day after day, you have some things to think about at night, you have a lot of
personal growth in one week, which they couldn’t offer that to all the staff up there,
because it’s too many. $3,000 a head for 5-day training? (M2 Page 7.)
House staff – we’ve got teams, so it would say be between six and 12 team
members per house, and they’re rotational carers, so it’s 24/7 care. So, for us to be
able to pull out a couple of people to do just a two-day training is going to make
the house short-staffed......And we don’t have a big enough casual pool at the
moment to back-fill that, because I don’t know if you know much about residential
care but it’s really expanding, and there’s a lot of new organisations. … there’s not
heaps of people that are qualified to do the role, because there’s so many other
organisations that are employing for the same. (M4 Page 5.)
5.2.7.3 Management support
At least one interviewee felt that the managers at the organisational level did not seem to realise the
range of work and therefore the level of support required for effective implementation; that there
needed to be investment in learning; that the staff needed to be skilled in Resilient Futures in order
to respond to young people’s needs regarding Resilient Futures.
I think the organisation would really like for everybody to be using the personal
resilience survey in the same way, but I don’t know if the organisation understands
that it’s difficult for people to do it if they haven’t been trained in the skills. And so,
training people, other staff members and stuff obviously takes time and energy and
resources and stuff, and so there’s been less of a commitment perhaps from upper
management around that stuff. (S7 Page 5.)
85
Ongoing support and conversation was also identified as necessary to maintain a focus on the
programme, and resourcing for that also implies a requirement for management support.
I think it was probably more at the forefront of our mind, and if I’m being honest,
it’s like anything, once you’re not having that ongoing feedback, those ongoing
conversations pretty consistently, it can taper off, and a million other things come
up to get in the way of that. So, it’s kind of, I guess, broken down the team slightly.
We’re still obviously incredibly close, and working really well together, but it’s just
finding the time on a regular basis to have those conversations. (S1 Page 14.)
5.2.7.4 Collaborative practice
Another constraint lay in working relationships with other agencies. Working with Resilient Futures in
a semi-closed environment, such as a small alternative education setting, meant that the approach
could be consistent, with all staff using the Resilient Futures skills and language. In one agency, this
was working well until there was a change in policy and an alternative education model was brought
in. This meant that new relationships with different agencies had to be developed, and this made it
harder to keep going with the Resilient Futures model, as the other agencies were not familiar with
the Resilient Futures language and intent. In this example, the main school adopted a philosophy of
Connecting Communities and started doing much more work with community groups. These
community groups were not familiar with Resilient Futures and it was not as well accepted and started
to get lost.
It’s not losing control, it’s just you’re having to integrate different people’s
philosophies, and, like, their approach to our learning programmes. It’s really
interesting. (S1 Page 18.)
5.2.8 Sustainability Over time, the effect of the initial Resilient Futures off-site training started to wane, as time lapsed
after undertaking training in Resilient Futures. Staff changes and other priorities taking precedence
also impacted the levels of support and enthusiasm for the project over time. Interviewees agreed
that bits and pieces of the Resilient Futures Project had stuck, but in some cases, that was all. It had
become watered down over time and it was agreed that time allocation was critical to the on-going
support of Resilient Futures.
While the initial training was greatly valued, it appeared that the impact waned over time, particularly
in relation to enthusiasm.
…staff move on and enthusiasm waned... we’ve had three or four leave. We’ve
done our best to retrain them, but it’s not that original (training)....it’s a much
abridged version of that.......there really was a lot of collaboration in the nine
months, really, after you do a training, probably after nine months, and then it just
dies off. It goes on the back-burner a little bit. You know, you can’t maintain
enthusiasm probably for – I think it’s amazing we maintained it for nine months.
(M2 Page 17.)
I think Resilient Futures loses momentum. I think for me, when I learnt it, I was so
on fire for Resilient Futures, and such a devotee, you could say. But I think over
time, it’s almost as if they need something new. You’ve done the initial
training…They need to do something to reignite……that initial enthusiasm for the
training, because I think we get a bit bogged down sometimes in our work. I think
86
it’s really everything else that takes priority over that.........and I think sometimes
we just need to be reminded. I think a reminder training would be fantastic because
we could, as I said, reignite that enthusiasm and passion for it. (S6 Page 11.)
Where the initial enthusiasm had not been supported by whole-of-team implementation, the effects
were much shorter-lived.
I found it (initial 5-day training) very valuable at the time, and I can’t really
remember a whole lot of the training now because I haven’t revisited it for a long
time. I think when you do, like, a first aid course or whatever, it’s very once off and
you know, three days later, you don’t know what to do with a snake bite because
you haven’t revisited it. Whereas with other training, if it is what you live and
breathe within the organisation, everyone’s using the same collective language,
and you’re talking about it at every meeting, it makes it a lot easier to remember
what you’ve learnt and to put it into place every single day. I haven’t been able to
do that, so it was valuable at the time but without having done that, it – it’s not for
me at the moment. (S9 Page 10.)
Additional training was suggested as a way of maintaining interest and enthusiasm for staff as well as
keeping the skills and concepts fresh in their minds.
… it’s about practice, everything’s about practice. The more practice you do the
better you get at it, so any training that you can get that supports that is just – is
gold, really, you’ve got to keep doing it all the time. It’s not just a one-off thing......if
we could go to training and stuff like this every – yeah, at least every six months,
that would be awesome, to go and sit in a camp, because it just energises you. (S2
Page 17.)
Even though we do that stuff, if you – if it’s not made explicitly aware to you from
time to time that that’s what you’re doing, then you forget. (S2 Page 19.)
As shown in the following examples, changes in personnel or practice within agencies impeded further
implementation or diffusion of the programme.
In one organisation, a priority afforded to SACE accreditation and lack of understanding of what the
programme really entailed reduced commitment to participate by a supporting agency.
… and it eventually became so ‘this is your book, if you want your SACE credits, ah,
complete this form, um, and basically the answers are in green’ … So there was no
sort of real delivery around, like, these are your strengths, and these are your
values, or and if those are your values this is where, you know, you could lie, or,
um, yeah, nothing really around resilience building. So yeah, so we opted to step
back from that, because we said we just didn’t want it to be – we didn’t want to
have that tokenistic role in it anymore. (M6 Page 2.)
Another agency discussed how they had originally been trained in Resilient Futures as a group and
then they were split up and moved across streams within their agency, so that there was less
opportunity to do the resilience work.
A third agency did not have external funding renewed for some of its programmes, which was where
Resilient Futures was being implemented, so all the Resilient Futures work was lost.
87
We’ve been working on this for three years, and places have radically changed in
their own right, because of funding, because of certain people leaving, or certain
people joining, or some of these things have gone through massive restructures.
(WRC2 Page 23.)
The transition from a defined and manualised programme to a more flexible model which resources
teaches and agencies to develop their own ways of working may also have had impacts for
sustainability.
I think just the way this project was sort of designed and funded, it had to go big
quickly, … but if we’d decided to find a real niche where we think this can fit a
problem, we might have had more success going forward. We’re not established
within the (education) system in any way, and I think that’s a problem for the
project going forward.....at the moment it’s really tough for us to say 'what are we
as a programme?' We’ve given a lot of flexibility out there, what is it that we as a
programme are actually contributing? When we’re a training programme, and
people come, they do our training, they get some content and some books, and
whatever else, and they go off and they go, and they’re happy that they’ve paid
their money. But if they come along to a training programme and then have to do
some consultancy stuff, where we go and consult with them to work out how’s it
going to work, ........then it’s much less of a clear product, if that makes sense. A
product in the sense of it’s something tangible that people, funders, schools,
government can understand and get behind. (WRC2 Page 19/20.)
While change in partner organisations is inevitable, the relatively small size of the programme and its
structural separation from delivery agencies appears to place it at risk. It seems likely that it will be
necessary to concentrate on embedding resiliency training in a particular system (for example, the
FLO system), in the first instance. This would enable the programme to establish depth and stability
(albeit at the expense of breadth), providing a base for future expansion.
88
6 Contexts and Mechanisms In this section context-mechanism-outcome configurations (CMO’s) are presented and described. In
realist evaluation, mechanisms describe underlying causes of outcomes, and contexts describe the
circumstances which affect whether or not those mechanisms operate.
The 2017 evaluation report provided a number of CMO configurations which were divided into four
groups. These groups related to selective targeting, reinforcement, satellites (contexts where a
trained staff member was not working with other trained staff members) and credibility and trust.
For the 2018 report, the CMO’s have been divided by the groups for whom the outcome is achieved.
They have been divided into CMO’s for young people, for workers and for agencies. Within those
groups the topics (selective targeting, reinforcement, satellites and credibility and trust) have been
used. An additional grouping, relating to effective and appropriate delivery has also been added.
Some CMO’s reference both young people and workers. The use of personal examples by workers to
improve credibility and trust and thereby increase young people’s engagement is an example of this.
The outcome in this instance is for the young person (engagement), and so the CMO is placed in the
young people’s category.
It is worth noting that there were no CMO’s which were developed out of the 2017 round of evaluation
that were contradicted by the 2018 round of evaluation. In the sections below the CMO’s that were
developed on the basis of the 2018 round of evaluation are shaded grey.
6.1 CMO’s for young people
6.1.1 Selective Targeting Effective targeting of young people to participate in Resilient Futures required staff members to have
good knowledge of the young people they were working with so that they could judge which young
people would be likely to engage with the material and choose the most suitable mode of delivery for
young people. For group settings it also required that staff had knowledge of peer relationships to
minimise disruptions.
Context Mechanism Outcome
Existing knowledge of potential participants by staff Group delivery of programme Selective targeting of young people who could work well together Selective targeting of readiness to change
Positive peer relationships Reduced disruptive behaviour in groups
Existing knowledge of potential participants by staff Group and individual programme options available
Good ‘fit’ between programme mode and individual learning style
Young people enabled to learn effectively
89
Selective targeting of group or individual delivery
Good fit between programme content and individual needs
Increased regulation of behaviour and reduced disruption in groups
In the 2018 round of interviews, another dimension of the impact of selecting students who would
work well together was identified. To be able to work effectively in groups there had to be trust
between participants. This enabled young people to make themselves vulnerable and engage with
the material.
Context Mechanism Outcome
RF provided in groups with high levels of trust
Young people feel emotionally safe
Young people are willing to engage and speak openly and honestly.
In contexts where agencies provide resilience training to all young people with whom they work, it
would not necessarily be possible to select young people who would work well together and targeting
young people who were ready to change would not be possible at all. That would suggest that the
programme would be less likely to be effective where agencies were restricted in their ability to select
young people to be involved as there would likely be greater variation in the suitability of the young
people involved and more variation in the quality of relationships between those involved.
6.1.2 Reinforcement The concept of reinforcement as a mechanism was identified in CMO’s relating to outcomes for both
agencies and young people in the 2017 round of interviews. Having multiple staff in an organisation
using the skills and common language were seen as reinforcing both skills and language for young
people and thus contributing to strengthened outcomes.
Young people’s previous experience with the language of positive psychology could impact in either
negatively or positively. Where the previous experiences were positive there could be a
reinforcement effect for prior learning and the young person may be more likely to engage. Where
there previous experience was negative, the similar language could cause the young person to
disengage with the programme.
Context Mechanism Outcome
Multiple staff in the organisation are using consistent language/skills in interactions with young person
Reinforcement of skills/language for young person
Strengthened outcomes for young people.
Young people with previous positive exposure to the language or concepts
Reinforcement of previous learning Triggering of previous positive experiences
Improved outcomes
Young people with previous negative exposure to the language
Language triggers reactions based on previous experience
Young people disengage
90
6.1.3 Credibility and trust Credibility and trust were two closely linked ideas that were reported in the 2017 interviews as being
important to engaging young people. There were two distinct aspects of this. The first related to the
credibility of the worker and the young person’s trust in the worker. The second related to the
credibility of the skills and the young person’s trust that those skills could provide positive outcomes.
Regardless of the form of delivery (teaching or mentoring, group or individual work), both forms of
trust were enabled by workers using the skills in their own life and being able to provide personal
examples of using the skills.
Context Mechanism Outcome
Workers use skills in their own life and are able to provide examples to young people
Increased credibility of and trust in workers by young people
Increased willingness of young people to engage
Workers use skills in their own life and are able to provide examples to young people
‘If it worked for them, it can work for me’ – increased credibility of skills/project
Increased willingness of young people to engage
In the 2018 round of interviews the counterpoint was identified whereby a lack of staff engagement
with the skills hampered their credibility and did not engage young people. As this is effectively the
opposite of the CMO as outlined in 2017 it supports that configuration.
Initial success with a skill was also suggested to help build the credibility of the wider skill set and, as
that credibility increased, young people were more likely to seek further instruction.
Context Mechanism Outcome
Staff not engaged, not using skills themselves
Lack of credibility with young people
Young people not engaging/trusting the facilitator
Young people have success with use of a skill that positively affects their life
Increased belief in the wider skill set
Young people ask for further instruction
6.1.4 Appropriateness Multiple interviews in the 2018 round of data collection identified that the method of delivery needed
to be appropriate to the needs of the young people. ‘Appropriateness’ consists of many elements
including group or one-on-one delivery, implicit or explicit delivery and using language that was
relevant and understandable. Another element was that young people working in groups had high
levels of trust and a feeling of safety within the group, which links back to the selective targeting of
groups to work together.
One of the common threads throughout determining the appropriate delivery methods was safety
and trust. Without these it was difficult for people to make themselves vulnerable, which was
required to deal with many of the issues which the resilience skills were used to address.
91
Context Mechanism Outcome
One-on-one delivery for young people with high levels of anxiety, traumatic backgrounds or in crisis.
Safety (lack of judgement, do not need to expose issues in front of group)
Young person engages with facilitator and content.
One-on-one delivery for younger/less mature people
Young person can learn at their own pace, work at own level
Young person engages with the content.
Group delivery in groups with high levels of trust
Peer reinforcement, sharing of ideas ‘Not the only one’
Improved engagement and learning outcomes Less sense of isolation, improved understanding of peers
Young people feel safe and a sense of belonging
Young people feel more confident about trying the RF skills
Improved outcomes
Young people with previous negative experience with positive psychology; Implicit/ ‘disguised’ delivery
Does not trigger previous negative memories
Young people do not ‘switch off’, continued willingness to engage
Good fit between programme content and the developmental needs of participants
Recognition of value by young people, willingness to learn/use
Increased self-regulation of behaviour
Young people engage with programme
Improvement in self-belief and positive self-talk
Increase in persistence and attainment of goals
Young people are in crisis Lack of mental capacity to deal with learning new skills
Lack of outcomes
Young people who have previously learned resilience skills are in crisis. Staff cue young people regarding use of strengths and resilience skills
Reminders enable young people to access previously learned skills or tap into different beliefs
Skills are used to address crisis; outcomes are improved. Positive outcomes increase chance of future use.
Young person has need of specific skill Staff ‘seizes the moment’, takes opportunity to teach skill when needed
Young person sees immediate relevance
Young person engages with the skills and can apply ‘in real time’.
6.2 CMO’s for workers
6.2.1 Reinforcement As noted under the CMO’s for young people, reinforcement was linked to outcomes for both agencies
and young people in the 2017 round of interviews. In the 2018 round of interviews two additional
CMO’s were identified that linked reinforcement to outcomes for workers.
92
For staff who were already familiar with the resilience skills and or positive psychology being part of
the project reinforced their learning prior to having been involved in Resilient Futures.
Where staff were already using resilience skills in their work with young people they were likely to feel
supported or that they were justified in teaching those skills as it was supported by previous research
and a credible backing organisation. This linked to staff being able to better justify what they were
doing and why as well as having greater confidence in the approaches they used.
Context Mechanism Outcome
Staff are already using resilience concepts in their life and or work
Staff identify with training material
Reinforcement of concepts and skills
Staff are already using resilience skills in their work
Staff feel supported and/or justified in what they have been doing due to the support of SAHMRI and the evidence upon which the project is based.
Staff are more able to justify what they do. Staff have greater confidence in what they do.
As noted in the 2017 report, a second round of training had a positive impact on enthusiasm and could
help to alleviate two issues: staff turnover and fading engagement over time. In the 2018 round of
interviews, one manager reported that the passage of time was linked to a reduction in enthusiasm
from the staff and then to less focus on the Resilient Futures project.
Context Mechanism Outcome
Time elapsed since initial training and less intense follow up by SAHMRI and colleagues
No longer new, less enthusiasm Less focus on RF by staff
6.2.2 Satellites In 2017, some staff members were physically located away from the rest of the staff within their team
or were the only staff member within a site to have been trained. This was identified as a context
which inhibited the implementation of the Resilient Futures project by those workers.
The exception to this was where an individual was highly motivated and an enthusiastic driver of the
project. Even in this situation, however, the project was not implemented as a Resilient Futures
programme, but rather incorporated aspects of the worker’s learning from the programme and
aspects of additional learning they had undertaken.
Context Mechanism Outcome
One or few staff within an office trained in Resilient Futures
No reinforcement of concepts/language for staff Reduced capacity for collaborative work
Reduced capacity for implementation
One or few staff within an office trained in Resilient Futures Trained staff member with high existing expertise
Personal commitment to positive psychology concepts Understanding of Resilient Futures skills and tools
Implementation within project operated by trained staff member, but not other programmes in the site
93
This was not further investigated during the 2018 round of the evaluation and no further CMO’s were
identified from the 2018 interviews.
6.2.3 Appropriateness For young people, the concept of appropriateness related largely to delivery methods being
appropriate to the needs and stage of development of the young people. For workers, the concept
relates more to an alignment between the workers own views, existing knowledge and or experiences
with the information and resources provided through the Project.
Where the project philosophies aligned with those of the staff member they were more likely to apply
skills learned through the project.
The level at which the training was pitched also needed to be appropriate, with higher level
development required for more skilled workers, and more structured content requested by less
experienced workers.
Context Mechanism Outcome
Alignment of Project and staff values/goals
“it fits really well with my values”
Staff member applies learning from training.
New skills and concepts provided through training
Staff understanding and knowledge increased
Increased ability to teach skills
6.2.4 Other Staff who applied the skills in their own lives reported that it had similar outcomes to those expected
for young people including improved personal relationships. They also noted that it improved their
credibility with young people, as discussed above.
A lack of implementation of learning (or a lack of learning) from the training was, unsurprisingly linked
to no change in practice. This could be due to a range of factors, including a lack of belief in the
project, lack of time or not having fully understood the training. Where the issue was not staff being
opposed to teaching the skills, it was linked to staff wanting more explicit instruction on how to deliver
the project.
Context Mechanism Outcome
Staff receive training and apply skills in own life
Experience success with skills Improved personal relationships Able to provide examples from own life, improved credibility with young people
Staff not applying the learning from RF training in class/mentoring
Staff not internalising RF skills Staff want step by step instruction (similar to phase 1 design, with ‘a manual to follow’)
Confidence to apply not developed.
94
6.3 CMO’s for agencies
6.3.1 Reinforcement One of the key messages that came out of the 2017 round of interviews was the importance of
common language within organisations. The development of a common language was seen as coming
out of having multiple staff undertaking training. Shared language was seen to contribute to
consistency of practice across the agencies.
Context Mechanism Outcome
Multiple staff receive training and discuss Resilient Futures in the workplace
Reinforcement of concepts/language for staff
Shared language and consistent agency practice
This was supported by respondents interviewed in 2018, who further suggested that common
language was important within agencies as it led to clearer communication between staff. WRC staff
reported that some agencies had introduced new policies and wellbeing and resilience management
roles however this was not discussed by interviewees from the agencies.
Context Mechanism Outcome
Multiple staff receive training and discuss Resilient Futures in the workplace
Common understanding of ideas and definition of terms
Fewer misunderstandings between staff and greater ability to be ‘on the same page’
Agency is already applying positive psychology concepts or skills
Managers and staff feel supported and or justified
Increased confidence at multiple levels within agencies in approaches used in the agency
In addition to the passage of time noted in the CMO’s for workers, two other contexts were identified
in which reinforcement had broken down. These were new priorities with organisations and staff
turnover. While there will always be competing priorities within organisations, the impact of staff
turnover could potentially be lessened by additional training courses. These could provide initial
training to new staff or act as a refresher for staff who had undertaken previous training.
Context Mechanism Outcome
New priorities and new demands on time and RF project becomes a lower priority
Staff don’t give RF as much thought as there is less reinforcement and less drive from managers.
Less focus on RF across organisation
Staff turnover with RF trained staff leaving and non-RF trained staff arriving
Different knowledge base for different staff, staff feeling less supported in RF
No common language or common understanding of skills and concepts
Another factor which could negatively impact reinforcement is a lack of commitment from staff to the
Resilient Futures project. Where staff do not believe in the value of the project and therefore do not
engage, it becomes less likely that the information and skills will be transferred between staff and so
the project does not diffuse across the agency.
95
Context Mechanism Outcome
Staff are not committed to the RF programme they are running
Lack of engagement by staff RF is not spread across the agency
6.3.2 Other Two additional CMO’s were identified that related to agency level outcomes. The first linked to
management support, and the second to agencies working together to deliver programmes.
Support from management or senior staff contributed to implementation by making staff feel both
accountable and supported. Having multiple staff trained and using the skills linked to improved team
relationships and the approaches described in the Resilient Futures project becoming embedded in
agency practice.
In a context where multiple agencies with different goals, knowledge levels and or approaches were
working together it created some confusion and tension between staff and it was difficult for those
agencies to work together.
Context Mechanism Outcome
Committed leadership or champion acting as an external driver for staff
Staff feel accountable and are supported with resources and direction
Staff implement the RF programme
Widespread staff involvement and engagement
Consistent approaches, common language
Resilient Futures embedded as part of how the organisation works
Staff use skills when dealing with each other
Staff feel supported.
Stronger team relationships
Multiple agencies working together with different goals and different levels of understanding/knowledge of RF
Creates confusion and tension Difficulty working together on RF
6.4 Updated Theories of Change As previously noted the evidence gathered for this evaluation largely supported the programme
theory model described in Section 1.4. There were, however, some adaptations to be made. The
diagram below outlines the updated model. Where links between boxes have been added or changed
they are in blue. Similarly, the boxes highlighted in blue have been added in response to the findings
from the evaluation.
Trust and credibility emerged as key factors in successfully engaging young people and so the link
between workers using the skills in their own life and the willingness of young people to engage with
the material has been added to the programme overview.
Achieving personal goals was added as an outcome for young people. These personal goals could be
similar to the highest level goals (retention in education and training, improved life outcomes), for
which the evaluation was not able to provide evidence. They could, however, also be lower level goals
such as improving a particular relationship or dealing with a specific issue that is causing difficulties.
96
Under the worker stream, one box was removed (workers within teams holding each other to account)
as the evidence from the interviews suggested that there was a direct link between the development
of common language and concepts and consistent practice within teams rather than improved
accountability linking the two.
The engagement of young people with the project as a whole was suggested to work through agencies
promoting Resilient Futures to young people and those young people trusting the workers and valuing
the intended outcomes of the programme. There were two issues with this. Firstly, workers reported
that at times they used implicit teaching strategies and ‘hid’ the resilience skills which suggests at least
some young people were being exposed to the skills or concepts without necessarily volunteering to
be part of the project. Secondly, participation was not always actually voluntary. One site provided
resilience programmes to all young people they worked with and as the young people were required
to choose at least one elective which contained aspects of the Resilient Futures project, their
participation was not voluntary. The boxes relating to this have therefore been removed. It does,
however, remains possible that voluntary participation leads to the greatest impacts. There were
examples in the earlier years of non-voluntary participation being linked to poor engagement.
97
A more detailed theory of change similar to that produced by the WRC team at the beginning of 2018
is presented below. Whether young people are introduced to the programme through group or one-
on-one setting trust, a sense of safety and both willingness and the ability to learn are critical to
enabling young people to engage with the material.
Trust and credibility have been discussed multiple times and contribute to a young person’s
willingness to engage. Willingness to engage will also be impacted by previous experiences with
similar material, both positive and negative, and whether the young person is ready to make changes.
Ability to change is linked largely to emotional maturity, the level at which the material is pitched and
whether the young people have other, more pressing issues such as homelessness or other crises that
reduce their capacity to engage.
Once young people have engaged, the path of expected outcomes is relatively straightforward.
Learning the skills is followed by applying the skills, experiencing success with the skills and then
incorporating them into their daily life. Prolonged use of the skills is expected to lead to greater self-
regulation, improved resilience and improved persistence which, in turn, link to achieving personal
goals and greater overall wellbeing. Again, these higher-level outcomes could not be tested in this
evaluation.
98
As noted above, there are several elements which can impact whether a young person engages with
the material. The diagram below provides examples of how these elements can prevent engagement.
The diagram may provide clues about issues to be addressed in any future development of the
program.
A theory of change for workers and agencies was also developed which shows the expected pathway
for the programme to impact workers and to become embedded as an agency approach. Training
workers and adapting the programme to be delivered to the needs of the agency and the clients are
both necessary to the development of a whole team approach. As with the overall model, common
language, improved communication and consistent messaging are important to attaining the desired
outcomes. As engaging young people and higher quality implementation are both linked to improved
outcomes for young people this theory of change also links to the theory of change for young people.
The potential outcomes include personal outcomes for workers, learning of new skills and concepts,
validation of the ways in which workers and agencies are working, development of stronger teams
with better communication and Resilient Futures becoming embedded as the default approach of the
agency in relation to strengthening young people’s resilience.
99
As with the theory of change for young people, there are a number of things which can reduce the
impact of change for agencies. These include a lack of staff engagement and potential opposition to
the project, lack of time, unsuitable delivery methods, competing interests and staff turnover. These
act as inhibitors to change and prevent both the development of common language and whole team
approaches. Where a worker actively opposes the project there is also the potential for negative
impacts on team relationships.
100
101
7 Conclusion As with the section on contexts and mechanisms, findings from the previous report have been
included here as they pertain to the project in its current form and were either supported by the
new data or not challenged by it.
The findings from this round of interviews (2018) and the previous round of interviews (2017) largely
support the current Resilient Futures Project model. The following elements of the model were
supported:
• Flexible approach balanced with intentional practice: Staff were supportive of the shift from
a highly structured, manualised project to the current, more fluid approach where
adaptation was encouraged. Adaptation was undertaken at organisational and individual
levels and included both structural and content changes. This does suggest that staff need
to be highly capable and comfortable with the content in order to be able to adapt it
appropriately.
• Voluntary involvement of young people: Involvement for most young people was either
voluntary or involved an element of choice (one organisation had young people choose one
of five electives that all incorporated Resilient Futures elements). Where participation was
not voluntary, greater resistance and disruption was reported. Interestingly, the two
organisations from which the greatest amount of quantitative data was available did not
have entirely voluntary involvement for their students. Both were in schools and included
resilience skills in ways that meant all students were included. In one, it was by causing
students to choose one of a set of electives, all of which included Resilient Futures work. In
the other all the students received case management and Resilient Futures was included
through that. These examples did not appear to have the same issues with forced
involvement as when young people were told that they had to participate in a specific
Resilient Futures course.
• Multiple workers trained, development of common language: In all organisations multiple
workers were trained. There were, however, some workers who did not work onsite with
others that had been trained. In sites where multiple workers had been trained, the use of a
common language was reported. This was not the case for workers who were based in sites
where others had not been trained. A lack of reinforcement was also reported as an issue
for those workers. Common language was reported as important to ensure that
communication between staff was clear, communication with young people was consistent.
It was also noted that having consistent language made it easier to identify young people’s
learning.
• Increased consistency in agency practice: This was reported where multiple workers had
been trained and, in particular, for one organisation where all the workers at the site had
received training. Over time, however, the respondents have started to notice a dilution of
the skills at the sites, as staff move on, or change roles and new staff enter the site who have
not undertaken the 5-day SAHMRI training. There was concern that agencies could never
afford the intensity and level of training that SAHMRI had offered, nor afford to replace staff
102
while they attended such an intensive course. This consideration only added to their
concerns as to the future of the Resilient Futures Project.
• Trusting the worker: In the programme theory model, young people’s trust in the workers
was seen as necessary for young people to participate in the project. Multiple respondents
reported that it was also important in creating a willingness to listen and take on board the
training. Two critical factors to creating trust were pre-existing relationships between
workers and young people, and workers’ use of the skills in their own life (which increased
credibility through young people believing that workers used and believed in the skills).
• Young people learn skills: There were multiple respondents who could identify outcomes for
young people. These ranged from increased knowledge of concepts to using the skills in their
own life (outside of their interactions with workers). There is no data at this point about
whether there was greater retention in relation to education or training or improved life
outcomes, but nor would that be expected given many of the programmes were not complete.
Long term outcomes such as these could only be examined through a longer-term evaluation.
• Reinforcement: This was reported as important for both workers and young people. For
young people, reinforcement through exposure through multiple staff members and
through different types of delivery (group and one-to-one) was linked to improved uptake.
For workers it was linked to embedding and using the skills in their own lives, which, in turn,
aided in creating credibility. It was also linked to the use of shared language both in that
workers embedded the language and that the shared language aided reinforcement for both
workers and young people.
Staff were more likely to embed the skills into their own life in contexts where multiple staff
had been trained and when the staff had a personal belief in the skills that were being
taught.
One worker, who did not work in a site where multiple staff had been trained, suggested
that a refresher training would be useful as they could not remember all the concepts, skills
or particular language. Further training or support for workers who are not in a team
environment or in an environment where they are the only one who has been trained could
be considered moving forward.
• Selective targeting: A number of factors were raised by respondents regarding selecting
potential participants. For both group programmes and one-on-one mentoring, selection of
young people with a level of emotional maturity and readiness to change (or at least listen)
was considered important. In group settings, the interaction of specific young people was
also a consideration in attempting to positively influence group dynamics, increase trust and
reduce disruption.
• Implicit and explicit teaching: Explicit teaching was seen by some respondents as more
suited to classroom settings and group work while implicit teaching was suitable for one-on-
one interactions. However, both explicit and implicit strategies can be used within both
group and one-to-one settings. Implicit teaching was most necessary when the young people
had previous negative experiences with positive psychology or would be resistant to the
103
formal language. Effective practitioners therefore needed to be able to swap between
explicit and implicit strategies according to the specific circumstances of the young people
and their responsiveness at different times. This requires a high level of expertise and
confidence on the part of staff.
• Provision of training: Provision of training was a critical strategy for getting staff to teach
resilience skills, or incorporate them in their practice. and in different ways, for those who
were. For staff who did not have a previous grounding in resilience skills or positive
psychology, the training provided skills, knowledge and resources which they could use. For
those who were already familiar with the content, the training provided a refresher on the
concepts, reinforced their practice, and in some cases ‘tweaked’ their perspective and
practice. Widespread training within a team or organisation facilitated a common
understanding of the skills and concepts and supported the use of common language.
One aspect of the programme model for which there was no evidence from the interviews was that
the development and use of common language and concepts would lead to workers within teams
holding each other accountable and that in turn would lead to increased consistency of practice.
Respondents instead drew a direct link from use of common language and concepts to more
consistent practice within teams.
There was also mixed evidence in relation to the young people’s engagement with the programme
occurring when agencies promoted their resilience programme to young people and the young
people trusted the worker and valued the intended outcomes of the project. The use of implicit
strategies designed to ‘hide’ the teaching of resilience skills for some young people suggests that this
was not the case, at least not in all settings. There were also sites where young people did not
actually volunteer to be involved as all the young people being worked with at that site were
exposed to the project. Trust and valuing the intended outcomes were, however, reported as
important factors impacting whether young people engaged with the material when they were
exposed to it.
Respondents reported outcomes for young people, workers and organisations. One of the key
outcomes for some workers was using the skills in their own life. This linked to outcomes for young
people by increasing the workers’ credibility. Encouraging this process to continue as new workers
are trained could improve outcomes for the young people they train. However, respondents
reported that when new workers had been trained in-house, the courses were shorter than the
original SAHMRI training and less intense, and there was little time for reflective practice.
Respondents noted that this resulted in less use of the resilience skills in practice with the young
people and where it was used, it was much less authentic.
Outcomes for young people included increased knowledge, changes in behaviour during group
sessions, the ability to talk through incidents after the fact and identify how they might have acted
differently and for some, perhaps most impressively, use of the skills outside of the interactions with
workers.
The quantitative data showed modest, statistically significant improvement on both the PERMA and
K10 scale scores incorporated into the student survey and statistically significant improvements for 4
of the 18 individual questions. There were no scores for which there were statistically significant
worsening of scores from T1 and T2.
104
On the whole, the qualitative data was more positive than the quantitative data which could be due
to a range of factors. This could include bias from respondents who wanted (consciously or
subconsciously) to highlight the value of what they had been doing. The bulk of the qualitative data
came from staff members across all of the agencies while the quantitative data was from young
people. Alternatively, it may simply be that the outcomes discussed in the interviews related to
specific examples of learning and use of skills, while the quantitative survey addressed the emotional
state and wellbeing of the young people and learning. It may be that using the skills tends to
precede broader changes in wellbeing and not enough time has passed for those changes to be fully
evident. This was particularly likely in circumstances where the young person had not completed
their exposure to the course. Low numbers of respondents from some agencies, and inability to
implement the questionnaire in others, reduced the potential for more detailed analysis
Common language and increased consistency in agency practices, both important aspects of the
programme theory, were also reported by workers. Both of these helped to create understanding
between staff and aid in articulating what they were doing and why. They also helped to create
consistency for young people when dealing with multiple members of staff.
7.1 Future Directions The results of this evaluation were for the most part positive. Most staff and organisations involved
with the programme believed it to be effective for at least some students and positive outcomes were
reported for both young people and staff. The survey data for young people showed improvements
for both self-reported wellbeing and psychological distress. While the improvements were small, there
are two additional factors to be considered. First, the change was measured over one semester and
it is possible that stronger change would be seen over the course of a full year. Second, there were
no statistically significant negative changes in the survey results.
This suggests that the programme is of value. A number of considerations for the future of the
program were identified through the evaluation. In this section we discuss these issues which are of
relevance should further funding for the programme be obtained.
The evaluation concludes that that the programme warrants continuation.
The evaluation further recommends that strategies for mainstreaming resilience skills training within
FLO, re-entry schools and youth services be considered.
7.1.1 Working with families Young people from families who did not have knowledge of or use Resilient Futures language, or who
did have knowledge of the language but opposed it, were identified as a group for whom the
programme was less likely to be effective. On the other hand, young people from families who were
supportive of and used the language were more likely to see positive outcomes.
Working with families, particularly those that do not have knowledge of the language or oppose it,
could therefore improve outcomes for young people by moving the families from less to more
supportive. This would require buy-in from family members and would necessitate additional work
from facilitators. The nature of the cohort involved in the programme suggests that it is possible,
potentially even likely, that the relationships between young people and their families may be less
stable or more strained than the general population and so careful consideration would have to be
given to strategies to engage the families.
105
7.1.2 Working with young people in crisis The general response from interviewees was that the programme was more likely to be effective for
young people whose lives were relatively stable and not in crisis. There was, however, some evidence
that skills learned through the programme could be employed to help young people work through
crises. It was also suggested that it would be possible to teach young people resilience skills even
when they were in the midst of ongoing difficulties such as homelessness (although presumably this
may be easier for those who are in shelters, with a less immediate sense of crisis and greater access
to support workers).
Most respondents, however, argued that teaching the skills is less likely to be effective for young in
crisis. Some implicit teaching may be possible through counselling programmes and crisis intervention
strategies.It seems likely that new ways to develop resilience may be more effective in organisations
such as homelessness services, and other crisis services.
7.1.3 Working with younger young people While the survey data did not provide a clear pattern of impacts based on the age of respondents,
many interviewees suggested that the programme was likely to be more effective for the older end of
the age group in this program, and those who were more able to think abstractly.
It was also suggested that a version of the programme targeted to younger young people (10-13 year
olds) could potentially provide them skills to help manage issues during their middle and upper teens.
Working with these target groups would likely require the development of a less cognitively focused
version of the programme that concentrates more on behaviours and less on the theory behind why
those behaviours are important. This is consistent with Piaget’s classifications of the differences
between formal operations – the ability to abstract and hypothesise – and concrete operations.
Teachers of younger cohorts often have expertise in developing teaching and learning programs for
those operating at a concrete level and may have much to offer to development of such a program.
7.1.4 Working with agencies to minimise disruption Disruption caused by changes within organisations impeded the ongoing implementation of Resilient
Futures within some partner organisations.. This took the form of people (particularly leaders)
changing roles, changes in priority as other programmes were brought in and staff turnover.
Organisations were complementary about the support provided by WRC staff. It could be that an
expansion of this support to work more in depth with partner organisations specifically on minimising
disruptions to the programme could help to alleviate these issues. Specific strategies could include
assistance with handover of responsibilities relating to Resilient Futures when staff change roles and
further training opportunities for new staff. Additional training opportunities could also provide a way
of refreshing training for staff who had already been through the training.
Managing the impact of other foci and programmes is perhaps more complex. In all organisations
there will be competing interests and time is a limiting factor. Organisations often need to choose
between different programmes rather than being able to run them side by side. Some schools in this
program found ways to integrate Resilient Futures teaching within other programs, and this may be a
strategy that could be more widely promoted and supported.
In contexts where organisations where the focus is fading (as distinct from a conscious decision to
stop implementation) then structured, regular support from WRC could provide the impetus to
maintain the programme. Further investigation of how this would be most suited to partner
organisations would likely be necessary.
106
7.1.5 Working with isolated staff While most of the staff members who worked in the partner agencies worked together in teams, that
was not the case for all. Staff members who were not located with other team members did not have
the opportunity to reinforce their learning through regular interactions with other staff. One such staff
member suggested that refresher training would be necessary to improve their retention of skills,
knowledge and language associated with the programme.
As refresher training was also raised as a potential option for renewing staff enthusiasm for the
programme and mitigating a loss of focus on the programme, a briefer training programme could be
developed specifically for staff who had already been through the initial training. ‘Community of
practice’ meetings across agencies providing the programme could also provide support for otherwise
isolated workers.
107
8 Appendices
8.1 Appendix 1 – SAHMRI Theories of Change 2018
108
109
8.2 Appendix 2 – Survey
8.3 Appendix 3 – Survey Data
Table 2. PERMA (Positive Emotions) Scale Score
N. T1 T2 Diff. p.
Site 1 110 2.85 2.99 0.14 .032
Site 2 32 2.76 2.88 0.12
Site 3 12 2.53 2.80 0.27
Total 154 2.80 2.95 0.15 .011
Table 3. K10 (Anxiety and Depression) Scale Score
N. T1 T2 Diff. p.
Site 1 110 2.72 2.62 -0.10
Site 2 30 2.86 2.72 -0.14
Site 3 8 2.81 2.51 -0.30
Total 148 2.75 2.64 -0.11 .013
Table 4. ONS (Office of National Statistics ) Scale Score
N. T1 T2 Diff. p.
Site 1 116 5.55 5.79 0.24
Site 2 31 5.35 5.44 0.09
Site 3 12 5.22 5.33 0.11
Total 159 5.49 5.69 0.20
Table 6. PERMA Scale Score by Age Group
N. T1 T2 Diff. p.
16 or less 39 2.76 2.77 0.01
17 years 42 2.80 3.13 0.33 0.004
18 years 33 2.92 2.98 0.06
19 or more 34 2.70 2.92 0.22
Table 7. PERMA Scale Score
N. T1 T2 Diff. p.
Male 73 2.85 3.07 0.22 .023
Female 72 2.75 2.84 0.09
Table 8. K10 Scale Score
N. T1 T2 Diff. p.
Male 69 2.55 2.40 -0.15 .029
Female 72 2.90 2.83 -0.07
110
Table 9. PERMA Questions
N. T1 T2 Diff. p.
Joyful 161 2.84 3.02 0.18 .009
Satisfied with personal relationships
157 2.86 3.20 0.34 .001
Table 10. PERMA Questions by Site
N. T1 T2 Diff. p.
Satisfied with personal relationships
Site 1 113 2.90 3.19 0.29 .015
Site 2 32 2.84 3.28 0.44 .024
Site 3 12 2.50 3.00 0.50
Table 11. PERMA Questions by Gender
N. T1 T2 Diff. p.
Joyful Male 77 2.95 3.26 0.31 .002
Female 75 2.73 2.83 0.10
Satisfied with personal relationships
Male 74 2.91 3.18 0.27 .043
Female 74 2.84 3.20 0.36 .018
Table 12. PERMA Questions by Age
N. T1 T2 Diff. p.
Joyful 16 or less 43 2.86 3.00 0.14
17 years 42 2.83 3.21 0.38 .003
18 years 33 2.94 2.94 0.00
19 or more 37 2.70 2.97 0.27
Satisfied with personal relationships
16 or less 41 3.00 2.95 -0.05
17 years 42 2.71 3.38 0.67 .002
18 years 33 3.00 3.18 0.18
19 or more 35 2.69 3.26 0.57 .005
Table 13. K10 Questions
N. T1 T2 Diff. p.
Nervous 160 3.18 2.93 -0.25 .000
So nervous could not calm down
157 2.26 2.08 -0.18 .013
111
Table 14. K10 Questions by Site
N. T1 T2 Diff. p.
Nervous Site 1 117 3.21 2.96 -0.25 .002
Site 2 32 3.34 3.00 -0.34
Site 3 11 2.45 2.36 -0.09
So nervous could not calm down Site 1 115 2.26 2.05 -0.21 .010
Site 2 31 2.29 2.23 -0.06
Site 3 11 2.18 1.91 0.27
Table 15. K10 Questions by Gender
N. T1 T2 Diff. p.
Nervous Male 76 3.00 2.75 -0.25 .014
Female 75 3.35 3.07 -0.28 .006
So nervous could not calm down
Male 74 2.03 1.82 -0.21 .032
Female 75 2.45 2.33 -0.12
Table 16. K10 Questions by Age
N. T1 T2 Diff. p.
Nervous 16 or less 42 3.26 2.90 -0.36 .009
17 years 42 3.10 3.00 -0.10
18 years 33 3.09 2.70 -0.39 .013
19 or more 37 3.24 3.03 -0.21
So nervous could not calm down 16 or less 42 2.48 2.10 -0.38 .002
17 years 40 2.20 2.20 0.00
18 years 33 2.12 1.94 -0.18
19 or more 37 2.16 2.08 -0.08
Table 17. ONS Questions by Site
N. T1 T2 Diff. p.
Happy Site 1 116 5.58 6.07 0.49 .041
Site 2 31 6.10 5.55 -0.55
Site 3 12 5.42 5.42 0.00
Table 18. Differences by Gender, Scales
T1 T2
K10 Scale N Mean N Mean
Male 74 2.54 72 2.39
Female 74 2.91 73 2.84
Diff. 0.37 0.45
p. .005 .002
112
Table 19. Differences by Gender, Questions
T1 T2
Joyful N Mean N Mean
Male 77 2.95 77 3.26
Female 75 2.73 75 2.83
Diff. 0.22 0.43
p. .005
Table 20. Differences by Gender, K10 Questions
T1 T2
Male Female Diff. p. Male Female Diff. p.
About how often did you feel tired out for no good reason? 3.05 3.47 0.42 0.009 2.89 3.36 0.47 0.007
About how often did you feel nervous? 3.01 3.35 0.34 0.035 2.75 3.07 0.32
About how often did you feel so nervous that nothing could calm you down? 2.01 2.45 0.44 0.007 1.82 2.33 0.51 0.004
About how often did you feel hopeless? 2.34 2.70 0.36 2.11 2.62 0.51 0.005
About how often did you feel depressed? 2.38 2.87 0.49 0.017 2.30 2.83 0.53 0.007
About how often did you feel that everything was an effort? 2.83 3.08 0.25 2.71 3.09 0.38 0.036
About how often did you feel so sad that nothing could cheer you up? 2.01 2.53 0.52 0.006 1.92 2.48 0.56 0.001
About how often did you feel worthless? 2.17 2.79 0.62 0.003 2.20 2.67 0.47 0.022
