CHARTING A NEW PATH

PART 3: ON THE HORIZON: PATH’S SYSTEM LESSONS

CHARTING A NEW PATH

2 THE CHANGE FOUNDATION

CHARTING A NEW PATH

ABOUT THE CHANGE FOUNDATION

The Change Foundation (TCF) is an independent health policy think-tank that works to inform positive change in Ontario’s health care system. With a firm commitment to engaging the voices of patients, family caregivers and health and community care providers, TCF explores contemporary health care issues through different projects and partnerships to evolve our health care system in Ontario and beyond. TCF was created in 1995 through an endowment from the Ontario Hospital Association and is dedicated to enhancing patient and caregiver experiences and Ontario’s quality of health care.


THE CHANGE FOUNDATION 3

ACKNOWLEDGEMENTS

The Change Foundation gratefully acknowledges the many people who were part of the Partners Advancing Transitions in Healthcare (PATH) project.

First and foremost, our sincere thanks to the patients and family caregivers who shared their stories, worked tirelessly on the co-design teams and project committees, participated in the testing of PATH solutions – and, on so many levels, helped us learn.

We thank the PATH partner organizations – and the leaders and health care providers within those organizations – for their commitment to the project and to improving the experience of patients and family caregivers in the west Northumberland community.

We gratefully acknowledge the contribution of Wendy Kolodziejczak, PATH Project Manager; Helen Brenner, Vice President of Patient Services, Chief Nursing Executive and PATH Executive Lead at Northumberland Hills Hospital; and the PATH project team for guiding and managing this complex, innovative project. We are indebted to Walter Wodchis and the Health Systems Policy Research Network (HSPRN) for their measurement expertise and guidance in the development of the local-level reporting system. And we acknowledge the hours and hours put in by QoC Health Inc., our technology partner who worked with patients, family caregivers and providers to create the platform that allowed many aspects of the project to unfold.

Finally, we want to thank the evaluation team whom we contracted to carry out The Change Foundation’s strategic- and system-level evaluation of PATH: Jacquie Dale and Ken Hoffman (One World Inc.), Tanya Darisi (The O’Halloran Group), Marc Langlois (Marc Langlois Consulting) and Pamela Smit (Veradus Consulting).


CHARTING A NEW PATH

GLOSSARY

Co-design team: The teams of patients, caregivers, providers and partners that came together to develop solutions for each of the five project elements. Also referred to as the project element teams.

Experience Based Co-Design (EBCD): An approach that enables staff and patients (or other service users) to co-design health care services and/or care pathways, together in partnership. EBCD was central to the PATH project.1

PATH community: West Northumberland region of Ontario.

PATH lead partner: Northumberland Hills Hospital (NHH) was the lead partner in the PATH project – they received the financial transfer from The Change Foundation and housed the PATH project team.

PATH partner organizations: The health and community care organizations in west Northumberland that came together to create the PATH partnership

PATH partnership: The full group of 12 partner organizations and the 39 patients and family caregivers that made up the partnership.

PATH patients: The patients and their caregivers who were enrolled in the PATH project to use the My Health Experience mobile technology tool.

PATH portal: The My Health Experience mobile technology tool that patients used to keep all the information about a patient’s evolving life and health story and fill in real-time surveys about their health care experiences. It also facilitated secure two-way communications between patients/caregivers and health care providers.

PATH project: Partners Advancing Transitions in Healthcare (PATH) project

PATH project elements: The five project components that were included in the west Northumberland PATH proposal.

PATH project team: A small administrative group, housed at NHH that provided project management and operations support to the PATH project.

1 http://www.kingsfund.org.uk/projects/ebcd/experience-based-co-design-description



PATH providers: The health care professionals who were involved in the PATH project that worked at any of the partner organizations.

PATH solutions: The solutions developed by the project element teams (also referred to as co-design teams) using experience-based co-design (EBCD).

PATH volunteer peer coaches: Formally trained volunteers who coached seniors and their caregivers in using the My Health Experience portal, helped seniors connect and communicate with providers and access resources in the community, and provided support and encouragement.

Project element teams: The five teams made up of the PATH partnership (patients, caregivers, providers, administrators) that worked on the five project elements to develop patient-driven solutions.

RISE team: The Respect, Information, Support and Empowerment (RISE) team - a group of 15 patients and caregivers who were involved in the five project teams, and who also met independently throughout the project to provide advice to all aspects of the project.


CHARTING A NEW PATH

THE ORIGIN OF PATH

The Change Foundation’s 2010-2015 strategic plan, Hearing the Stories, Changing the Story, was focused on patient engagement and improving the patient experience in Ontario. The Partners Advancing Transitions in Healthcare (PATH) project was one of two showcase engagement projects that formed the basis of our work and our learnings over that five-year period. PATH was a made-in-Ontario patient engagement project, developed in partnership with the west Northumberland community. The premise of the PATH project was to create an environment conducive for patient engagement at every level of the local health care system – engaging with, working alongside and truly integrating patients and family caregivers into the health care fabric.

Charting a New PATH is a three-part report that describes what the Foundation has learned as participants in, and funders of, the PATH project. Given the complexity of the PATH project, that required a leap of faith by all those involved, we learned a lot. We have reflected on the major achievements, the unexpected opportunities that emerged, but also on the ambitions of the project that weren’t fully realized. And even though we didn’t achieve everything we set out to do, we don’t see that as failure – we see it as a learning and growing opportunity. It’s in that spirit, that we share our findings.

We believe our reflections and commentary can help inform the work of others in the health care sector who are in the midst of or about to embark on patient engagement activities in their organizations, communities or regions.

These reports are aimed at those who are interested in pursuing these concepts more deeply and more widely.

1) PART 1— Redefining Patient and Provider Partnerships: Proposal and Project Development 2) PART 2—Creating meaningful partnerships in care: Lessons From West Northumberland3) PART 3—On The Horizon: PATH’s System Lessons



PART 3:

ON THE HORIZON: PATH’S SYSTEM LESSONS

THIS REPORT IS THE THIRD IN A THREE-PART SERIES. IN ON THE HORIZON: PATH’S SYSTEM LESSONS, THE CHANGE FOUNDATION REFLECTS ON THE LESSONS LEARNED FROM OUR INVOLVEMENT IN THE PATH PROJECT THAT HAVE SYSTEM-WIDE RELEVANCE.

INFLUENCING SYSTEMS IS NOT AN EASY TASK. WE HAD BIG AMBITIONS, AND ALTHOUGH WE ACHIEVED A LOT, WE DID NOT GET EVERYTHING DONE THAT WE HAD HOPED FOR. WE DON’T CONSIDER THIS A FAILURE, BECAUSE WE LEARNED A LOT ALONG THE WAY, ESPECIALLY AS IT RELATES TO SYSTEM INFLUENCE AND THE ABILITY TO QUANTIFY THE IMPACT. OUR REFLECTIONS ARE OFFERED UP IN THE SPIRIT OF “FAIL FORWARD”2 – BECAUSE WE WANT TO SHIFT HOW WE PERCEIVE AND TALK ABOUT FAILURE, AND BEGIN TO SEE IT AS A RICH LEARNING OPPORTUNITY.

2 http://www.failforward.org


CHARTING A NEW PATH

OUR EXPECTATION OF A TRUE PARTNERSHIP WAS NEW – EXPLICITLY RECOGNIZING THE POWER IMBALANCE WHILE SIMULTANEOUSLY CREATING TRUST.

With our involvement in the PATH project, we did not want to operate like a traditional granting organization. We wanted to build a true partnership of a new kind – one in which we would collaborate and be actively involved in the development of the project. We wanted to learn and experience innovation, as well as both success and failure, alongside the PATH community. But we understood, ultimately, that it wouldn’t be possible to alleviate the perception of a power imbalance. There would always be one, and we needed to keep it top of mind in all of our interactions.

In creating this new kind of partnership, we had to learn to be comfortable with uncertainty. The emergent nature of a multi-partner, co-design initiative within an environment that is unpredictable, dynamic and complex is not for the faint-hearted. We knew that issues would come up that only the funder, in the end, could resolve. However, we found it very useful to sit through the discussions when issues arose, rather than be presented with a problem for resolution after the fact. This is not a traditional way that most funders, including government funders and traditional foundations, work. But the nature of this project required actual collaboration between the funder and the project team, both working to make it better.

It is a style of working with partners that we would use again and we would recommend for others.

This kind of partnership means spending time developing a relationship. The success of this model required a high level of trust from the PATH project leadership. In fact, a staff person from the Foundation co-chaired the PATH project Steering Committee. We were, therefore, present at the meetings when decisions were being made.



As the funder, we had to “walk the fine line between supporting the community partners’ work and interfering with it.”3 This required sensitivity on the part of TCF’s team—understanding that we had one vote, just like everyone else, and knowing when to stay silent and let the community work through local issues.

We created a project charter that outlined the role of the PATH project team and the role of The Change Foundation, which helped draw clear boundaries. This structure and approach was effective, and we would recommend it.

As new reforms are developed and introduced in Ontario – such as Health Links, Integrated Funding Models, the road map for home and community care, and potential primary care networks—the nature of the partnership between the funder (government) and the community partners needs to be re-examined. Innovation requires a willingness to take a risk and a leap of faith by both partners. This needs to be spelled out in advance and embraced.

DEVELOPING A MULTI-PRONGED EVALUATION STRATEGY FROM THE OUTSET WAS KEY IN OFFERING LOCAL AND SYSTEM LEVEL INSIGHTS.

Given the complexity of this project, and the possibility for impact both locally and at a system-level, we knew two evaluation approaches would be required. The PATH project team was responsible for the local level evaluation (as discussed in Report Two: Creating Meaningful Partnerships in Care: Lessons from West Northumberland. The Change Foundation took the lead on the strategic- and system-level evaluation.

An external team, One World Inc., was contracted to undertake this system-level evaluation. For this evaluation, we asked two simple but hard to measure, questions:

1. What was the value and impact of involving patients and family caregivers in initiatives to improve care?

3 Langlois M (2015) Creating the Space to Lead Change. An Independent Analysis of the Change Foundation’s Journey on the Frontline of Healthcare Transitions. Toronto: The Change Foundation. www.changefoundation.com.


CHARTING A NEW PATH

2. What was the value and impact of working through a broad partnership of health and social care organizations to address problems with care?

The five-person evaluation team triangulated data from multiple sources using the following mixed method approach to capture the project’s complexity:

• A retrospective analysis of project documents to provide insight into the context, relationships and history of the project (n=65 documents).

• Live observation of the work of the project committees and teams to uncover information and insights about the process, its value, and contribution to systems change (n=30 meetings).

• Partnership Assessment surveys and Social Network mapping conducted twice over the course of the project to assess the level and quality of partner engagement (Dec. 2013: n=70; Jan. 2015: n=32).

• Key informant interviews completed in person or over the phone to gain further insight into the value and impact of participation in PATH, as well as to identify gaps and challenges (n=20).

• Focus groups in January 2015 to assess the process and impact of the pilot (n=55 participants in 7 focus groups).

The evaluation design and analysis was guided by The Change Foundation’s theory of change and logic model (see Appendix 1 in Part 1). In this report, See Appendix 1 for a discussion of the data analysis methods and Appendix 2 for the evaluation questions and the data sources used for each.

This evaluation was not intended as a summative assessment of PATH. Rather, the approach was adaptive with a learning focus, in the vein of participatory action research4. In addition, given the timelines of the PATH project and the piloting of the designed solutions, data collection did not start in earnest until the fall of 2014.

4 Participatory action research (PAR) is an approach to research in communities that emphasizes participation and action. It seeks to understand the world by trying to change it, collaboratively and following reflection. PAR emphasizes collective inquiry and experimentation grounded in experience and social history. Within a PAR process, “communities of inquiry and action evolve and address questions and issues that are significant for those who participate as co-researchers”. Reason, P. and Bradbury, H. (2008) (eds) The Sage Handbook of Action Research: Participative Inquiry and Practice. Sage, CA.



THE VALUE AND IMPACT OF INVOLVING PATIENTS AND FAMILY CAREGIVERS AS EQUAL PARTNERS WAS UNQUESTIONABLE

As a foundational principle, the PATH project took patient and family caregiver involvement beyond tokenism. West Northumberland has a considerable number of seniors and family caregivers within a fairly small population base, and this helped to ensure broad participation. The corresponding infographic breaks down how individuals from west Northumberland participated in PATH.

patients and caregivers participated on project element teams to develop solutions, with a number of these patients and caregivers participating on more than one team

patients and caregivers provided input through the RISE team

patients and caregivers participated in the pilot of the mobile technology (as of summer 2015)

volunteers provided support as peer coaches

39

25

15

120

31

patients shared their health care stories


CHARTING A NEW PATH

There was always more than one patient or family caregiver in any PATH project team to reflect best practice in patient engagement. Together, they provided multiple perspectives and insights about experiences with health care encounters. As a result, the PATH project benefitted from a diversity of patient and caregiver experiences with health care in west Northumberland.

The evaluation team conducted an independent survey of the patients and family caregivers to assess whether they felt their contributions were valued, and found the following:

• 93% of patients and family caregivers reported that their point of view was respected. They talked about how they felt respected and valued by providers in the design process.

• 73% of patients and family caregivers reported that they had developed new skills.• 82% of patients and caregivers reported that they gained new insights and knowledge

from participating in co-design.

The health care providers involved with the PATH project reported that it was much more impactful to hear things directly from seniors and family caregivers than from other providers. When the same themes came up in multiple senior and family caregiver stories, the discussion would move from isolated experiences to an exploration of systemic factors and issues. Figure 1 shows that both PATH partners and providers rated the impact of patient and caregiver stories particularly high.

One of the most telling PATH statistics was the drop in the number of times patients had to repeat their health information once the hand-held My Health Experience technology was up and running. Previously, this was a constant irritation for the seniors and family caregivers, and it was an issue that came up every time they were asked for their input and feedback. They explicitly asked that the PATH project find a way for patients to share information once directly with their providers electronically. This My Health Experience technology was built into a secure portal with strong results. By June 2015, 100% of patients reported that they did not have to repeat their health condition and information unnecessarily during a visit, as shown in Figure 2.

A technology tool of this nature would not have been included if seniors and family caregivers had not taken part in the design. This is a clear example of how patient and caregiver feedback influenced the PATH project’s priorities.



0 1 2 3 4 5 6 7

Senior, patient and caregiver stories helped me get more engaged in the issues

Stories were used to create common understanding of the challenges

Stories were used to find ways to improve patient and caregiver experiences

6.03

6.16

6.06

MEAN RATING

May

/14

Jun

/14

Jul/

14

Au

g/1

4

Sep

/14

Oct

/14

No

v/14

Dec

/14

Jan

/15

Feb

/15

Mar

/15

Ap

r/15

May

/15

Jun

e/15

PE

RC

EN

TAG

E

(CU

MU

LAT

IVE

AN

D P

OS

ITIV

E R

ES

PO

NS

E) 100%

80%

60%

40%

20%

0

Did you feel that you hadto repeat your health condition and informationunnecessarily?

Not at all

Sometimes

Frequently

Always

Percent Positive Response

PATIENT (EXPERIENCES) INDICATORS WITH ALL PROVIDERS

Figure 1

Figure 2: Over a 14-month period, between 75% – 100 % of patients (N=328) said they didn’t have to repeat their health information unnecessarily.

Figure 1 illustrates partner and provider ratings (6.03 – 6.16) of the impact the stories (Scale: 1 [not really] – 7: [very much]).


CHARTING A NEW PATH

0

1

2

3

4

5

6

7

5.72 4.7 5.09

I have been able to contribute to decision making

Partners Care ProvidersCohort

Seniors/Patients/Caregivers

ME

AN

RA

TIN

G

The PATH project team did an excellent job of setting up processes for patient engagement in the project – from story-telling, to training, to embedding patients in the project infrastructure and at decision-making tables. Patients and caregivers were truly equals in the process.

However, our data indicates that as the PATH project shifted from the intensive co-design phase into the pilot, testing and solution implementation phase, the role of patients and family caregivers changed. At this point, some patients and family caregivers felt a bit disengaged. This is to be expected, however it is an important reminder to be very clear and purposeful about roles, and the changing nature of those roles, as decisions move into an implementation phase within provider organizations. Finding ways for patients and family caregivers to maintain their strong connection to the project and the partner organizations in a different way is a good strategy for facilitating ongoing support for the project.

Ironically, the heavy emphasis and support for patient engagement may have actually discouraged provider engagement in some instances. Patients and family caregivers had a more positive perspective on the balance of input they had, compared to providers, as illustrated in Figure 3.

Figure 3: Providers rated their contribution to decision-making lower than patients and family caregivers – 4.7 on average out of 7, compared to 5.09 by patients and family caregivers.

FOUNDATION REFLECTIONS ON VALUE OF PATIENT AND FAMILY CAREGIVER ENGAGEMENT



0

1

2

3

4

5

6

7

6.11 4.95

6.36

There is a shared vision for the project

Partners Care ProvidersCohort

Seniors/Patients/Caregivers

ME

AN

RA

TIN

G

This difference also played out in questions about whether there was a shared vision for the project, as shown in Figure 4.

Figure 4: Providers were less likely to share the vision for the project than patients and family caregivers – 4.95 out of 7 compared to 6.36 by patients and family caregivers.

Some partners and providers were concerned about the early focus on technology and the development of the portal. The portal was intended to respond to the frustration of patients and family caregivers with communication and information sharing. In the evaluation interviews, some partners and providers said that they felt that it negated consideration of other change options such as shared care plans, protocols for data sharing and patient transfers, or scaling a patient-family engagement model.

To be clear, the My Health Experience technology tool was part of the original west Northumberland PATH proposal, so the PATH project team felt that it was something they had to implement. Given the project’s short timeframe, this likely did mean that other suggestions for change were not given full consideration, and were not as developed as My Health Experience.

Upon reflection, The Change Foundation would recommend adding an additional step at the outset of a project of this nature. In the competitive process to select the PATH community, we were confident that we had chosen a community ready and committed to pursuing


CHARTING A NEW PATH

the key features of PATH. However, it might have been beneficial to meet with the west Northumberland community partnership for a deliberative dialogue on the components of the proposal before the project began. This would have provided the partnership with an opportunity to consider other options to achieve the PATH objectives within the funding timelines.

Another reflection is that the impact of the co-design process didn’t spread equally across all 12 partner organizations. There was a high concentration of capacity for patient engagement developed within the lead PATH partner (Northumberland Hills Hospital), which is understandable. However, it is important to be purposeful about ensuring that a similar capacity can be built and sustained within all partner organizations. If only parts of the local health system are patient-centered, patients and family caregivers may not have an optimal overall experience. In retrospect, two years was probably not enough time to ensure that the culture change could take hold as deeply in the other organizations as it did at the hospital. Changing culture across multiple organization, with very different cultures and priorities, is a very large undertaking, that would take a number of years. On the other hand, a different project organization or approach, such as a co-leadership model, could have helped the partner organizations achieve more change within their own walls. We can’t be sure, but it is something others undertaking this work should look at.

Members of the PATH project team form Northumberland Hills Hospital.



ENGAGING BROAD PARTNERSHIPS OF PROVIDERS AND WORKING ACROSS ORGANIZATIONS

The PATH partnership included 12 health and community care organizations, with 190 health care providers directly involved. The engagement work at the beginning of the project was very positive. The PATH patients, providers and partners trained together, heard the patient stories, sat through co-design workshops as part of the work of project element teams. It was an invigorating and enjoyable process for all involved.

Survey results showed the two-year PATH project to be a positive experience overall:

70%of partners and providers reported increased responsiveness to the needs of patients and family caregivers

80%of PATH partners and 70% of providers involved in PATH reported that they were able to make a meaningful contribution to the project

86%of PATH partners rated the benefits of participation in the process as outweighing the drawbacks of participation

83% of PATH partners reported they developed valuable relationships through PATH

60%of patients and family caregivers, partners and providers reported that PATH improved connections between multiple services, programs and systems


CHARTING A NEW PATH

PATH partners and providers also reported that they gained a better understanding of the role and services of the other PATH partner organizations – an interesting indication of what providers know in terms of available services for patients outside of their own offerings.

Spin-offs were also seen as a result of relationships established through PATH. In many ways, these were unanticipated opportunities that come from working differently and encouraging innovation, as the PATH project did. For example, the hospital leveraged the relationships made through PATH to support other initiatives, including engaging the YMCA and the Family Health Team in the Safe Mobility Committee, the Falls Prevention Initiative, and the Medically Complex Patient project.

The PATH evaluation team actually mapped the social networks of all the PATH partners (patients, family caregivers, partner organizations and providers) showing connections and relationships prior to the establishment of PATH (i.e., at the proposal stage) and six months later. The spider web diagrams in Figures 5 and 6 show how much more interconnected people became over the course of the PATH project.

Figure 5: Connections reported prior to PATH (Proposal Stage).



Figure 6: Connections reported six months after PATH began (December 2013).

The network mapping shows a greater density of connection among local partners and care providers after PATH introduced them to a new way of working together. They knew of each other before, but had not collaborated. The map also shows that, prior to PATH, patients and family caregivers were dispersed around the periphery of the network. As a result of PATH’s co-design process, they moved from the periphery to the core.

A detailed analysis of the network maps shows that:• Patients and family caregivers reported the greatest increase in the number of

connections within the PATH project – reporting almost 10 times as many on average compared to partner organizations and providers.

• Partners reported an average increase of 75% in the number of new connections. • Providers also reported an increase in their connections, although to a much lower

extent – an average increase of 14%.


CHARTING A NEW PATH

As a result, we can identify quite clearly that this type of work connects people in a community who were not necessarily working together before. It is logical to assume that over time, these relationships will evolve into discussions about more integrated care. However, at this point, we do not have data to support this.

It’s important to note that the PATH project did develop a model for patient- and family-centered care through an experience based co-design process. However, as the realities of time and funding were brought to bear, the roll out of this model to the partner organizations was put on hold. This is definitely work that could continue in the future.

FINAL REFLECTIONS ON BROAD PARTNERSHIPS AND WORKING ACROSS ORGANIZATIONS

Establishing greater knowledge and connections across partner organizations is clearly beneficial, but trying to actually redesign program and service protocols across organizations is difficult. Budgets are based on location of service. Accountability agreements are organizationally based. In future, what may be needed is a pooled fund. A model for this does not exist yet, but one needs to be developed. Integrated funding models and the proposed expansion of LHIN oversight, may be paving the way for sharing funds and accountabilities across organizations.

There was also the unique challenge of physician engagement in the co-design process. This is not a new issue for quality improvement projects but certainly was part of the PATH project’s complex environment. Physicians were invited to participate on the co-design teams and help design solutions, but workload, payment structures and other priorities made it hard for them to commit the time. Although this challenge was not unique to PATH, there was no clear way, locally, to get past these barriers and increase the number of physician participants.

Lastly, there is the issue of sustainability. Critics often refer to Ontario as the land of pilot projects. This project could be included on that list, as it was time limited and, at the time of publication, has yet to have secured funding to support its spread. TCF struggled considerably with whether we should extend the funding so that the work could continue. In the end, we extended funding for a short period, but had to stay true to our purpose – we are an organization that supports and facilitates incubation, and we had to face the fact that we are small and have very limited resources.



LESSONS LEARNEDIn a nut shell, designing and implementing community-based, cross-sectoral, multi-partner provision of health services, with patients and families as part of the team, is not a straight-forward process. However, with the positive impacts on clinical outcomes and safety in organizations that have a patient-centred culture5, there is growing interest in this approach. Furthermore, it is aligned with the direction in which the Ontario health system is moving.


5 http://isqua.org/docs/default-source/education-/glenn-robert-isqua-webinar-slides-jan-2014.pdf?sfvrsn=0

10TOP


CHARTING A NEW PATH

Both the role of the funder and the relationship between funder and community need to be explicit, transparent and constructive. A project charter outlining accountabilities, roles and responsibilities is a critical starting point, however ongoing nurturing of the relationship is needed over the course of the project.

The relationship must be built on openness, trust and a willingness to “fail forward” together. Agreeing up front that there is always something to be learned even when things don’t go according to plan, gives permission for open dialogue and course correction.

There is an opportunity to revisit the actual winning proposal before agreements are signed and the work begins. This is the time for an open dialogue between the funder and the project team, now that it is clear you are partners. What exactly do you want to accomplish? Does the project team have other ideas? Were there things they didn’t put in their proposal because they might be considered too off-the-wall? Are there things that you, as the funder, hoped to see, but didn’t, which could now be put on the table? Has anything changed since the proposal was submitted that affects the approach?

Initial orientation, training and capacity building needs to be delivered with patients, family caregivers and providers together. Making time for these joint sessions will be harder for some providers, patients and caregivers than others, but it is imperative. This needs to be taken into account when designing and scheduling the sessions. Questions of timing and compensation will come into play. For issues of compensation, refer to our Should Money Come Into It? report.

Do not underestimate the power of storytelling. Use it to start the process, and at the beginning of any meeting. What may initially seem to be isolated problems, add up to a picture of system issues. There is a tipping point and everyone in the room knows when it has been reached.

There is a clear pattern of great enthusiasm and momentum as work gets off the ground – this is particularly true for the patients and families, as this is often the first time they have been involved in this kind of work. As the project moves from design to implementation, most of the day-to-day work swings more to the provider organizations and staff. During this period there is a danger that the patients and

10TOP

IF YOU ARE ABOUT TO EMBARK ON A CO-DESIGN OR PATIENT ENGAGEMENT PROCESS, WE HAVE CREATED THIS TOP TEN LIST OF THE KEY ISSUES AND POINTS TO BEAR IN MIND:

1

2

3

4

5

6

LESSONS LEARNED



families become disengaged, as the team process becomes more about reporting on results, than it is about collaboration and creating new ideas. Right from the outset, it is important to anticipate this shift and create an expectation of it from the start, through clear and ongoing communication. It is normal, and it does not diminish the importance of the patient and family contributions. The reality is that the intensity of engagement changes over the life of the work. We began to think of it as a pendulum. Being upfront about the ebbs and flows of the engagement process helps avoid disappointment.

Pay attention to the balance between the time and energy spent engaging with patients and families and the time and energy spent engaging with providers. This is particularly important as it relates to providers and staff who work in partner organizations other than the lead one (in PATH’s case, the hospital). When a project deliberately sets out to bring patients and families into the centre of decision-making, it will naturally spend a lot of time establishing that process. However, further into the project, the implementation of change will rely on staff and provider commitment to the vision. If this is not secured at the start, success will be harder to achieve.

A concerted focus on the spread of engagement capacity across the partner organizations should be explicit from the beginning of the project. This should not be left until later to think about, once there is proof of concept in the lead partner organization.

At the mid-point of the work the team should have some sense of where things are heading. This is a good time for a brainstorming session to identify any system barriers that could be a problem once the external funding comes to an end. Barriers could include payment mechanisms, organizational-specific accountability agreements, LHIN rules, MOHLTC policy or any other system legacies that the team thinks may get in the way of continuation and spread. The Ministry has recently indicated a willingness to relax some rules or tolerate a low-rules environment for some collaborative efforts. This should be capitalized on if appropriate.

Identify, early on, who the likely champions are for sustainability and spread and bring them into the tent as early as possible.

The Change Foundation is enormously proud of the work of the PATH project and the west Northumberland community. Watching the enthusiasm, capacity development and commitment of all involved, was a true honour. As the work continues, Ontario’s health system will benefit from all that has been learned, and Ontario patients, family caregivers and health care providers will benefit from better health care experiences.

7

8

9

10


CHARTING A NEW PATH: A THREE-PART REPORT

APPENDICES:

APPENDIX 1: METHODS FOR DATA ANALYSISAPPENDIX 2: EVALUATION OUTCOMES, INDICATORS AND DATA SOURCE



Qualitative data were subjected to a thematic analysis. Conversation and observation notes were initially reviewed by one of the evaluation team members, who identified tentative themes. These tentative themes were then reviewed by other team members, and differences in interpretation were resolved through discussion. Analytic decisions were guided by the evaluation objectives and questions.

The quantitative analysis was completed using SPSS 22. To complete the survey analysis, three groups were identified: PATH partners, care providers, and patients/caregivers. Only completed surveys were included in the analysis. An analysis of variance was conducted to assess any difference in experience between participant groups and between committees for each of the survey topic areas. Significance was determined at p<.05. The Bonferonni correction for multiple comparisons was used when assessing main effects and interactions on topic items.

An effort was made to minimize the limitations of any one method by using mixed methods and triangulating data from multiple sources. The limitation of self-report was minimized by assuring participants of confidentiality and emphasizing the value of open and honest feedback.

It was a challenge to engage PATH patients and providers in the Partnership Assessment survey in 2015. The project had shifted substantially between the first and second assessment, and many of the patients/caregivers and care providers were no longer involved in the activities of PATH and did not participate in the survey. As a result, the 2015 survey was only completed by about half the number of participants as the 2013 survey. The attrition rate limited the ability to interpret the third network map. To avoid misinterpretation, the network maps from the 2013 survey – network before and after the initiation of PATH – were included.

APPENDIX 1: METHODS FOR DATA ANALYSIS


CHARTING A NEW PATH

OUTCOME INDICATORS METHODS & MEASURESPATIENT/CAREGIVER

(O1) Positive, respectful, productive relationships

Patient/caregiver (p/c) and providers’ perceptions of positive demonstration of respect and productivity:

Retrospective AnalysisKey informant interviews Tracking Partnership Assessment

(O1) Increased meaningful engagement in co-creating knowledge and shaping change

Patient/caregiver and providers’ perceptions of the value of the process and the quality of the engagementPatient/caregiver operationalization of “meaningful”Positive shift in culture towards engagement of patient/caregiverLevel of participation and contribution to the processProject outputs that support/reflect positive change

Retrospective AnalysisKey informant interviews Participant ObservationChange StoriesTrackingPartnership Assessment

(O1) Increased capacity of both providers and patients/caregivers for partnering

Identification of new skills, stronger sense of efficacy, and other personal impacts

Key informant interviews

PARTNERSHIPS

(O2) Greater use of Experience-Based Knowledge in shaping decisions & solutions

Perceived change and identification of a positive contributionProject outputs that support/reflect positive change

RetrospectiveKey Informant InterviewsPartnership Assessment

(O2) Greater levels of shared accountability for transitions

Perceived change and identification of new accountability processes

RetrospectivePartnership AssessmentKey Informant Interviews

(O2) Stronger, more productive relationships and collaborations

Perceived change and identification of new collaborations

RetrospectivePartnership AssessmentStakeholder Network MapKey Informant Interviews

(O2) Alignment of resources to support ongoing EBD

Resource allocation to patient involvement activities

Partnership AssessmentKey Informant Interviews

APPENDIX 2 – EVALUATION OUTCOMES, INDICATORS AND DATA SOURCE



OUTCOME INDICATORS METHODS & MEASURES

PROJECT ELEMENT OUTCOMES

(O5) Generativity and “spin-offs”

New & different kinds of solutions (stakeholder-defined)Positive change in staff experience

Tracking Key Informant Interviews

(O5) Enhanced patient involvement in improvement processes

Patient involvement activitiesPositive change in staff experiencePositive change in patient/caregiver experience

Key informant interviewsFocus GroupsChange StoriesSecondary Analysis

(O5) Change in policies, processes, and practices that enhance care transitions

Positive examples of change in policies, process and practices

Key Informant InterviewsFocus Groups

LONG-TERM OUTCOMES

(O6) Organizational environments that support enhanced patient and caregiver experiences

Shifts in organizational practices and culture that reflect Patient-Centred Care (knowledge; attitudes; behaviours; products)

Change StoriesKey Informant Interviews

CONTACT US

The Change FoundationP.O. Box 42200 Front Street West, Suite 2501Toronto, ON M5V 3M1

www.changefoundation.ca

Copyright: The Change Foundation 2016

http://www.changefoundation.com

Documents

CHARTING A NEW PATH