Quality Systems and Outcomes Measurement Project

A Service Effectiveness Framework: moving beyond compliance to quality outcomes for consumers

Foreword

As Minister for Disabilities in South Australia I am committed to delivering high quality services to people with disability, their families and carers.

In December 2013, the Department for Communities and Social Inclusion commissioned the ‘Quality Systems and Outcomes Measurement Project’ to determine how individuals, families and funding bodies know that disability support services are effective in achieving positive outcomes for individuals. This is a question currently challenging all jurisdictions in Australia looking to move beyond minimum standards in quality improvement.

The project has produced the Service Effectiveness framework a guide to measure and monitor the performance of disability service providers in providing support that makes a difference in people’s lives. This unifying framework enables:

Individuals and families: to have additional information to compare between, or reflect on, the performance of providers in areas of importance to them.

Providers: to review and monitor their service delivery, which provides a basis for supervising direct support practice and reporting about service delivery performance.

Funders and external reviewers: to review and critique the service delivery outcomes of providers consistent with regulations, standards or other accountability requirements.

A key feature of the framework is the provision of support which increases opportunities for choice and control and individualised lifestyles, increases life quality and establishes safeguards for people with disability living in the community.

Page | 2

A toolkit has been developed including four guides designed to assist individuals and families, providers and funders in the development and understanding of effective services. Providers will be able to identify and measure service outcomes within their organisations, while individuals and families will have a guide for their own enquiries and observations.

It is my hope that this report and the associated toolkit will generate interest and discussion around the importance of quality assurance in services to people with disability; and contribute to the public consultation regarding the proposed National Quality and Safeguards Framework under the National Disability Insurance Scheme.

Hon Tony Piccolo MP

Minister for DisabilitiesJune 2015

Page | 3

AcknowledgementsDisability SA wishes to acknowledge the consultants for their work on the project, including:

Dr Chris Fyffe, Grimwood Pty Ltd, mcfyffe@grimwood.net.au

Professor Christine Bigby, Living with Disability Research Centre, La Trobe University, C.Bigby@latrobe.edu.au

Professor Jacinta Douglas, Living with Disability Research Centre, La Trobe University, J.Douglas@latrobe.edu.au

Chelsea Miller, chelseamiller@gmail.com

Disability SA also wishes to acknowledge the valuable contributions made by all individuals including people with disability, carers and non-government organisations that gave their time to actively participate in the project.

ISBN No 978-1-921649-62-2

Stock images may appear in this publication. These images are for illustrative purposes only, and people depicted in these images may not be associated with the publication’s subject matter.

The information in this publication can be provided in an alternative format or another language on request; phone Martin Lockwood on +61 8 8415 4202 or email him at martin.lockwood@dsci.sa.gov.au

Page | 4

Executive summaryDisability SA established the Quality Systems and Outcomes Measurement Project in December 2013 in response to the question: How can individuals, families and funders know that disability support services are effective in achieving outcomes? This question is challenging jurisdictions right across Australia, as part of striving to be sure that support services for people with disability are responding to improve the lives of people with disability.

The project has produced a Service Effectiveness Framework (the framework). The framework uses tailored organisational effectiveness indicators to measure and monitor the performance of disability service providers in providing support that makes a difference in people’s lives.

This unifying framework enables:

Individuals and families: to have additional information to compare between, or reflect on, the performance of providers in areas of importance to them, as well as information gained from their informal networks and personal preferences.

Providers: to review and monitor their service delivery, which provides a basis for supervising direct support practice and reporting about service delivery performance to the people they support and to their Board and Chief Executive Officer.

Funders and external reviewers: to review and critique the service delivery outcomes of providers consistent with regulations, standards or other accountability requirements.

The framework changes expectations of providers: this is not a template of what providers must report but an emphasis on how to respond. This is a formative and developmental process with significant benefits arising for provider accountability to individuals and funder/government.

The framework is derived from four core individual outcome domains. The service effectiveness indicators (the indicators) reflect how well each provider responds to the people being supported, given their service model/s, business model and program logic. These indicators can be used by organisations to identify and measure service outcomes within their organisations and by individuals and families to guide their own enquiries and observations of organisations. An overview of the framework is illustrated below.

Page | 5

Implementation will not follow a uniform path – but depends on each provider developing measures of their effectiveness in making a difference in the lives of people that they support.

A Toolkit including four guides has been developed to assist service providers, individuals and funders in the further development and understanding of service effectiveness measures and indicators.

Guide 1 – Establishing indicators of service effectiveness: outlines for providers how to develop indicators and measures particular for each organisation.

Guide 2 – Monitoring service delivery: is organised according to the outcome domains and provides information for anyone when visiting service providers and observing service delivery about ‘what to see and what to ask about’.

Guide 3 – Observing indicators of service effectiveness: proposes a process for observing service delivery and asking questions when individuals, perhaps with their family members, are visiting service providers. This guide could also be used by internal review teams or external assessors.

Guide 4 – Signposts of effective services: offers tips for identifying effective and ineffective services.

Page | 6

Part 1: Quality Systems and Outcome Measurement Project

Setting the sceneWith the continuing implementation of the National Disability Insurance Scheme (NDIS) it is critical to recognise that there will be significant changes to the disability sector for both people with disability and service providers.

In the current environment, service providers predominantly rely on funding via ‘block grants’ from the State Government via Disability SA. These funding arrangements mean that the State Government is the main purchaser of disability support; has a contractual relationship with each service provider; and regulates safety and quality.

Under the NDIS, funding for disability supports is allocated to each eligible individual, not a service provider. This will mean that the Government will no longer have a contractual relationship with service providers. The primary funding relationships will be between the person with disability and the provider of supports. It is expected that these new arrangements will raise challenges for service providers to compete on price, quality and customer experience; and to operate in an environment where payments are retrospective and contracts for supports are individualised.

With this in mind, the findings of this project are considered in the context of this major reform and acknowledge the changing roles of the Government, service providers and people with disability.

Introduction to ProjectThe Quality Systems and Outcome Measurement project undertaken by Disability SA recognises the benefits of an integrated quality system. The project was designed to complement and strengthen existing government quality monitoring processes whilst responding to the need for improvement within the current quality system with a particular focus on:

Improving the quality of services provided to people with disability and ensuring that services are focused on positive outcomes for the individuals concerned.

Supporting service providers to prepare for future requirements of the National Standards for Disability Services and the forthcoming National Quality and Safeguards Framework under the NDIS.

Assisting in positioning South Australia for the full implementation of the NDIS.

The project approached these issues by focusing on the effectiveness of services to provide support that makes a difference – achieve outcomes - in the lives of people

Page | 7

with disability. That is, how to promote and monitor service delivery effectiveness beyond current requirements for accreditation that tend to focus on policy and procedures of organisations. As a result, the framework has an increased focus on practice which is the actual provision of support and what happens during, and as a result of, service delivery. This report has four parts:

1. Findings from the Quality Systems and Outcomes Measurement project are based on:

a) review of government documents and literature about quality systems;b) accreditation outcome measurement;c) mapping of the existing Disability SA quality functions; andd) critique of the accreditation systems used in South Australia and other

examples.

2. A description of the Service Effectiveness Framework.

3. Summary of the Service Effectiveness Framework.

4. The Appendix includes a definition and discussion about key terms and concepts used in the report.

A separate Toolkit includes the four guides for use by providers, individuals and families and external reviewers.

Guide 1 – Establishing indicators of service effectiveness.

Guide 2 – Monitoring service delivery.

Guide 3 – Observing indicators of service effectiveness.

Guide 4 – Signposts of effective services.

The project has identified service effectiveness measures which reflect some but not all personal outcomes. The service effectiveness indicators align with the goals for the NDIS of increasing choice and control, and social and economic participation for individuals and families. Attention to safety and collaboration with individuals and families is a component of the framework.

The NDIS market place provides service providers with the opportunity to describe their ‘products’: what they are delivering and for how much, what difference their support makes for people with disability, and what types of support they are skilled at providing. Attention to service effectiveness aims to facilitate the NDIS transition by strengthening the information available to individuals and families as the basis for selecting services, as well as providing the opportunity to provide feedback and satisfaction ratings.

Page | 8

A note about terminologyWe have used ‘people’ or ‘individuals’ throughout the document, as well as people with disability. Organisations providing disability support are referred to as ‘organisations’ or ‘service providers’ of ‘programs’ or ‘services’.

Project method

Where the project startedIn the current environment in South Australia it was important to first ask the question, ‘What is required of a government funded quality system?’ The simple answer is that the funder needs to know about the performance and effectiveness of the disability service delivery that it funds. There are three broad requirements that drive this need to know, which are:

1. Ensuring efficient and accountable expenditure of public funds.

2. Minimising, and managing, risks to vulnerable children and adults with disability.

3. Enabling effectiveness of service delivery consistent with public policy directions. That is, by encouraging support that makes a positive difference in the lives of people with disability and their families.

These three current requirements involve a variety of monitoring, compliance and developmental activities. These are usually described as a quality system and include components of quality assurance and quality improvement. It is anticipated that these three requirements will also be relevant for the National Disability Insurance Agency (NDIA).

How the project was undertakenThe project was collaborative. Meetings and workshops held between July–October 2014 created opportunities for input and feedback for individuals and families currently receiving funded services, non-government providers and Disability SA staff. The project benefited from this collaboration with representatives from stakeholder groups, rather than being limited to a desk top review. Everyone involved has contributed to the project’s evolution and ultimate conclusions.

In addition, the purpose and nature of existing Disability SA quality systems were reviewed and mapped; literature about quality, accreditation and practice implementation reviewed; and feedback received about the role of South Australian quality management accreditation systems. Workshops with providers and representatives of quality management accreditation systems plus a review of departmental documentation and literature contributed to a critique of the current Disability SA quality management accreditation processes. The framework provides

Page | 9

organisations with a means of providing evidence that the support they provide through various programs, makes a difference in people’s lives arising in four critical individual outcome domains:

1. Choice and control

2. Personal capability

3. Relationships and community inclusion

4. Purposeful life and meaningful activities.

Findings

Review of literature about quality systems, accreditation and outcomesDespite the popularity of quality systems and performance management there is surprisingly little research about their implementation in human services, or the disability sector in particular. The review of literature gave an overview of the issues about quality systems and accreditation processes. Various quality systems use service standards as a means of determining if services are meeting people’s needs. However, there is a lack of consensus about what outcomes to measure and how to measure them. Indeed, there is ongoing debate about the implementation and effectiveness of regulatory systems that assess against standards and quality systems generally.

There is concern that quality systems and compliance approaches have not been sensitive to the experience of the individuals receiving support or to the approaches taken by staff delivering that support. This has led to recognising the importance of observation to determine the quality of support practices. Observational methods are critical for measuring the quality of staff practice and the lived experience of people, particularly those with severe and profound disability who cannot easily represent themselves. Many staff often work in dispersed unsupervised and isolated home or community locations. They act alone, apart from the presence of the person being supported. As a first step to improving the quality of support practices, someone other than the person with disability, (who may not in any case be able to voice his or her concerns), needs to be aware of the way support is provided and the experience of the person with disability in receiving it. Observation cannot be subject to the type of falsification that has occurred in paperwork processes. There are examples from overseas of independent monitoring teams which include people with experience and expertise at the service level – people with disability, family members and professionals. Observation should be regularly undertaken by supervisors and managers, individuals and their families and from time to time by external reviewers. That is, people who know what staff practices to look for and what to ask people receiving support and their support staff.

Page | 10

There is emerging research about the organisational features that lead to positive outcomes for people being supported, which complements the research about types of staff practice that lead to better outcomes. Important organisational features identified are leadership and shared values; positive culture and organisation-wide coherence about the outcomes to be achieved; organisation-wide orientation and induction training relevant to the mission and values of the organisation; evidence based staff best practice and personalised planning and delivery; direct involvement of individuals and families; and monitoring performance.

Mapping of the existing Disability SA quality functionsIn order to locate the Quality Systems and Outcome Measurement project within the context of Disability SA’s quality processes, the functions currently being performed by Disability SA were mapped against the service regulatory and monitoring roles of government. Overall, the mapping process revealed that to date the South Australian quality system has emphasised quality assurance activities related to primary functions such as financial accountability, contractual compliance and individual risk management. There has been less capacity for determining service effectiveness and facilitating quality improvement. The Quality Systems and Outcome Measurement project provides an innovative mechanism towards developing sector capacity in quality improvement systems.

Critique of the accreditation systems used in SA disability sectorCurrently the three most commonly used quality accreditation systems by Disability SA funded non-government organisations are the Australian Service Excellence Standards (ASES), ISO 9001-2008 and the Quality Improvement Council – Health and Community Services Standards (QIC). These existing third party accreditation programs are used by providers to help demonstrate they are meeting the National Standards for Disability Services.

Service standards describe key areas of importance against which services must demonstrate their compliance. Judging whether standards are met involves considering evidence about an organisation’s processes and whether specific indicators of outputs, processes, outcomes and staff practice are present. In Australia, and beyond, a variety of processes ranging from self-assessment to accreditation and various independent audit processes are used to determine whether standards have been achieved.

Increasingly evidence suggests that collecting information to show that standards are being met is not an effective way of demonstrating that individual outcomes are being achieved, or that organisation are adopting good staff practice. This is because standards based methods are predominantly paper and process based and

Page | 11

while they are able to be replicated, the evidence collected is not necessarily reflective of actual staff practices and quality of service delivery.

There are many tools, frameworks and approaches to measuring outcomes and these have diverse purposes, strengths and weaknesses depending on the priorities of the intended user. Criteria for selecting outcome measures have included the traditional measurement parameters of reliability and validity along with feasibility, cost and burden of data collection, transparency, relevance of results and how understandable they are to a lay audience. As part of validating outcome measures, several countries are emphasising the role of independent review teams using observational rather than paper based methods of evidence collection (for example, in the UK and New Zealand).

DiscussionAccreditation, practice quality and the importance of observation. The existing quality system used in South Australia (and nationally) is heavily focused upon compliance. It has the important function of minimising risk and ensuring organisational compliance with legislation and the National Standards for Disability Services (Disability Standards).

Accreditation emphasises desk top audits of paperwork and processes rather than service delivery, observation and outcomes to ascertain organisational capacity. Little information is directly collected from individuals and families, particularly those who have difficulty communicating, and little attention is paid to practice of support staff. Providers can be assessed as meeting the Disability Standards without regard for how effectively the support they provide makes a difference in people’s lives, that is, achieve outcomes. In the absence of direct information about achievement of outcomes, there is debate about how well the existing accreditation system aligns with the Disability Standards. There is potential for existing accreditation systems to develop and respond to meeting changing requirements for measuring outcomes.

Although systems for monitoring, including standards have been adopted across many human service sectors, there is negligible research about their effectiveness. Compliance with various quality systems has become separate from practice and the goal of improving support for people with disability. This separation brings with it the risk that quality systems potentially direct resources and organisational effort away from practice and improving outcomes for people with disability. Indeed, comments from the field suggest that years of attention to meeting the Disability Standards has resulted in improvement in organisational processes but only negligible improvement in the standard of staff support practice.

Across the field there is broad agreement from all perspectives that ‘what makes a real difference’ depends on knowing and responding to the needs of each person

Page | 12

with a disability. Practice and organisational processes are both important for ensuring personal outcomes and effective services. It is now recognised that what staff at various levels in the organisation do and the model of support practice are directly related to achieving outcomes for individuals being supported. In the case of direct support staff for example, using appropriate means of communication, using an enabling relationship, having the means to know about a person even if they do not themselves know the person well, being observed and coached by supervisors and practice leaders all contribute to good practice. Organisational attention to staff practice and effectiveness also incorporates opportunities to respond to risk and safeguarding. Observation of direct support practice, by at least staff supervisors, is a critical element of monitoring service effectiveness.

Outcomes, satisfaction and service effectivenessIndividuals, families and funders want information about provider service delivery effectiveness. ‘What are the characteristics of a good service?’ ‘Is this a good service?’ Funders need to be able to monitor and regulate providers, ‘Is what was funded being effectively provided?’ Individuals and their families want to compare between organisations or know about the performance of their chosen organisation. Individuals and families rely most on their networks and are often unsure about the validity of provider marketing information. Not all individuals and families have networks to draw on. Some people with more severe impairments cannot represent themselves or their experiences. Importantly too, the perspectives of individuals and their families are not always the same.

Satisfaction with a service is an important source of information for funders and individuals and families, as well as feedback to providers. Service effectiveness is not the same as satisfaction. Satisfaction describes someone’s personal experience. Service effectiveness captures the overall performance of a service, considering what has happened for the many people being supported and over time.

Satisfaction provides individual information about the experience of a service. Research reveals people tend to be generally satisfied with vastly differing circumstances and service delivery quality. This underlines the importance of distinguishing between information about personal preferences and satisfaction and evidence for service effectiveness.

For example, a service may be consistently increasing the opportunities for choice and control for individuals on the indicators and measures indicated, (such as staff arriving on time so people can plan their days), but that individual may not be satisfied with their experience. Conversely, staff may arrive late consistently and an individual may say they are satisfied due to liking the staff, even though the staff are not meeting the performance requirement for the service.

Page | 13

Identifying which outcomes matter involves making practical decisions about the most relevant outcomes rather than measuring all possible outcomes. The situation has been a little confused for several reasons. These include attempts to be overly comprehensive by including every possible individual outcome as if they were of equal relevance, reducing personal outcomes (for example control over life decisions) to very limited range of simple general indicators that are not well tailored to people with varying impairment levels (for example choice of meal preference), and confounding individual outcomes (for example community inclusion) with organisational outcomes (for example financial cost of supporting community inclusion).

There has been a search for the perfect outcome measurement tool and less attention given to other variables (such as organisational practice capability and staff practices). Developments in other countries have seen public information about organisational performance, such as systems to rate organisations so that individuals and families can compare between organisations. Rating of services is being done independently, publicly and transparently through government regulators and funders. There is less reliance on paperwork and greater emphasis on observation of practice within services with observations being undertaken from several perspectives (people with lived experience and professional specialists knowledgeable about the specific service situation). The United Kingdom in particular is reforming the Quality Care Commission following scandals within a residential setting that had successfully passed the accreditation requirements.

There is little information about how to collect service level information about outcomes for people with disability and changes over time as a result of receipt of support. Such data collection needs to be practical, simple, cost and time effective/efficient; and sensitive to priority areas, rather than attempting to monitor and describe everything. Comparative information about provider effectiveness is not available currently to the South Australian Government (or elsewhere in Australia) for individuals and families, and many services do not gather data about their overall effectiveness. While all of the organisations consulted said they had information about each person and how they were supported, none kept aggregate performance data about the overall service effectiveness of the provider. Individuals and families had no expectation of gaining information about an organisation’s effectiveness, and therefore were likely to rely on ‘word of mouth’ and informal networks for information. Some people being supported by providers reported frustration with the lack of attention to their service delivery experience.

Accreditation attends to organisational capacity, processes and policies; service effectiveness attends to what is achieved for individuals as a consequence of direct support practice. An effective service is one that provides support that makes a

Page | 14

difference in people’s lives. How a service ensures the quality of staff support received by a person with a disability should be the essence of the performance of the organisation and a quality monitoring system.

Page | 15

Part 2: Service Effectiveness Framework

Why service effectiveness?There is a gap in the current quality system and that gap is service effectiveness. Typically external and internal monitoring, review and accreditation processes limit their attention to organisational processes, assuming that good organisational processes mean the quality of staff support practices are assured. Experience and research is revealing this assumption is not correct. More attention needs to be directed to service delivery effectiveness for several reasons. Data about service effectiveness enables organisations to be confident the support they provide makes a difference in people’s lives; individuals can know the strengths of organisations and compare between organisations in the context of their own personal priorities; and government funders can be sure key public policy goals are being progressed with the funding allocated. Effective services, that provide support which increases opportunities for choice and control and individualised lifestyles, will increase life quality and establish safeguards for people with disability living in the community. Service effectiveness information provides answers to core questions from the perspectives of individuals and families, providers and funders:

1. How do individuals choose the right service for them?

2. How do providers know they are effective in meeting specified outcomes?

3. How does government know public policy goals are being progressed; that funded services are effective?

Outline of the frameworkThe framework is one part of a quality system. It is not a ‘quality’ framework in the sense this term has been used. It is a way to measure if and how the support provided by funders of disability services makes a difference in people’s lives. It provides a way for individuals and families, and government, to identify effective services and for providers to be confident they are delivering effective services. It is a mechanism for everyone.

The framework aims to satisfy government funder requirements for monitoring accountability and effectiveness and contribute to informed selection of disability service providers by individuals and families. From the perspectives of funders, individuals and their families, the goal is public information about the performance of organisations. The goal is to monitor and publicly describe services in terms of their ability to organise and deliver support that ‘makes a difference in people’s lives.’ The result for organisations is specification of what they provide and how effective they have been in providing it.

Page | 16

The framework is derived from four core individual outcome domains. The service effectiveness indicators reflect how well each provider responds to the people being supported, given their service model/s, business model and program logic. These indicators can be used by organisations to identify and measure service outcomes within their organisations and by individuals and families to guide their own enquiries and observations of organisations.

Disability service organisations will develop their service effectiveness indicators based from one or more of the four individual outcome domains. Each provider can specify ‘how they provide support to make a difference in people’s lives’ by specifying their model of staff practice and intended outcomes. Evidence for effectiveness will be collected by providers from various sources; aggregate trend data about changes in individual outcomes, internal and external observation of practice, and responses by support staff and individuals/families who use a service to questions from external evaluators or other individuals and families.

Page | 17

There is guidance for organisations developing service effectiveness indicators and how these might be measured using aggregate data systems and observation of staff practice. There is also guidance for review teams, individuals and families when visiting and observing services – for example, what to look for; what to ask about to identify effective and ineffective support provision by services. For individuals and families, this information is in addition to personal preferences, priorities and budget considerations.

Individual outcome domains are the foundation of the frameworkConsultations and review of literature supported the selection of four key individual outcome domains as the foundation of the framework. Using terminology common throughout public policy and research, the four individual outcome domains are:

1. Choice and control

2. Personal capability

3. Relationships and community inclusion

4. Purposeful life and meaningful activities.

These domains are significant because the vulnerability of people with disability is reduced if they have more control over their lives, are able to live life more independently and capably, and have more community members in their lives.

Aligning individual outcome domains and Service Effectiveness DomainsWhile individual outcome domains are at the core of the framework, it is the evidence from providers at the service level which is the vehicle for determining effectiveness. The focus is on the effectiveness of the service for all the people who use it rather than considering each person individually. This framework is concerned with the collective and trend data about these domains for all individuals supported by an organisation. Providers are simply being asked to demonstrate the effectiveness of their approach to providing support.

Each individual outcome domain aligns with a service effectiveness domain. The different orientation of funders given public policy priorities, individuals and service providers in relation to the outcome domains are shown in the table below.

Individual outcome domains and service effectiveness domains

Public policy prioritiesFunded disability services increase

Choice and control

Page | 18

Individual outcome domains‘I have a say over my support’

Provider service effectiveness domains‘We make sure that you are in control of our support – important for all providers.’

Public policy prioritiesPersonal capability

Individual outcome domains‘I receive the support that I need.’

Provider service effectiveness domains‘We deliver the optimal support for you.’

Public policy prioritiesRelationships and community inclusion

Individual outcome domains‘I am supported to be active in my community with other community members’

Provider service effectiveness domains‘We support you to be active in your local community with community members – relevant for some providers’

Public policy prioritiesPurposeful lives and meaningful activities

Individual outcome domains‘I am supported to live how I want to live, doing what I want to do.’

Provider service effectiveness domains‘We support you to live how you want to live, doing what you want to do – relevant for some providers.’

Choice and control means for each individual ‘I have a say over my support’. People living with disability who have choice and control are able to act on preferences about immediate and longer term aspects of their lives. Service providers would demonstrate that, collectively, the individuals who they support are able to make decisions about their everyday lives and have control over the impact of these

Page | 19

decisions. The ‘choice and control’ outcome domain incorporates collaboration with individuals and families, individual safety, creation of opportunities and enabling dignity of risk. This outcome domain is relevant for all providers as it is important for all people with disability as it demonstrates how, from a service perspective of providing support: We make sure you are in control of our support.

Personal capability means for each individual ‘I can get the support I need’. People living with disability are supported to achieve, maintain or recover their maximum individual capability in the community and at home. Providers would demonstrate that, at a service or program level, they deliver effective, tailored support that optimises one or more of the following: learning, skills development and maintenance, and/or personal and health care, and/or behaviour support, and/or total communication support, and/or mobility and physical capacity and/or emotional well-being and recovery or other relevant evidence based practices. This outcome domain will be relevant for some service providers supporting some people with disability who require support in specific areas such as skill development, behaviour, or communication. The requirement is to demonstrate how, from a service perspective: We deliver the optimal support you need.

Relationships and community inclusion means for each individual: ‘I am supported to be active in my community with other community members’. People living with disability are supported to develop and maintain relationships with community members including relationships with peers. Providers would demonstrate that, at a service or program level, they deliver tailored support that links and involves people with disability with various community groups and social relationships. This outcome domain will be relevant for some providers supporting those people with disability requiring assistance to make links in their local communities to demonstrate how, from a service perspective: We support you to be active in your local community with community members.

Purposeful lives and meaningful activities means for each individual: ‘I am supported to live how I want to live, doing what I want to do.’ This means people living with disability participate and engage in home and community life every day. Service providers would demonstrate, at a service or program level, that they provide individually-focused support that ensures people living with disability are planning for, and enabling them to lead lives that matter to them and are doing things that are important to them, such as employment, hobbies, volunteering, home life or learning and education. This outcome domain will be relevant for some service providers supporting people with disability who require assistance to fully participate and engage in daily life to demonstrate how, from a service perspective: We support you to live how you want to live, doing what you want to do.

Page | 20

The framework specifies service effectiveness domains. Each service provider would develop service effectiveness indicators and measures relevant to their program/s, or in the language of the NDIS, their support products.

Service effectiveness indicators and measuresService effectiveness indicators will be relevant across all types of organisations but the specifics will differ for each organisation. The service effectiveness indicators developed by each organisation, will be consistent with its operations, expertise and priorities. Service effectiveness indicators are derived from service effectiveness domains (see the table in the last section). Each provider would develop indicators to describe the success or impact of each type of service, program or support (the products) it plans to deliver and decide how these are best measured at service level. Service effectiveness indicators are not a description of what happens for each person, but an overall indicator of the impact or achievement of the support the service delivers. Service effectiveness indicators consider the cumulative impact of the support provided by a service to respond to people receiving support. Service effectiveness indicators answer questions like: What is this service successful at providing that makes a difference in people’s lives? What is this service good at?

Service effectiveness measures are the evidence gathered through data and observation to demonstrate how the service effectiveness indicator has been achieved. Service effectiveness measures are not service types, like in home support, respite or shared living. Service types are not an achievement on their own. Effectiveness measures ask the next question: what is the evidence that people supported are better off in their lives as a consequence of receiving support?

The table below describes the relationship between service effectiveness domains, indicators and measures. Refer to the Toolkit - Guide 1: Establishing Service Effectiveness Indicators and Measures for some examples for providers developing indicators and measures.

Service effectiveness domains, indicators and measures

Service effectiveness domainsDefined in the Services Effectiveness Framework

Service effectiveness indicatorDeveloped by each provider

Service effectiveness measuresEvidence for the indicators developed by each provider

Page | 21

Service effectiveness domains‘We make sure that you are in control of our support.’

Service effectiveness indicatorImportant for all providers

Service effectiveness measuresData and observation

Service effectiveness domains‘We deliver the optimal support for you.’

Service effectiveness indicatorRelevant for some providers

Service effectiveness measuresData and observation

Service effectiveness domains‘We support you to be active in your local community with community.’

Service effectiveness indicatorRelevant for some providers

Service effectiveness measuresData and observation

Service effectiveness domains‘We support you to live how you want to live, doing what you want to do.’

Service effectiveness indicatorRelevant for some providers

Service effectiveness measuresData and observation

Who the organisation is supporting (or aiming to support in a developing market) determines which of the service effectiveness domains is relevant as not all domains will be relevant to each provider. For services supporting people who easily manage and direct their own support (for example, people with high personal capability such as many people with physical and sensory impairments) demonstrating an effective response to increasing choice and control may be sufficient. For these people,

Page | 22

choice and control will act as the catalyst that enables them to increase their own independence or relationships in the community as they wish. For example, a service effectiveness indicator (or support product) may be defined as ‘providing staff in your home that match your lifestyle, arrive on time, and provide skilled support to match your needs.’ If that were to be the support product, then the task for the organisation is to find a way to measure this product which can be internally and externally verified. Relevant measures might be records of staff arrival times, feedback from individuals on a frequent basis about arrival time, and/or observation of what staff are doing, that is staff practices.

Services supporting people who find it difficult to direct their own support, such as those who have cognitive impairments (intellectual disability, autism, acquired brain injury, and psychiatric disabilities) and more severe impairments or dual disabilities, will need to demonstrate more than choice and control. Primarily because people with cognitive impairment or complex multiple needs, require support to increase their experience and ability to choose, to be engaged in meaningful activities, and to link to the community and to maximise their abilities for their chosen lifestyle.

Developing service effectiveness indicatorsThe framework is not a rigid system which service providers must adapt their internal operations to fit into. It is a process whereby providers can develop a means to demonstrate how their internal operations and the supports they deliver make a difference in the lives of people with disability. Service effectiveness often also links directly to organisational strategic planning, performance evaluation and reporting against mission statements. At the least, service effectiveness indicators and measures, complement existing compliance and accreditation processes and provide stronger evidence of meeting the Disability Standards. At best, service effectiveness indicators should assist providers to define and monitor their support products purchased by people with a disability using NDIA funds.

The framework is relevant for all disability service organisations including low and high intensity services, different service types and designs, as well as different organisational forms (for example large versus small or profit and not for profit). Service effectiveness indicators and measures can be tailored to reflect the needs of people with different disabilities, different types and intensity of support needs, across the spectrum of ages and the life cycle (such as children and adults). Indictors and measures could also reflect the varying cultural, indigenous and geographic (remote/rural/urban) considerations that providers must be sensitive to.

Service effectiveness indicators developed by service providers would be determined from the needs of the people they are supporting, at what cost, how, how often and people’s personal priorities. Service effectiveness indicators could also be

Page | 23

for the organisation overall – that is, ‘this is what we always attend to’ or could be program specific reflecting different programs and circumstances of people supported. The description and evidence of the measures to meet the service effectiveness indicators for an organisation overall will also vary depending on who is being supported, where, the model and the intensity of the support. The support offerings, or products, are then judged to be effective services in one or more of the service effectiveness domains. As already noted, this is not reporting against traditional service types – with individualised funding these service types will cease to exist in the same way. (Refer to the Toolkit - Guide 1: Establishing Service Effectiveness Indicators and Measures for further details).

Service effectiveness can be monitored internally and externally using the same information, data and observations. Service effectiveness measures will be based on longitudinal performance data developed by each organisation. Service effectiveness indicators can be developed along many different measurement dimensions, dependent on what the organisation is aiming to provide and achieve for whom. Examples of service effectiveness indicators (or support products) are:

We ensure personal care staff arrive on time so people can maintain their preferred lifestyle.

We provide reliable household assistance so people can maintain their preferred lifestyles.

We maximise personal control using design and technology, which also minimises the need/ cost for support staff.

We maximise learning and development and reduce the intensity/cost of support.

We support people to develop skills to self-manage their support arrangements.

We develop tenancy and coping skills to maintain stable housing.

We enable people to have a typical pattern of life (work, volunteer, socialise and/or learn).

We increase the time people can spend in typical community settings.

We increase the relationships people have with community members.

We support people to be engaged in paid and volunteer work.

We respond to people’s changing needs.

We increase people’s engagement in meaningful activities as part of their day to day life.

Page | 24

Fundamental to the framework is the development of indicators that can be measured. Organisations need to be able to demonstrate to people external to the organisation, as well internally to Board members, senior managers and supervisors, how their specified domains of effective service delivery are satisfied. A measurable service effectiveness indicator requires:

A description and rationale for the measure, aggregate performance data evidenced across time (longitudinally).

Observation of support and discussion with the individual and or their family and support staff by supervisors and managers, and complemented from time to time by an external team if possible (such as a service review or evaluation).

Measuring service effectiveness indicatorsMeasurement of service effectiveness indicators will arise from what each organisation is aiming to do, within each service effectiveness domain. For example, for ‘choice and control’, ‘We make sure you are in control of our support’ requires trend data and supporting observations about having observed, or been informed about measures.

For example, the measurement of ‘choice and control’ may include:

Examples of records and trend data demonstrating ‘choice and control’.

o Staff arrival times and reliability.

o Feedback from individuals ‘shift by shift’. Opportunities for people to choose and train their own staff.

o Reduction of restrictive practices, including the use of chemical restraint.

o Contingency and back up arrangements are planned for and used.

o Flexibility possible for available staff hours.

o Proportion or change in amount of time spent in the community.

o Opportunities to change daily routines.

Examples of observation and feedback from individuals and their families, and support staff that what is planned is occurring.

o Staff have the skills and are providing good support for each person – doing what is wanted in the right way.

o Individuals are spending time in community settings.

Page | 25

o Individuals are engaged in daily life rather than spending time doing nothing.

o Staff are spending more time providing support than paperwork and administration.

In summary, the task for each organisation is to provide verifiable aggregate data about their effectiveness against one or more service effectiveness domains and to demonstrate good practice during observations by supervisory staff, individuals and families and independent evaluation teams. The questions asked by individuals and families and through internal practice review processes, are also relevant for any external evaluation team (Refer to the Guides in the Toolkit).

Emphasising the importance of observation of staff practiceIdentifying effective service delivery depends on observation. Observations provide a check on longitudinal and trend data.

The Toolkit contains guides about observation. Observations can be undertaken internally by the organisation, by individuals and families and in a more formalised approach by review teams that have multiple perspectives, such as individuals with disability, family members and professionals relevant to the service delivery being observed – with the option for unannounced visits in home and community settings.

Observations involve visiting, asking and observing.

Visit people being supported in their home, workplace, learning setting, community setting – with their permission.

Observe direct support staff providing support. The Toolkit contains resource materials to guide observations and visits.

Ask direct support staff, people receiving support, and their families, as well as managers and supervisors.

Benefits of the Service Effectiveness FrameworkOne of the challenges for the NDIS and a market approach to support products is ensuring information is available so that individuals and families can benefit from the competition between providers. Informed consumers are essential if the NDIS is to meet its goals. If organisations make progress in the domains of service effectiveness, then the information available to people with disability and their families would be greatly enhanced. Individuals and families would become more informed consumers; progress would be consistent with the NDIS aims (choice and control; increase social and economic participation); and providers would be more

Page | 26

oriented to a market perspective through a clearer specification of their support offerings (that is, products) and their effectiveness.

Meeting the information needs of all involved.There are three groups interested in whether government funded services make a difference in the lives of people with disability.

1. Individuals and families want to be able to identify good services based on reliable information and where possible to compare services and to make informed choices about services to meet their needs, budget and circumstances. People using disability support services also want to give feedback about services and express their satisfaction (or not) with services; and to obtain, compare and contrast information from other individuals and families. It is also important for individuals and families to know what are reasonable expectations to have of providers in terms of the breadth and boundaries of the service response/s being offered.

2. Organisations providing disability support want to develop, deliver and monitor effectiveness of the support they deliver against their mission statement, business model, target populations, specialisation, service design and local community priorities. These activities are largely internal but form the basis of responses to external stakeholders if or as requested. This includes providing information to, involving and seeking feedback from, current and prospective people with disabilities about service delivery performance, effectiveness and satisfaction; and satisfying government reporting, funding and accountability requirements.

3. Government has public policy goals and therefore an interest in knowing the effectiveness of service delivery consistent with these policy directions. That is, to know that services make a positive difference in the lives of people with disability and their families. Government is also concerned with minimising and managing risks to vulnerable children and adults with disabilities and ensuring efficient and accountable expenditure of public funds. Increasing attention to service effectiveness will contribute to better expenditure of government funds and greater protections for people with disability.

Implications for implementationService effectiveness indicators provide relevant information for providers, individuals and families and government. It is possible to modify and change the domains without undermining the intent of the framework. The framework can adapt and change as it does not stipulate a rigid process or specify set content.

Page | 27

Nonetheless from all perspectives, some effort is required to develop, observe, measure and monitor indicators of service effectiveness.

Information about service effectiveness empowers people with disability to manage and control their own support and get the right support for them. NDIS will not achieve as much as it could if individuals and families don’t have information about what constitutes good support and the performance of providers. Each individual and or family will need to know what is important to them, and research, visit and ask questions of potential service providers. Individuals and families can compare organisational information on the service effectiveness indicators that are important and relevant to them and can choose the service they prefer based on all information sources and their own preferences.

Knowing about service effectiveness is important to government funders as evidence that public funds are progressing public policy goals, that is, disability support providers are making a difference in people’s lives. At funder level, there would be data available, including from direct observation, about each organisation’s effectiveness in key policy areas in addition to any satisfaction, complaints and feedback comments of people using services; and other compliance and accreditation information.

The implication for services providers is the need to develop customised indicators and measures, rather than adopt another ‘recipe’ or ‘tick and flick’ approach. Chief Executive Officers and their Boards should expect to monitor support practice performance based from aggregate data to the same extent and with the same interest as they do financial and compliance data. Support staff can expect observation of their practice (that is, how they provide support) to be part of supervision and training and to be asked their opinion about what happens at service level. Supervisors can expect to be leading, observing and modelling good practice.

Each person with disability, and/or their family, can expect to be asked about their experiences of service effectiveness and to be part of practice being observed at least by internal supervisors, and by external review teams with their consent.

Page | 28

Part 3: summaryThe concept of service effectiveness is new for the disability sector. Service effectiveness establishes a framework against which organisations can demonstrate their performance to individuals and families and to funders.

This direction towards providers demonstrating their effectiveness to achieve outcomes is consistent with the NDIS in regard to the:

development of a market in support products

need for providers to describe their products in terms of the support they provide and the difference it makes to people’s lives, rather than traditional service types

for individuals and families to know what support is on offer and how effective a provider is in achieving specified outcomes.

This publicly available information then supplements the personal priorities, satisfaction and other sources of informal information available to individuals and families when making their decisions.

The framework addresses the tension for providers between responding to all possible individual outcomes and the limits of what is possible for any one service provider. That is, defining their support products. With knowledge about service effectiveness, individuals and families are better placed to choose and direct available support offerings. Gaps and opportunities in support delivery become explicit revealing opportunities for providers to prepare and respond in the NDIS market place.

Next steps – using the Service Effectiveness FrameworkThe framework can be introduced and developed in several ways. Implementation by providers could be readily incorporated into strategic planning and governance processes. Providers can independently use these materials to demonstrate internally and externally their service effectiveness.

Individuals and families could also use these materials independently as part of comparing, choosing and reviewing their provider arrangements. It is noted that individuals have to be aware of what services are available in order to compare them and in some geographic areas there is not a choice of providers (or no provider) and a shortage of support workers. Pursuing service effectiveness is not relevant under conditions where there are no choices of provider, which often occurs in some rural areas or for specific types of disabilities.

Page | 29

The creation of a credible independent system for evaluating and publishing service effectiveness information is a longer term task beyond this initial implementation stage. Such a system would need a formal coordinating, mediating and mandating role from the funding body and this function is likely to sit within the future responsibilities of the NDIA. Such developments could occur in conjunction with initiatives to provide publicly available satisfaction ratings from people using services and other regulatory roles undertaken within a nationally consistent quality and safeguards framework. There is potential for implementation through the extension of existing accreditation requirements.

In conclusion, this project clearly demonstrates the significant benefits to individuals, families, service providers and funders alike in using a service effectiveness framework to monitor the performance of disability service providers in providing support that makes a difference in people’s lives. Disability SA is committed to sharing this resource publicly with the intention of:

encouraging widespread discussion

helping create an impetus for change

helping people with disability, families and providers prepare for the full scheme implementation of the NDIS

contributing to the discussions about the development of the NDIS National Quality and Safeguards Framework.

Page | 30

Part 4: appendix

Discussion of key ideas and termsThe following terms are important to the framework. The terms are described together with some of the issues associated with their use. Note: items in italics in text have separate entries.

AccreditationExternal accreditation for health and human services against national, state or sector service standards in key areas is linked to target groups (such as, children, families, and people with disability) and service types (such as, respite, rehabilitation, residential). Accreditation is a form of quality assurance. Meeting service standards based from independent third party accreditation continues to be a minimum expectation on providers in order to deliver services. There are different disability quality systems throughout Australia. Research and experience has found that accreditation systems are not sensitive to improving support for people with disability.

ComplianceCompliance is related to quality assurance and monitored through regulation. Compliance involves the creation of standards and monitoring of compliance with these standards. Typically compliance relies on organisational level process and paperwork based reporting.

Individual outcome domainsThere are many possible individual outcomes. Four key individual outcome domains have been selected to underpin the service effectiveness domains. These individual outcome domains are: choice and control, personal capability, relationships and community inclusion and purposeful lives and meaningful activities.

Observation of supportResearch and experience show that an important part of knowing if a service is effective, is observing how support is provided.

OutcomesAttention is starting to be paid to how to achieve outcomes for individuals. Outcomes are the result of service delivery, that is, what difference the support has made in someone’s life. In broad terms, positive outcomes relate to ensuring the safety and wellbeing of vulnerable individuals who are receiving publicly funded services and supports. Research and experience has revealed what staff and organisational

Page | 31

practices are needed to achieve outcomes, that is, to make a difference in people’s lives.

Quality assuranceQuality assurance requires monitoring and maintenance of minimum acceptable standards, and is particularly relevant in relation to accountability for public expenditure and safeguarding vulnerable people. Quality assurance is related to compliance.

Quality improvementQuality improvement is focused upon providers building opportunities for people living with disability and their families. Quality improvement looks at development beyond minimum standards. Helping organisations to continually improve the effectiveness of the services they deliver is an important component of quality improvement beyond what is expected for compliance and quality assurance.

Quality systemsQuality systems involve internal and external activities to an organisation that incorporate both quality assurance and quality improvement strategies including the use of input, output, process and outcome measures to understand organisational performance and lead to positive outcomes for people being supported.

Service effectiveness domainsService effectiveness domains align with the four categories of individual outcome domains, but take the perspective of the performance of the provider. Effective disability support providers achieve in one or more of these domains: choice and control, personal capability, relationships and community inclusion and purposeful lives and meaningful activities.

Service effectiveness indicators and measuresService effectiveness indicators and measures capture how each service responds to the circumstances of the people being supported to make a difference in their lives. These indicators are reflective of provider effectiveness to respond to the individuals they support overall. Service effectiveness indicators and measures describe what is likely to happen for each person as the organisation responds to their individual circumstances. All organisations would develop service effectiveness measures relevant to choice and control, that is, ‘We make sure you are in control of our support by ….’. The ‘by’ is the service effectiveness indicator.

Put another way, service effectiveness indicators are service level measures of the success or impact of each service/program providing the supports (the products) it purports to deliver. Service effectiveness indicators are a measure across the

Page | 32

organisation (or program) of the organisation’s success in responding to the service effectiveness domains it prioritises. Service effectiveness measures are how each organisation monitors progress of each service effectiveness indicator. Meeting service effectiveness measures requires longitudinal and trend data as well as observation of staff support practice.

Staff support practicesStaff practice refers to the range of ways individuals with disabilities are supported. This includes aids, equipment and design, assist dogs, direct support and personal care workers, community development and case managers and professional staff (such as therapists).

Achieving individual outcomes is inextricably linked to staff practices, that is, what staff are doing. Research and experience show that an important part of knowing if a service is effective, is observing how direct support is provided. Supervisors, external reviewers and people receiving support are all involved in observing staff practice.

Page | 33

Toolkit

Quality Systems and Outcomes Measurement Project

A Service Effectiveness Framework: moving beyond compliance to quality outcomes for consumers

IntroductionService effectiveness indicators help providers to be more explicit about what they are offering and how well, so individuals can then choose or review their support. For an organisation to do this requires performance data about service delivery including observation of practice and performance over time.

Every service must demonstrate how to maximise choice and control as relevant to the people being supported, and what is being provided for the available cost. There is also a role for external reviewers and assessors critiquing organisational effectiveness. This is likely to increase in the future with the move to public information about organisational performance which can be compared.

Knowing what to look for and ask about to identify good practice is important for all of these perspectives.

The four guides in this toolkit have been designed with these multiple requirements in mind. The intention has been to have the same information available for everyone, as everyone is interested in service effectiveness, albeit from differing standpoints. As a result, some of the material is repeated in different formats across the guides. The project report is relevant background reading.

Guide 1 – Establishing indicators of service effectiveness: outlines for providers how to develop service effectiveness indicators and measures particular for each organisation.

Guide 2 – Monitoring service delivery: is organised according to the outcome domains and provides information for anyone when visiting an organisation and observing service delivery about ‘what to see and what to ask about’. The guide describes features of good organisational and staff support practice, identified from research and feedback, and associated with ‘making a difference in people’s lives’. This information is relevant for individuals and their families, independent assessors and reviewers and government regulators or funders. That is, all roles and interests associated with monitoring and reviewing direct service delivery.

Page | 34

Guide 3 – Observing indicators of service effectiveness: proposes a process for observing service delivery and asking questions when individuals, perhaps with their family members, are visiting services. The guide builds from the outcome domains and arranges the material to reflect how a visit or observation takes place. The guide recognises the range of considerations which may be important to individuals and families, including the characteristics of good organisational and support practice. This guide could also be used by internal review teams or external assessors.

Guide 4 – Signposts of effective services: offers tips for identifying effective and ineffective services. This material could be used by individuals in conjunction with Guide 3; by internal review teams and external assessors as a screening process; and as part of staff supervision, coaching and training.

Using the Service Effectiveness FrameworkThe framework can be introduced and developed in several ways. Implementation by providers could be readily incorporated into strategic planning and governance processes. Providers can independently use these materials to demonstrate internally and externally their service effectiveness.

Individuals and families could also use these materials independently as part of comparing, choosing and reviewing their provider arrangements. It is noted that individuals have to be aware of what services are available in order to compare them and in some geographic areas there is not a choice of providers (or no provider) and a shortage of support workers. Pursuing service effectiveness is not relevant under conditions where there are no choices of provider, which often occurs in some rural areas or for specific types of disabilities.

The creation of a credible independent system for evaluating and publishing service effectiveness information is a longer term task beyond this initial implementation stage. Such a system would need a formal coordinating, mediating and mandating role from the funding body and this function is likely to sit within the future responsibilities of the NDIA. Such developments could occur in conjunction with initiatives to provide publicly available satisfaction ratings from people using services and other regulatory roles undertaken within a nationally consistent quality and safeguards framework. There is potential for implementation through the extension of existing accreditation requirements.

There are significant benefits to individuals, families, service providers and funders alike in using a service effectiveness framework to monitor the performance of disability service providers in providing support that makes a difference in people’s lives. Disability SA is committed to sharing this resource publicly with the intention of:

encouraging widespread discussion

Page | 35

helping create an impetus for change

helping people with disability, families and providers prepare for the full scheme implementation of the NDIS

contributing to the discussions about the development of the NDIS National Quality and Safeguards Framework.

Page | 36

Guide 1 – Establishing service effectiveness indicators and measures

Primary audience: Providers

Secondary usage: Assessors for the purpose of evaluation and review

Purpose of this guide: The purpose of this section is to assist disability support providers to set up service effectiveness indicators and measures relevant for their organisation, expertise and priorities.

Background reading: Report – Part 2: Service Effectiveness Framework

Process for providersOrganisations can commence this process now, irrespective of how fully or quickly the framework develops systemically. This entire framework is about strengthening organisations and helping them to communicate their strengths to the market place, so that people with disabilities have clearer Information about each organisation’s offerings and the organisation’s track record with delivery.

In order to use service effectiveness indicators and measures, services providers will need to work through four steps:

1. Incorporate the development into strategic planning and review

2. Develop service-specific indicators

3. Identify how these service-specific indicators will be measured

4. Evaluate service specific indicators and measures internally and externally. Each of these steps is considered in the following sections.

1. Incorporate the development of service effectiveness indicators into strategic planning and review

The development of service effectiveness indicators fits well with strategic planning processes and for many organisations much of the information needed will be a regular aspect of strategic review.

Strategic decisions about the organisation’s mission, business model, target populations, specialisation, service design and local community priorities underpin the development of service effectiveness indicators. Details of current or intended target groups and the organisation’s model or models of support are needed. Target group details include key characteristics such as age, support needs, and geographic boundaries; support model information includes details about intensity, practice, cost, staffing and other resources.

Page | 37

2. Develop indicators of service effectiveness.The strategic intent of an organisation determines what the right service effectiveness indicators are for that organisation. Indicators are specific to each provider and describe what the provider does (or wants to do) best; that is, what is (or could be) the organisation’s support products in the market place. They provide individuals with a means of choosing between providers and reviewing their current support.

Providers decide their service effectiveness indicators and how these are best measured. Service effectiveness indicators describe the success or impact of each service (or program) in providing the service and supports (the products) the organisation purports to deliver. Service effectiveness indicators need to align with one or more of the four service effectiveness domains:

1. Choice and control: ‘We make sure you are in control of our support.’

2. Personal capability: ‘We deliver the optimal support for you.’

3. Relationships and community inclusion: ‘We support you to be active in your local community with community members.’

4. Purposeful life and meaningful activities: ‘We support you to live how you want to live, doing what you want to do.’

For some smaller or homogeneous organisations, service effectiveness indicators will be applicable across the entire organisation. For example, an organisation might have the mission to increase the social participation of people with disability in their local community. This indicator applies regardless of the details of the variety of program offerings. For other more complex, multi program and multi target group organisations, service effectiveness indicators may apply at program level.

Service effectiveness indicators are not a description of what happens for each person, but overall how the impact or achievement of the service can be described and monitored. Service effectiveness indicators answer questions like: ‘What is this service successful at providing that makes a difference in people’s lives?’ ‘What is this service good at?’

The framework is flexible and will be different for different organisations. It is not prescriptive and can be customised by providers to describe and prove what each organisation is good at and does best. One to three service effectiveness indicators are sufficient. Monitoring these indicators can be readily incorporated in to review of the strategic plan and provide valuable, but frequently absent, information for Boards and senior staff about organisational performance.

Page | 38

Where to startThe best place to start is to ask overall, ‘What are we good at?’ ‘What do we do well and how could this be measured?’ As a starting point, some suggestions about typical parameters of support which could be the basis for service effectiveness indicators are provided below:

What people being supported are doing and where. Consider, links to the community, early childhood intervention, opportunities for childhood or adult education and learning, employment, training and volunteering, increased mobility. Also important is support which increases people’s personal capability in areas such as skill development, emotional wellbeing, health and many more.

Adaptations within the personal and physical environment including design, technology, personal aids and equipment which enable greater independence and capability and less reliance on paid staff.

Features of the direct support practice implemented. These include parameters such as supervision, rostering and scheduling, matching between staff and individuals (such as, appropriate age, gender, culture), flexibility, reliability, staff training, tailoring of staff teams and personalisation of support.

Features of the organisation, including culture, leadership, involvement of individuals and families, and performance information. Consider how the organisation ensures that every person supported has an individual support plan that is revised every six months to ensure people’s goals are monitored and met.

Service effectiveness indicators are measurable. Consider the two examples below:

Example 1: Our organisation provides respite care for adults including taking people on social outings.

Example 2: Our organisation increases or maintains the number and variety of relationships each person has.

Example 1 is not an example of a service effectiveness indicator because it is a statement without a measurable component. In contrast, example 2 is an example of a service effectiveness indicator because it contains the measurable component of the number as well as the variety of relationships.

Service effectiveness indicators are not simply sentences to describe features of the organisation. They must align with the service effectiveness domains and be measurable.

Page | 39

Service effectiveness indicators don’t just ‘stand alone’ but need to be appropriately measured – that is to reveal how each organisation is making a difference in people’s lives.

Service effectiveness measures are not service types or hours of service, like day or in home support, respite or shared living. Service types are not an achievement on their own and do not capture the outcomes for individuals. Effectiveness measures ask the next question: what is the evidence that people supported are better off in their lives as a consequence of receiving support?

Aligning indicators with the service effectiveness domainsService effectiveness indicators (what the organisation does well) are required to fit into one or more of the four outcome domains. Providers don’t have to align with all outcome domains but they do have to satisfy the first: choice and control. This is because having choice and control in one’s life is essential for all people. For people who can direct their own support, providers simply need to demonstrate how they support that to happen. People can manage their own lives, relationships, interests, work and so on after that. For some people, who aren’t able to self-direct their support (such as people with cognitive impairments) providers need to have additional indicators and measures in one or more of the other three domains. This is because to maximise choice and control some people will require support with decision making, assistance to be more independent, to plan, to form relationships or to participate in the community.

3. How to measure specific service effectiveness indicators.The framework depends on measurable indicators. Service effectiveness measures are the evidence gathered through data and observation to demonstrate how the service effectiveness indicator has been achieved. Indicators can be measured from aggregate and longitudinal service or program level data, including observation and feedback. This data includes information from reviews, personal outcome tools, feedback interviews, surveys or observations. Not all of these – but the approach that is relevant and feasible for each organisation.

Working out how to measure service effectiveness indicators, requires decisions about what information to gather and how to gather that information which is relevant to who is being supported and how. Measures could be calculated at a set frequency (such as annually, after each staff shift) or could use social network scales or hours in the community.

None of the framework is prescriptive. Just as each organisation can define its own service effectiveness indicators, so too the way they are measured will be specific to each organisation. Measurement should be meaningful, relevant and not onerous. Some organisations will be doing some measuring of their services already. Some

Page | 40

organisations, even if they have some data on service performance, only have the data as it relates to individuals at an individual level. The level at which the data is available is an important distinction of the framework. The data required in this framework is at organisation, service or program level. It is aggregate data not personal data, although it may be derived from individual data.

The reason aggregate data is required is so that organisations can demonstrate that their service is effective over time and that changes have occurred or been maintained. Information on one person achieving personal goals in an organisation does not constitute evidence about overall organisational performance. Data at the level of the individual is very relevant for the individual, but very limited for demonstrating what the organisation has achieved to inform judgements by other consumers, funders, donors or regulators.

Knowing personal data in isolation does not demonstrate service effectiveness; neither does service effectiveness data guarantee what will happen for each individual being supported. Service effectiveness data describes what is likely to occur.

A working exampleThe first task for an organisation is to define their service effectiveness indicator/s. These indicators then need to be aligned to the outcome domains and associated measures need to be developed. Table 3 is a working example of this process. The process cannot be completed without information about who is being supported and how. For example, if the organisation is supporting people who can choose their social relationships, then meeting the choice and control domain could be the only requirement of the framework. The measure/s would reflect this for the people being supported.

Page | 41

A working example of defined and aligned service effectiveness indicators

Service effectiveness indicatorOur organisation increases or maintains the number and variety of relationships each person has.

Measures of choice and control, examples:

Data over timeProportion of people who increase/maintain their relationships

Average number of relationships across individuals supported

Change in these metrics over time

FeedbackAggregate Information from interviews with people supported

The most appropriate measure may be feedback from individuals about the effectiveness of support.

However, if the organisation was supporting people who need support with choice and control, additional measures may be important, such as measures of relationships and community inclusion, such as:

Measures of relationships and community inclusion

Data over timeProportion of people who increase/maintain their relationships

Number of different community settings visited

Number of volunteers working with people

ObservationPeople interacting with community members as part of a community group

The details below give some other ideas about measures arising from this one example of a service effectiveness indicator. Whether measures are needed beyond choice and control hinges on who is being supported.

Working example: service effectiveness domains, indicators and measures

Example of service effectiveness indicatorOur organisation increases or maintains the number and variety of relationships each person has.

Page | 42

Service effectiveness domains for this indicator

1. We make sure you are in control of our support (all providers).

Measures here could include:

How people choose their relationships

How many relationships and change over time

Meeting people’s goals for relationships

How people describe their social relationships

2. We deliver the optimal support for you (some providers).

Measures here could include:

Learning about forming social relationships

Skills learned in order to access community

Learning about decision making

3. We support you to be active in your local community with community members (some providers).

Measures here could include:

How many activities are shared with community members

Number and variety of community settings accessed

4. We support you to live how you want to live, doing what you want to do (some providers).

Measures here could include:

Individual lifestyles supported

Increased engagement in daily activities (less time doing nothing)

Further direction about what to look for and what to ask relevant to each outcome domain is presented in Guide 2: Relationship between Outcome Domains and What to See, What to Ask About which also provides additional examples and ideas about service effectiveness indicators and measures. (Refer also to the Report - Part 2: Service Effectiveness.)

Page | 43

4. Evaluate service effectiveness indicators and measures both internally and independently.

Observational methods are critical for measuring the lived experience of people who can’t easily express their point of view particularly those with severe and profound disability.

The framework introduces the potential for independent evaluation teams directly observing staff at work (see Guides 2, 3 and 4). This is in addition to internal supervision and monitoring processes. Individuals and families should be able to visit organisations and ask and look for the same things using Guides 3 and 4.

Staff in services often work in relatively unsupervised situations, have wide discretion and have to exercise considerable judgment in their work. It is important that someone other than the individual (who may not in any case be able to voice their concerns) and the support staff knows what is occurring well or not at service delivery level. Observations cannot be subject to the type of falsification or omission that has occurred in paper and process driven quality assessment systems.

Organisations can observe staff practice nowOrganisations need internal review processes to review the data, observation and feedback underpinning the service effectiveness indicators. This internal process is comparable to an internal financial audit team and is possible now.

Providers can examine their own organisations and evaluate how effective they are really being. Internal observation is an essential part of the framework. It can be a highly productive exercise for organisations and is essential for verifying actual performance (service effectiveness measures) against claims made (service effectiveness indicators).

Page | 44

Guide 2 – Relationship between outcome domains and what to see; what to ask about

Primary audience: Everyone interested in the effectiveness of disability support services

Usage: Individuals can use this guide to check if an organisation’s service delivery matches their needs and priorities.

Providers can use this guide to further define their service effectiveness indicators and to monitor and evaluate their effectiveness in these areas.

Government and external assessors can use this guide to monitor and evaluate service effectiveness of providers.

Purpose of this guide: The purpose of this guide is to provide prompted guidance for all of the above listed people so that they may observe if a service is effective in the area (outcome domain) that is a priority to them.

The four outcome domains:

1. Choice and control.

2. Personal capability.

3. Relationships and community inclusion.

4. Purposeful life and meaningful activities.

The one essential domain:

Not all outcome domains are relevant to all individuals or to all providers. The one outcome domain that is essential is the first: choice and control. Every organisation must demonstrate that they achieve results in this outcome domain. Providers must define service effectiveness indicators and measures that prove they make a difference in people’s lives in the area of choice and control.

The information below presents an overview of ‘what to ask’ and ‘what to observe’ for each outcome domain. Note how different information arises from what is asked about and what can be observed. It is likely there will be different priorities for each person. The items are viewed as examples rather than specific requirements.

Page | 45

Outcome domains: what to ask and what to look for in service delivery

Outcome domain 1

Choice and controlFor the individual this means:

I have a say over my support.

For providers this means:

We make sure you are in control of our support.

All providers must be able to demonstrate they effectively provide results in this outcome domain.

What to askHow does individual planning occur? Are individual plan goals met?

How does detailed planning about service delivery, including a review process, occur?

What process is used for personalised planning and delivery including evaluating personal outcomes using relevant tools?

How are design, aids and equipment, technology, built environment used to enable safe, healthy environments, maximising independence, capability and safety and minimising need for support staff?

What contingency and safeguarding planning occurs to minimise risk for each person?

How are preferred lifestyles maintained?

How are individuals involved in planning?

Is there direct involvement of each person in planning and implementation of their support arrangements?

For people who can’t communicate their own needs and desires, are there processes to represent their perspective? Insight into an individual’s perspective could be achieved by having two people contributing on their behalf, for example, family and support staff?

Page | 46

How do people know what to expect?

Do staff arrive on time?

How are changes of staffing communicated to each individual and in a timely manner?

Are staff skilled for the support required?

Are staff competent in the mode of communication an individual uses?

Are staff skilled for maximising mobility?

Do staff use lifelong learning, health and personal care, recovery, and/or positive behaviour techniques?

What to look forIs personalised support evident? For example:

Personalised design, technology, aids and equipment are present and are used.

Times for sleep, food, and medication are personalised and vary.

There is choice for individuals about who spends time together and when.

There are signs of personal self-expression, for example, posters on the walls, personal spaces and possessions.

People are doing as much as possible for themselves; staff are not waiting on them.

Physical assistance is tailored for each person.

Each person is not doing exactly what they were doing last visit.

Is the person at the centre of what’s happening?

Staff take the time necessary to allow each individual to respond.

Staff speak to, not about or over, individuals.

Staff are patient and listen.

The person who lives in the house, answers the door or controls how this happens.

Is there a positive relationship?

Do staff like and respect the people they support?

Do direct support staff have rapport with the person they are supporting?

Page | 47

Who is in control?

Are staff controlling the environment, the people, or the timetable?

Are direct support staff competent?

Are people restricted in any way?

Are people confident to speak up?

Is the person relaxed?

Does the atmosphere ‘feel comfortable’?

Does everyone seem happy? Both the people supported and the staff? Is this setting stressful or anxiety provoking?

Page | 48

Outcome domain 2

Personal capabilityFor the individual this means:

I can get the support I need.

For providers this means:

We deliver the optimal support for you.

This outcome domain is not relevant to all individuals or all providers.

What to askHow are direct support staff skilled and supervised?

Do individuals have timely access to professional staff?

How are design, aids and equipment, technology, and the built environment used to enable a safe, healthy environment, maximising independence, capability and safety and minimising need for support staff?

Are staff skilled to provide the support required?

Are staff competent in the mode of communication an individual uses?

Are staff skilled for maximising mobility?

Do staff ensure lifelong learning, appropriate health and personal care, communication, active support and/or positive behaviour techniques?

Do staff know how to promote community inclusion and know how to link people to community members so people are interacting, not just physically present?

Page | 49

Do staff support decision making?

Do staff promote engagement and active support?

How is health and mental health screening undertaken?

Who conducts health check screening?

Who reviews medication?

Who reviews dental checks? How do staff identify and cope if an individual is experiencing distress, discomfort, illness or pain?

What to look forStaff who know how to, and are, supporting people using the relevant practice skills.

Is specialised support evident? For example:

Design, technology, aids and equipment are present and are used.

People are doing as much as possible for themselves; staff are not ‘waiting on’ them.

There is tailored physical assistance for each individual.

Staff practices are timely and do not contribute to discomfort or distress.

People are engaged in whatever is occurring – at home or in the community.

Staff are not getting in the way of people meeting other community members.

Individuals are not left uncomfortable, or distressed and do not have possible illnesses which are not being attended to.

People are not routinely sedated.

Page | 50

Outcome domain 3

Social relationships and community inclusionFor the individual this means:

I am supported to be active in my community with other community members.

For providers this means:

We support you to be active in your local community with community members.

This outcome domain is not relevant to all individuals or all providers.

What to askHow is participation in the community encouraged for each person?

Does each person have a community typical lifestyle which changes from time to time consistent with what each person wants to do?

How is each person involved with mainstream services such as health and mental health, transport, education and work?

How is each person developing or maintaining (as relevant) a spectrum of informal relationships? Such as, peer relationships and personalised relationships.

Are there relationships beyond group-based relationships and relationships with paid staff?

What to look forWhen observing in the community,

People are known, greeted and participating in community and business settings such as the library, a neighbourhood house, swimming pool, cafes and shops.

Each person is supported (not as a group but as an individual) at home and in a community setting.

Each person interacts with people who are not staff (this may require staff assistance).

People have varied and active roles in the community side by side with other community members.

Page | 51

Outcome domain 4

Purposeful lives and meaningful activitiesFor the individual this means:

I am supported to live how I want to live, doing what I want to do.

For providers this means:

We support you to live how you want to live, doing what you want to do.

This outcome domain is not relevant to all individuals or all providers.

What to askAsk about whether people supported have the same patterns of life as other community members.

Ask how people are engaged in their daily life, such as:

Living in locations where there are opportunities for practical individual transport arrangements (including walking).

Provided support that enables people to live individualised lives, to work, to learn, to volunteer and to engage in leisure activities.

What to look forWhen visiting at home or in the community are individuals engaged in/enjoying what they are doing.

This means people not doing nothing and not being alone all the time; they are doing something meaningful.

People are involved in what is going on regardless of their support needs.

There is modification of space to allow for different support needs. There is modification of activities provided to allow for different support needs.

Page | 52

Guide 3 – What to observe and questions to ask about direct support practicePrimary audience: Individuals and, where applicable, family members

Secondary audience: Independent assessors and internal staff for the purpose of monitoring their organisation’s effectiveness or for preparing for assessment.

Purpose of this guide: The purpose of this section is to assist individuals and, where applicable family members, to decide if a service is right for them.

This section gives guidance to internal and independent review teams about what to ask and what to observe as part of service delivery.

OverviewThis guide provides questions to ask and points at things to observe in order to determine if a service has a ‘track record’ of being effective and ‘making a difference in people’s lives’.

Each person can question each organisation of interest to check whether or not the service being offered is a good match to personal needs and priorities.

To be effective, organisations need to be able to demonstrate their performance in one or more outcome domains.

What these four outcome domains mean for providers is:

Choice and control: ‘We make sure you are in control of our support.’

Personal capability: ‘We deliver the optimal support for you.’

Social relationships and community inclusion: ‘We support you to be active in your local community with community members.’

Purposeful life and meaningful activities: ‘We support you to live how you want to live, doing what you want to do.’

What these four outcome domains mean for individuals are:

Choice and control: ‘I have a say over my support.’

Personal capability: ‘I can get the support I need.’

Social relationships and community inclusion: ‘I am supported to be active in my community with other community members.’

Page | 53

Purposeful life and meaningful activities: ‘I am supported to live how I want to live, doing what I want to do.’

Note: the following lists contain repetition of the previous section. The intended audience of the two sections is different, but the content is largely the same. This guide and Guide 2 can be used together or independently of one another.

Remember that every organisation does not have to satisfy each question or prompt.

First steps for individualsThe first step is for each person, perhaps with family members, to clearly define primary needs and preferences through an individual planning process, by:

Identifying the available funding;

Identifying what support is wanted and why;

Describing any of the critical or preferred elements of the support delivery. For example, is the gender, age and language spoken by staff important? Which staff skills are priorities?

Check out all sources of information about an organisationThe next step is choosing the right service or reflecting on what is offered by your current provider.

You can gather information in two ways:

Ask others

Other people who have used services have vital information that may or may not be relevant for you. You can read testimonials, ask around in your network and, where available, you may want to consider reading satisfaction ratings on the service(s) you are interested in.

Visit the provider or providers

The purpose of visiting providers is to observe the organisation and service delivery, and ask questions of the staff and people being supported. Suggested questions that will help you to determine if the service is right for you are listed below. You can ask about the items in these questions or use the questions to guide what you observe in practice.

Service effectiveness questions – what to ask providersThere are questions that can be asked of providers that will reveal a lot about the kind of service being delivered. All questions have come from the four outcome domains (above) and what makes an effective service. Not all questions will be

Page | 54

relevant to all people or services. Some questions will be more important to you than others.

Talking with individuals being supported as well as direct support staff and their supervisors and managers ensures you are getting the right information. Sometimes what supervisors and managers think is happening is not the case in practice.

Read the following questions and think about what are the important areas for you to explore. These questions are not intended as a checklist but as prompts that can be used to guide discussion and observation.

Involvement of individuals and families in support delivery

Can I speak to other people supported by you (with their permission)?

How are people living with disability involved in your organisation?

o What is the role of families? How do you involve families and keep them informed?

o What do you do if you think the perspectives of families are not the same as individuals?

Do you have a process to receive positive and general feedback?

o Do you have an achievements feedback process?

o Do you have a complaints process that people use?

Questions on flexibility of supportAn organisation having limits isn’t a bad thing. By asking these questions, you are conducting a screening or a matching process. It is important for individuals and providers to know each other’s limits and flexibilities and where they do and don’t meet.

Flexibility of service delivery is an essential ingredient of personalised and responsive service delivery. Questions to ask:

If I use this service, will I be able to use the hours allocated to me in a flexible way (in ways that matter to me)?

o Can the hours change on a flexible basis?

o Am I able to use my funding flexibly?

o Do I have to do what everyone else does?

Does each person you support have specific opportunities in the community?

Page | 55

Given my individual needs, does your organisation have limits that would mean those needs aren’t met?

o Are there flexibilities in your policies and operation to allow for my needs?

Service effectiveness in action – what to look forJust as asking questions can reveal a lot about the kind of service being delivered, another powerful thing that you can do to determine if the service is providing what is important to you is to observe what happens in practice.

During your visit, there are certain things to look for that will show you whether or not the service is an effective one. That is, is it a service that makes a difference in people’s lives? Is it the right service for you? Not all services or all organisations will do everything that you need or would prefer. The list of things to look for below outlines different areas. Some of them may be more relevant or important to you than others.

General observations - what can you see now?

What are people doing? What do people say they get to do?

o How engaged and involved are people? Or are they bored, waiting or not occupied?

o Is the place too quiet? Or is it too chaotic?

o What’s happening for people with the highest support needs? Are they getting the support they need or being left alone?

What are the staff doing? What are people doing and what is the quality of staff responses? (This matters more than many physical aspects of the setting.)

o Are some people getting too much support when they could be acting more independently?

o Is there a reason for what is going on now? Is it usual or not?

Observations of the atmosphere

Does the atmosphere feel comfortable or stressful to you?

o Do people being supported seem happy or anxious?

o Are the staff welcoming or defensive? Are they happy to talk?

Who is controlling the environment?

o Are interactions between staff and individuals respectful?

Page | 56

o Are support staff interested in each person?

o Do staff suit themselves in what they do or are they attentive to what individuals need?

o Are you allowed and encouraged to visit all aspects of a service? If some areas are restricted, why? Was there an explanation given?

o How willing is the organisation to give you time to talk with direct support and senior staff?

o How willing is the organisation to connect you with other individuals and families who use the service?

Reflections about direct support staff

How did you find staff in relation to:

How they relate to people being supported

o Considerate, understanding and non-judgemental?

o Respectful? Including office staff?

o Empowering not controlling?

o Responsive to each person’s needs?

o Was there energy around service provision; did staff seem to care? Were staff happy in their jobs?

o Did staff display patience and listen?

o Did staff take the time necessary to allow each individual to respond?

o Did staff speak to, not about or over, individuals?

How informed and trained were staff?

o Knowledgeable of local services?

o Using individualised and preference based staff practice?

Questions to ask the CEO and/or senior staff

Values and direction

o What is the vision of your organisation? What are the values of your organisation?

o How are your values demonstrated in your service delivery?

Page | 57

o How do you know that staff have the right attitudes as well as the right skills?

Organisational performance

o How well do you meet the needs of people being supported?

o How well do you measure quality of what is provided including the perspective of the individual and families?

o How well do you measure achievement against the organisational vision and mission?

o How well are you tracking results data and using individual outcomes information?

o How do you measure organisation-wide outcomes against the organisation’s mission or goals?

o How do your business systems support direct support practice? For example, how do you gather data about implementation of individual plans?

o What is the role of the Board, CEO and managers in service delivery and staff practice?

What is the model of staff support practice?

o Does the organisation have a model of practice (i.e. what staff do)? And is it understood throughout the organisation? What is the rationale and evidence base for your approach to support provision?

o Do you use restrictive practices?

o What are your practices with the use of medication?

o How do your staff work in teams around each person supported? How is support coordinated?

o How do you introduce evidence based best practice?

Individual planning processes

o What is your individual service planning and design process? How is this process reviewed? How will you know if a person is progressing?

o How is safety as well as opportunity maximised? What are your processes for safeguarding?

o How will you know if a person is slipping back and what will you do?

Page | 58

Staff recruitment and training

How well do you ensure staff competency? How do you encourage collaboration within staff teams and with individuals and their families?

o How do you recruit staff?

o Are staff observed regularly by supervisors?

o Do you let staff go if it is needed?

o Do you observe staff in direct support as part of recruitment?

o What are staff trained to be competent in? (for example, Active Support, Positive Behaviour Support, communication, Auslan signing, specific personal care, mobility training, etc).

o What is your staff turnover rate? How long have your staff been working for you?

Summing up your visitHave you spoken to everyone you want to?

Senior staff? CEO?

Direct support staff?

People supported?

Their families?

When you spoke with these people, did they all seem to share a vision of the service/organisation? Is what they said consistent?

Are you comfortable with the goals of the organisation?

How has the organisation demonstrated to you it is an effective organisation? That is, has the organisation shown you how the support offered makes a difference in people’s lives in the ways intended?

Reflecting upon your primary needs, can this organisation provide what you want? That is, locality, accessibility, cost, type and style of support, etc.

Can the organisation provide you with links to peers? Is this important to you?

Is the organisation open to external ideas from others experienced in the field including research?

Page | 59

Guide 4 – Signposts of effective servicesPrimary audience: Individuals seeking or reviewing support

Secondary audience: Any internal and external practice review process; staff supervision and training

Purpose of this guide: This section repeats some of the material in other guides, but from a different perspective. This material is particularly relevant for individual and families new to the disability support system or when the individuals who are being supported can’t easily express themselves.

The information also gives guidance for internal monitoring and training and to independent review teams about what to ask about and what to observe when assessing service delivery.

Overview of this guideThis guide describes what can be observed and asked about to determine what are effective and what are ineffective services. Effective services are able to demonstrate that they make a difference in people’s lives.

Individuals and, where applicable, families could use this guide as a way of making an informed choice about a service. Any choice based upon the following signposts of service effectiveness would also need to take into account personal priorities such as, location, cost, familiarity, impressions, friendship networks, advice from other people using the service and information about satisfaction from others who have used the service.

The information is based upon the four outcome domains (see the Report - Part 2) in which providers should be able to demonstrate their effectiveness (as relevant):

Choice and control: ‘We make sure you are in control of our support.’

Personal capability: ‘We deliver the optimal support for you.’

Relationships and community inclusion: ‘We support you to be active in your local community with community members.’

Purposeful life and meaningful activities: ‘We support you to live how you want to live, doing what you want to do.’

Page | 60

What these four outcome domains mean for individuals are:

Choice and control: ‘I have a say over my support.’

Personal capability: ‘I can get the support I need.’

Social relationships and community inclusion: ‘I am supported to be active in my community with other community members.’

Purposeful life and meaningful activities: ‘I am supported to live how I want to live, doing what I want to do.’

The following information describes example features of services that are effective in these domains and are therefore effective services.

Signposts to an effective serviceAn effective service has many of the following features.

Flexibility and personalisationAn effective service encourages a high level of flexibility and personalisation relevant to each person being supported.

Examples of flexibility are:

Each person being able to use allocated support hours flexibly on a daily basis.

Each person’s opportunities in the community are specific to them.

Possible support arrangements are not solely determined by program and organisational rules.

Encourages autonomy and capabilityAn effective service allows, encourages and respects the autonomy and capability of people being supported, not only in the lifestyle choices they make, but also their involvement in running their support services.

Examples of a service that encourages autonomy and capability are:

Each person is involved in decisions about planning their support.

Staff are trained in supported decision making, if relevant.

Staff are reliable, can listen well and follow instructions well.

Staff respect the privacy of people being supported.

Page | 61

Clear direction and strong valuesThere are common values found throughout the organisation; the direction is clear to everyone. Staff and the people being supported understand what outcomes the service is aiming to achieve. Attempts are not made by the service to do everything.

Examples of a service and/or an organisation with a clear direction include:

All staff are motivated to make each person’s life the life that person wants.

Senior staff are interested in what support staff do.

There is an understanding of the values and goals for the organisation including within the Board, managers, supervisory and support staff and family and wider community members. Chief Executive Officers and their and Board members are actively involved in service delivery and drive service quality.

Individuals and families are part of the support team.

Arrangements to promote best practiceEffective services have systems in place to promote and support best practice at service level. For example:

There is organisational data and/or information and systems that demonstrate the effectiveness of the organisation.

There is organisation-wide orientation and induction training relevant to the mission and values of the organisation and the people being supported.

There is ongoing training that emphasises the model of practice of the organisation (such as Active Support, positive behavioural support, communication, quality of life, recovery, family centred practice, strengths-based practice) for the particular people being supported.

The organisation is open to new ideas and research. There is an evidence base underpinning what staff are trained and supervised to do, relevant to the people being supported.

The resources of the organisation are devoted to staff practice and front line management of how well each person is supported.

Support staff know that their practice is noticed by supervisors and managers and the organisation strongly promotes good practice and strong values.

There are processes to minimise, or, in the best-case scenario, never place unfamiliar staff with people. For example, staff are trained in shadow shifts.

Page | 62

Activities are flexible and are not fixed by organisational boundaries or restrictions. Consider capacity to make changes in rosters and scheduling.

The service enables visible and meaningful interactions with community members. Evidence is available on how people are supported to build meaningful relationships within the community.

The organisation looks to limit medication and restrictive practices.

The service is committed to maintaining a staff team for each person. There is reliability of staff with processes to minimise and manage staff changes. .

Staff workplace safety is balanced with personal outcomes for individuals. Staff professional practice and conduct are maintained. Workplace safety standards and other legal and compliance obligations are met.

Open organisational practices and principles.The organisation is open and wants to involve individuals in their service delivery.

Examples include:

The organisation encourages potential service users to speak to other people supported by the organisation.

Senior staff spend the time to show and explain the service to potential service users. They accompany visitors, show the service, and answer any questions. Senior staff are available for individuals and families to meet and talk to.

Staff are supported and respected by the organisation.

Information is readily available on the phone and online for people considering using the service.

The organisation welcomes complaints and uses feedback to improve services.

The organisation encourages ‘dropping in’ (where this is appropriate).

The organisation has meaningful information available about satisfaction with service delivery.

The organisation listens and responds in a timely manner keeping individuals and staff well informed.

Individuals and families are involved in performance reviews of each worker and annual reviews of ‘my service’, for example my staff arrive on time: ‘always, sometimes, never’.

Page | 63

Services are reasonably priced for what is provided.

Signs of an ineffective or poor serviceAn ineffective service has many of the following features.

Poor organisational processes

Staff are not matched to the gender, culture and age preferences of people wanting support.

Staff are unreliable and inconsistent. They don’t turn up on time or at all and the service does not respond in a timely way.

Support staff do not seem to be respected by senior staff.

Senior staff are unavailable to speak with. They are not willing to give of their time or seem to consider themselves as ‘too important’ for this.

The office and staff facilities dominate the space. Areas of service delivery are not warm and welcoming or do not appear to be as important as office areas.

People are restricted; there are ‘out of bounds’ and ‘staff only’ areas – even in people’s homes.

The environment looks unsafe, neglected or has concerning elements.

Organisational or group-based practices dominate. Signs include: lots of organisational rules, rules are not flexible or personalised, and there are signs of set routines for food, sleep, medication, areas are too neat and quiet.

Staff are under- or not skilled, trained and /or supervised.

Poor team work

There is a high staff turnover, therefore staff don’t know individuals. There’s no rapport.

Senior staff are not involved with service delivery.

Practice quality is not considered the business of the CEO and Board.

Communication books/case management files between family and service are not kept up to date or contain information that is too general rather than specific to the individual.

Poor staff supportThe following are things to look for which demonstrate that support staff are not doing what they should be doing.

Page | 64

Individuals are either doing nothing or always doing the same things. Signs include a person is still in the same chair an hour later, or wearing the same clothes a week later.

The staff do for rather than with individuals. Signs include: staff moving someone in a wheelchair without asking them or letting them know, staff turning the heating up if they’re cold without consultation, staff washing people’s dishes and cooking people’s meals.

Individuals are looking uncomfortable, stressed or distressed.

Individuals and support staff express dissatisfaction.

The organisation emphasises behavioural restrictive practices and relies on medication.

People’s hygiene is lacking.

Individuals aren’t dressed properly for the weather or occasion or for their personality.

People are grouped together. People with challenging behaviours are all grouped together in one room or one house. Large numbers of people live in the one house or meet up in one setting. People are taken on group bus outings.

People are isolated from the community. There are no, few or poor activities available to enable people to have meaningful interaction with community members.

People are unable to leave their home. Reasons offered could include transport, group demands, too hard for staff or not enough staff.

Poor staff interactionsHow staff relate to people is fundamental to a service being effective or not. This can manifest in many ways. Poor staff interactions can be a reflection of a singular staff member, or of poor organisational culture and practice.

Signs that staff support, and potentially the organisation, are not effective include:

Interactions between staff and individuals are disrespectful. Body language is aggressive or unresponsive.

Staff speak about individuals rather than to them.

Staff are controlling, they insist and direct rather than speak with and consult.

Staff don’t respect the confidentiality of individuals.

Page | 65

Staff openly complain about the organisation – about other staff, senior staff, individuals.

Staff are defensive or fearful. They don’t want to answer questions.

Staff take individuals for granted and ‘brush them off’.

When out in the community, staff talk to community members rather than introducing the individual and facilitating meaningful interaction.

There is evidence of behavioural incidents, neglect or abuse which is not being responded to.

The atmosphere ‘feels’ rigid, stressful, fearful, uneasy, unwelcoming, or boring.

Page | 66