Using WHOQOL in NZ mental health and addiction services · AMHOIC workshop, 11 Nov. 2015. Structure of the workshop: •Background to the WHOQOL-BREF (Rex) • Use in mental health

Using WHOQOL in NZ mental health and addiction services

5th Australasian Mental Health and Outcomes Information Conference,

Queenstown, 11 November 2015

AMHOIC workshop, 11 Nov. 2015

Our names

Rex Billington – AUTChris Krägeloh – AUTMelissa Rowthorn – ConnectSarah Andrews – Emerge AatearoaRoss Phillips – PathwaysJillian Pennington – Wild BambooJessica Gu AUT

Your names


Structure of the workshop:

•Background to the WHOQOL-BREF (Rex)• Use in mental health outcomes assessment (Chris)• Do the test and score it (All)• Analyse the results (Rex and Chris)• Implementation strategies in NGOs (Melissa, Sarah & Ross)• Storing and presenting scores (Jillian. Sarah)• Ground rules for use and the future in modifying the tool in mental health recovery organisations (Rex)• Summary and contacts (Chris)

Activities of the NZ WHOQOL Group• Collaboration is among AUT University, Mental Health

Recovery NGOs, some DHBs and Platform Trust including Te Pou, Wild Bamboo and Lantern.

• We provide general support and advice on the use of the tools; provide general guidelines on scoring and interpretation.

• We monitor the use of the WHOQOL tools in NZ to avoid misuse and keep a current data base of users and collaborate on improvisation.


1. The original development of the WHOQOL tools and their purpose:

• WHOQOL = World Health Organisation Quality of Life Scale.

• Health-related QOL assessment.

• Developed in the 1990s cross-culturally in 14 countries. Expanded to over 35 country versions since.

• 2005 estimate of 123 researchers in 67 centres and 39 countries involved in studies using WHOQOL instruments.


Rationale behind WHOQOL development• Need for measurement of health beyond traditional morbidity

and mortality to include impact of disease and impairment on daily activities and behaviour.

• Desire to find out what patients and clients felt about themselves to supplement what experts thought.

• Introduces a humanistic element to health care to balance mechanistic medical approaches and that positively affects clinican-patient relationship.

• Unsatisfactory and culturally biased translations of similar UK and North American measures.


The World Health Organisation has defined health as:

“A state of complete physical, mental and social wellbeing not merely the absence of disease and infirmity”.

Thus the measurement of health and the effects ofhealth care must include not only an indication ofchanges in the frequency and severity of illness butalso an estimation of well being.


Definition of QOL“individuals’ perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns. It is a broad ranging concept affected in a complex way by the person’s physical health, psychological state, personal beliefs, social relationships and their relationship to salient features of their environment” (WHOQOL Group, 1995, p.1405).

People can be satisfied with their QOL despite ill health or disability. And regain QOL after an accident.


• Experts meeting – Geneva 1991 - multifaceted scale agreed.

• Recruited 15 collaborating centres who conducted country-level focus group with consumers, patients and health experts to examine domains and develop facets – ideas returned to Geneva and sorted.

• Next collaborating centres generated 2,500 questions. Translated/back-translated. Again sorted in Geneva.

Developmental process 1991-1996


• 236 items and 29 facets in the first international trial to examine construct validity of domains and facets and to select “best” items.

• 19 centres involved in the field testing of the WHOQOL with 8,294 respondents involved.

• Selection of 24 facets on basis of their importance and 100 items on the basis of item analysis data.

Developmental process 1991-1996


Properties of the WHOQOL• Comprehensive and holistic multidimensional profile.

• Subjective approach: “Have you enough money to meet your needs?”

• Objective approach “How much money do you have?”

• Cross-culturally developed with many languages.

• Standardised 5-point Likert rating scale covering 4 dimensions - intensity (how much), frequency (how often), evaluation (how satisfied), capacity (are you able).


Uses of the WHOQOL• Medical practice

• Medical audit

• Clinical trials

• Research

• Policy making


Versions and features

• WHOQOL-100 = core generic instrument• WHOQOL-BREF = abbreviated 26-item version.

• Additional modules = HIV, OLD, SRPB, DIS

• Some country versions have optional additional national items (including New Zealand with 5).


WHOQOL-100 Domains and FacetsDomain Facets incorporated within domains

• Overall Quality of Life and General Health

1. Physical Health • Energy and fatigue• Pain and discomfort • Sleep and rest

2. Psychological • Bodily image and appearance• Negative feelings• Positive feelings• Self-esteem• Thinking, learning, memory and

concentration

3. Level of Independence • Mobility • Activities of daily living• Dependence on medicinal

substances and medical aids • Work capacity

4 items per facet

4. Social Relations • Personal relationships• Social support• Sexual activity

5. Environment • Financial resources• Physical safety and security• Health and social care: availability

and quality • Home environment • Opportunities for acquiring new

information and skills• Participation in and opportunities for

recreation / leisure• Physical environment (pollution/

noise / traffic/ climate)• Transport

6. Spirituality/Religion/ • Religion / Spirituality / Personal beliefs Personal Beliefs (Single facet)

WHOQOL-100 Domains and Facets

4 items per facet

Physical health• pain and discomfort• dependence on medicinal

substances and medical aids• energy and fatigue • mobility• sleep and rest• activities of daily living• work capacity

Environment

• physical safety and security• physical environment• financial resources• opportunities for acquiring

new information and skills• recreation and leisure

activities• home environment• health and social care• transportation

Psychological health• positive affect• spirituality• thinking, learning, memory

and concentration • bodily image and appearance• self esteem• negative affect

Social relationships

• personal relationships• sexual activity • social support

The WHOQOL-BREF QOL profile:

1 item per facet

Originally studies in New Zealand using the WHOQOL-BREF used the Australian or English versions.

In 2008, Prof. Rex Billington and colleagues founded the New Zealand WHOQOL Group. One of the early goals of this group was to validate the 26 item BREF instrument for use in New Zealand and thus develop a New Zealand version.


Development of the New Zealand version of the WHOQOL-BREFIn 2009 to 2011 NZ National items were developed from focus groups and tested for their suitability.

Data from the general population were obtained using a random sample from the national electoral roll, and analysed using confirmatory factor analysis and Rasch analysis.

During the last few years, researchers at AUT and elsewhere have collected data in a large variety of settings, which have now been pooled to produce national reference values.


Development of New Zealand national Items:

Conducted 13 focus groups with general community members, people with disabilities, and health experts.

WHOQOL team wrote potential new items based on themes that had emerged from those focus groups.

24 new items were sent out with the WHOQOL-BREF to the random sample of 3,000 participants from the national electoral role (n=808).


Items were excluded if they exhibited the following: ceiling or floor effect

low correlation with total QOL and likely parent domain

decreased reliability of the domains

does not discriminate between sick and well

duplicates an existing core item

results in unacceptable CFA fit indices if included


Final selected new national items are:Psychological domain:• To what extent do you feel you have control over your life?

• To what extent are you able to manage personal difficulties?

• To what extent are you feel respected by others?

• How satisfied are you that you are able to meet the expectations placed on you?

Social domain:• To what extent do you have feelings of belonging?


Reliability and validity values:




Administer a questionnaire to an individual

Either self-administered or interviewer-administered

Note: A time frame of two weeks is indicated in the assessment. It asks us to think about our lives over the last two weeks.

Using the WHOQOL-BREF is easy


Let’s fill one in!

•Read the instructions.•Time yourself.•Discuss and compare ratings with a colleague.•What are your thoughts about the questions?


Different measures for different purposes

• By service users/clients about themselves e.g. WHOQOL (subjective).

• By clinician about service user/client. – HONOS (subjective)

• By management to stakeholders about service delivery. e.g. KPIs (social outcome measure - from service user information) EXPLain KPI


Purposes and uses of the WHOQOL-BREF in mental health services assessment

•Leads service users to assess themselves and their progress.

•Facilitates discussion between case manager and service user to review priorities, progress and goals.

•Aids service provider to review the service programmes through user progress.

•Provides feedback to stakeholders.


Service user level

• Identifies strengths• Self-rated measures that give people a voice.

• Measure and track own wellbeing.

• Permits individual goal setting.

• Encouraged self-direction, empowerment, and taking responsibility.


Case manager level

• Helps understand service user concerns and living situation.

• Serves as a basis for longer term planning with the service user.

• Provides feedback and helps track the service user’s progress.

• Complements the social outcome indicators now available.


Service and organisational level

• Builds evidence and contextualises the service.

• Complements good recovery planning practices.

• Service evaluations, particularly new services.

• Assists in reviewing strategic direction for the service provider.•Shareing infromation for learning

• Comparisons are possible because the WHOQOL is a globally recognised and validated outcome tool.



Stakeholder level

• Aggregate data provided to stakeholders in numerical tabular form.

• Additional attribution information also required.

Items for discussion• How do you define success?

• absolute standard: e.g. an individual reaches a certain cut-off score, such as the mean of a comparison group, such as the national reference values provide.

• relative standard: e.g. the score of an individual shows a positive change, regardless of the final level.


Items for discussion

• Attribution: Even if you notice change in scores, how do you know this change was due to the intervention programme or other factors?

•Should qualifying qualitative information accompany qualitative findings?


Scoring the WHOQOL-BREF

• Scoring individual facet scores (31 facets)

• Scoring domains. (4 domains)


NB: The two global items are not included in domain score calculations.

You can include the NZ national items in domain score calculations. It will make the social domain more robust as it will now be four items instead of three.

However, as you present them, please make it clear that your scores contain national items.

Discuss changes and decide what to do. Especially in areas targeted for change last time. Reflect on these changes and possible explanations for them. Consider what has happened in this person’s life over the last 3-6 months. Look at the areas scoring low, particularly those most important to the client. Consider together anything that could be done over the next 3-6 months to improve this area of life. Look at high scores and discuss why they may have gone up.

The service user might want to do this alone.

The service user might want your support.

The service user might want to do this with a family member.

The service user might want to take this to another service to work on.


Service user feedback

• Entry scores recorded for each facet.• Most important facets may be prioritized.• Next administrations depends on service use. (3

or 6 months usual).• Exiting the service administration.

• Compare changes after each administration?. • Compare changes in those items of special

priority to the service user.

domain scor

Value of domain level analysis

Calculate domain scores

Some items are negatively worded, and are thus transformed from negatively framed questions to positively framed questions. This is done mathematically by subtracting the client’s score from 6. Reverse code the three negatively-worded items: Q3, Q4, Q26 now – this is easy to identify on the scoring sheet. For example:

Original response Q3 = 1, thus 6-1 = 5 (recoded response)Original response Q4 = 2, thus 6-2 = 4 (recoded response)Original response Q26 = 4, thus 6-4 = 3 (recoded response)

Examining the response patterns:You might decide to discard assessments from participants with 20% or more missing answers. However, the reasons for missing those questions could be interesting information by itself (if you are able to access that information).

As you record the data on a spreadsheet, watch out for response sets. Very obvious response patterns are easy to detect, such as “all 1s” or “all 5s”. The negative worded questions (Q3, 4, & 26) allow you to detect those: If someone really has such high QOL with lots of 5s, then the negatively worded questions should be 1s.

Other response patterns are harder to detect and require statistical methods, although even those are limited. A large sample size typically takes care of this problem.


Missing scoresUnder some circumstances (such as when the sample size is very small), you may decide to impute data. In other words, the missing item values are calculated as the rounded average of the other scores on the same domain.

However, you only do that if at least 50% of the items belonging to the domain were answered:

Physical (≥4) Psychological (≥ 7)

Social (≥ 3) Environmental (≥4)


Check domains to ensure there were no mistakes

Ensure domain scores fall within the expected range:

Physical Domain range = 7 to 35 (7 items)

Psychological Domain range = 6 to 30 (6 items)

NZ Psychological Domain range = 10 to 50 (10 items)

Social Domain range = 3 to 15 (3 items)

NZ Social Domain range = 4 to 20 (4 items)

Environment Domain = 8 to 40 (8 items)


Check for response sets

Very obvious response patterns are easy to detect, such as “all 1s” or “all 5s”.

The negative worded questions (Q3, 4, & 26) allow you to detect those: If someone really has such high QOL with lots of 5s, then the negatively worded questions should be 1s.

Other response patterns are harder to detect and require statistical methods, although even those are limited.

A large sample size typically takes care of this problem.


Individualised graphs could show these changes easily

Mary’s WHOQOL-BREF domain scores over the year

0

5

10

15

20

25

Mary's WHOQOL 2010

Jan

April

Aug

Dec

Administration points


Outcome assessment – the bigger picture – organisation /service New Zealand comparisons

Use it to learn and improve! QOL data can be used along withother information so we can dothings like:

• Compare different groups

• Compare interventions

• Identify influences on QOL

• Evaluate programmes

• Measure outcomes5.004.003.002.00

environmental

40

30

20

10

0

Frequ

ency



Estimating outcomes for each facet• Entry scores recorded.• Next administrations depends on service use. (3

or 6 months usual)• Exiting the service administration.

• Compare changes annually for stakeholders. • Item by item mean average changes calculated.• Compare changes in those items of special

priority to the service.


Ordinal versus interval scales

• WHOQOL is an ordinal scale. Measuring change is best using an interval scale.

• Problem of all Likert type scales used in measuring change.

• AUT will produce conversion tables next year. NB: these will be for domain scores only.

Item for discussion

• Response shift: Over time time in service, standards and expectations may change. If a service user’s score has not increased in the expected direction, to what extent could this be a reflection of lack of change or of a shift in expectations that masks the fact that the client’s circumstances have actually improved?


• Representatives of a community of practice supportedby the NZ WHOQOL Group & Platform Trust

• NGO MH and Addiction recovery services• Providing community based support services:

Service delivery is typically - strengths focussed, goal orientated, focussed on life-skill development, encouraging learning & increased insight, gaining new coping skills, community integration and social reconnection drawing on consumer informed and psychosocial models of recovery

Introducing ourselves


Using the WHOQOL-BREF in practice


Two main uses – different yet inter-related

As a practice tool - to increase self determination for clientsAs part of an organisation / service level outcome measurement suite -to increase client feedback about the impact of services on their lives

Early questions, decision making, engagement

•The Rationale – why evaluate subjective HRQOL during service delivery? Where does it sit within an outcome suite?•What is the core ‘fit’ with the service / organisation’s mission, vision, values, service delivery ethic or primary service delivery models?•Warm up to roll out – consultation and collaboration – clients included.


Roll Out Essentials

•Integration into strategic planning and business planning, reviews, reporting and overt demonstration of leadership support•Communication – information for clients / stakeholders•Staff training packages with a practical emphasis •Integration into service delivery frameworks - entry, exit, reviews•Support for implementation – champions, team meetings, coaching•Monitoring uptake and addressing where it is slow / no uptake •Reviews of progress and challenges imbedded in routine meetings•Staged and paced in consideration of the current context •Methods of feedback & follow-up - clients / staff / services / organisation•Resources and IT systems requirements


Implementation Drivers

• Alignment and fit• Implementation planning and

goals • Leadership support• Empowerment [ clients, staff,

services ]• Staff development • Evaluative Learning and CQI

focus• Explorative and Developmental

Evaluation


0

1

2

3

4

5

6

7

8

9

10

Knowledgeabout the

history of OMin NZ

Knowledge of'outcomeinformedpractice'

Importanceof giving /receivingFeedback

aboutoutcomes

Confidencegiving /

receivingfeedbackabout OM

Importanceof

recognisingwho is

obtainingrecovery

outcome andwho is not

Perceivedvalue of OMto your role

Perceivedvalue of

measuringoutcomes to

clients

Average ratings for each question where 1= low knowledge / perceived

value and 10= high knowledge / perceived

value

Focus points of the 7 questions asked about in the pre-training survey

Staff self-assessed areas of need prior to the development of an outcome informed practice training

Staff training needs – some qualitative comments:• “I would like to be confident that the measurement tool we use is the best available and adds

real value to our practice and relationships.”• “People would need to know it is working. Peer Support needs to know that Peer Support is

working”• “More about historical measures to present outcome measures.”• “Would like to be able to articulate more clearly to team members, the relevance of client

outcomes to service delivery to support work practices.”• “We need to have the list of DHB outcomes sought …do not know what the outcomes are that

are expected of us.”• “Being able to use the information effectively on client’s recovery journey. To be able to put it

out in a positive manner so clients can be able to contribute well towards their goal.“• “Cultural diversity and measurement verses mainstream outcome measurement to identify

needs assessment and requirements”• “Evidence based practice. More information on the relevance to us in the context of the

changing face of mental health support work.”• “Discussion on graphs and simplicity for peers to understand and PSW to explain to peers”



As a practice tool – staff & client observations The meaning assigned by clients to ratings

• Insight into their world• Recognising strengths & resources • Establishing client’s self determined priorities for change• Enabling feedback about interventions on HRQOL

Translating this into personal planning• What's going on for me now?• What do I want the future to look like?• What have I learnt? Stimulating a growth mindset.

Context of a trusting relationship ++• Enhances relationship but trust a pre requisite• Can be misperceived as “a test” / service outcome / staff performance measure• Engagement and responses can depend on person’s state / situation


Individual client results


“Certainly evidence of people using QOL as conversation starter…or opening a wondering about what’s happened. I’m surprised about how useful that has been in helping people focus conversations it provides a neat little structure with a number of domains. (Emerge Aotearoa Practice Leader, 2013)

“It touches on areas we don’t usually go into with clients so we can gain some more insignts into where clients are at.” “It is good for gaining insights and self awareness in clients” ( CSR staff 2015)

“Continuous use of the tool provides opportunities to compare previous results and shows whats working and what requires improvement … improves support to an individual we work with because it enables them to identify their choices needs and thoughts” (CSR Team Leader 2015)

Client quotes:

“Its good, it’s visual and its something you have done your self. It’s not like somebody had interpreted it for you. You have had to do it yourself, so you can’t say somebody else thought that, you must have thought that at the time. It's a really good thing.”

“I was worried some answers may show a set back…it’s hard to be completely honest…I felt I needed to keep something back”

“I’m honest with him so he can help me” “Reminded me of therapy”

“The timing aspect is interesting; if you get me on a good day then its good, if you got me on a bad day then it wouldn’t be so good”


In Organisations as a CQI / Outcome Measure• Service level

– Expected and identified patterns of change– Identifying at risk individuals / those not thriving in the BAU context– Used alongside other evidence sources– Direct / indirect impacts on HRQOL can be explored– Outcomes of service level interventions can be explored

• Organization level– Trends over time for service population / subgroups– Responses to systemic intervention and changes– Commonalities and differences between services – Minority groups and subpopulation CQI evaluation– Health promotion role

• Sector learning [ future potential]


In Organisations – Domain Level Analysis

Richmond: Mean domain ratings (initial cohort)

57% 55% 55%62%59% 58% 57%

63%59% 57% 59% 63%58% 58% 58% 63%

0%10%20%30%40%50%60%70%80%90%

100%

Physical domain Psychological domain Social/ Relationshipsdomain

Environmental domain

Mea

n %

rat

ing

s

mean baseline mean 3 month mean 6 month mean 9 month


In Organisations – Facet Level Analysis

0

1

2

3

4

5

6

-1000 -500 0 500 1000 1500 2000

Self ratings on WHOQOL BREF

From 1-5

Days within CSR

Changes in facet 19 ratings among a cohort of complete’s [entry, review exit] clients at CSR ‘Satisfaction with self’ on the

WHOQOL BREF over time

The Attribution DilemmaQualitative Data collection: an unresolved challenge:

Exploring ways to create further meaning from the quantitative results that remain aligned to the self determination objectives of the tool’s use and purpose in the outcome suite


Implementation challenges:

Technical Problems- Difficulty accessing data- Expense of collecting information- Difficulty providing analysis and feedback in a timely manner- Dislike of paperwork

Measurement problems- Lack of adequate measures- Difficulty obtaining follow-up information

Organisational Issues- Preference for operating subjectively or on the basis of authority- Fear of control by management- Fear of empowerment of workers


(source: Binner & Topolski, 1994)

Current Areas of Focus

• Data collection rates

• Data analysis – service level reports – collaborative NGO work with Wild Bamboo

• Feedback processes and use of the feedback to inform individual service planning

• Work in progress – using the tool to its full potential as a CQI and service / organisational outcome measure





“Recovery is happening when people can live well in the presence or absence of mental health problems”

Scottish Recovery Network


Computer storage and analyses

• Graphic outputs:For individual service users.For management and stakeholders.

• Possibility of presenting the BREF and collecting responses on line?

Future

Continual improvement of the BREF:• Additional mental health recovery items. Melissa

Rowthorn.• Possibly discarding some items.• Updating reference values periodically.• Developing change score tables. (Ordinal to interval

scaling problem)• Item banking. (for specificity and flexibility)• Attribution studies.• Collaboration with wild bamboo

Additional Versions• Does your client think this measure is too brief - do you

want more detail to work on? The WHOQOL-100 is available.

• Is your client particularly interested in spirituality, making sense of their recovery in terms of personal life beliefs and values? Consider the additional Spirituality module WHOQOL-SRPB.

• Your client’s first language is not English – contact us to see if we can source a version in your client’s first language.


Items for discussion

• What suggestions do you have for future developments.


Purpose of this workshop1. To give you very brief background of the original

development of the WHOQOL tools and their purpose.2. To complete the questionnaire and be able to score it.3. To show the uses of the NZ WHOQOL-BREF in mental

health recovery programmes in NZ.4. To show you how the NZ WHOQOL scores can be

recorded and analysed for different uses.5. To discuss how the WHOQOL is introduced into service

use.6. Relevancy and future modifications for use in mental

health outcomes assessment.


The NZ Mental Health Recovery WHOQOL Group who meet bi-annually.

Ross Philips - PathwaysPhillipa Gaines – Lattice/PlatformSarah Andrews – Emerge AotearoaMelissa Rowthorn -ConnectPatricia Hsu – WCDHBRoss Phillips – PathwaysJames Gordon – Wild BambooMark Smith – Te PouAnd 20 Others.

How to contact us

Adjunct Professor Rex BillingtonTel.: 09 921 9999 extension 7586email: [email protected]

Associate Professor Chris KrägelohTel.: 09 921 9999 extension 7103email: [email protected]

• We are based on AUT University’s North Shore (Akoranga) Campus.

How to contact us

website

• For people who are registering for the first time, they can do that directly from the WHOQOL webpages login page: https://pcrc.aut.ac.nz/new-zealand-whoqol/login. The registration request will come through to Chris K. or Rex B. for the time being. They will agree the request. Then there will be a couple of simple steps to follow which you will receive through the e-mail.

https://pcrc.aut.ac.nz/new-zealand-whoqol/login

Thank you!

Documents

Using WHOQOL in NZ mental health and addiction services · AMHOIC workshop, 11 Nov. 2015. Structure of the workshop: •Background to the WHOQOL-BREF (Rex) • Use in mental health