Embed Size (px)
Citation preview
VOICE
Developing a New Measure of Service User Perceptions of Inpatient Care,
using Participatory Research Methods
Jo Evans Service User Research Enterprise (SURE)
Institute of [email protected]
Service User Research Enterprise
• Undertakes research that tests the effectiveness of services and treatments from the perspective of people with mental health problems
• Employs people with experience of using mental health services
• Involves service users collaboratively in the whole research process: from design to data collection, through to data analysis and dissemination of results
Why Involve Service Users?
• The inclusion of the service user perspective is vital in developing evidence based practice
• Traditional approaches can ignore issues around acceptability of treatment • New policies and practices can be made on the basis of an inadequate evidence base
• Research priorities and perspectives of service users can be different to those of traditional researchers
• Consumers’ Perspectives on ECT (Rose et al, 2003): professional papers reported much higher satisfaction with ECT than user reports.
• Use of a different methodology: participatory research• Involves service users in the whole process and assumes they are experts• Reduces the power relations between researcher and researched• Allows participants to feel more relaxed and give honest and open responses• Develops different outcomes and measures
Outline
• The development of a service user generated measure of perceptions of acute psychiatric care.
• The major issues raised by service users, which were included in the measure
Phases of Measure Development
• Sampling and Recruitment
• Developing a Topic Guide
• Data Collection
• Analysis and Measure Development
• Psychometric Testing
Sample and Recruitment
Purposive Sample: Service users admitted to hospital in the previous 2 years. Qualitative Phase: Local MIND day centre and community mental health teamsQuantitative Phase: Acute and forensic wards and intensive care units.
Demographics:
Qualitative Phase (n=37) Quantitative Phase (n=185)
Gender 43% male57% female
62% male38% female.
Age 20-66 yearsMedian 45 years
19-76Median 37 years
Ethnicity 49% White51% Black and Minority Ethnic Groups
51% White48% Black and Minority Ethnic Groups1% Not Known
Diagnosis 51% Schizophrenia/Schizo-Affective Disorder49% Other
41% Schizophrenia/Schizo-Affective Disorder59% Other
Developing a Topic Guide
Literature Review• Comprehensive literature search produced suggested themes
Reference Group• Service users and representatives from mental health organisations discussed
and expanded on the draft topic guide
Pilot Study• Tested the topic guide and focus group process• The session was audio-taped, transcribed and the data analysed• The topic guide was amended and used as the basis for discussion in the
focus groups
Data Collection
4 sets of focus groups of service users met to discuss their experiences of acute care
• Held at non-NHS sites to enable fuller disclosure• Facilitated by 2 service user researchers • Each session was audio-taped and transcribed
Each group met twice for the purpose of respondent validation• At the first group, the topic guide was used to prompt discussion• The data was subject to a thematic analysis and the main issues were fed back
to the second group• 90% of participants came back to the second group
Measure Generation
Item Generation• The whole data was subject to a second thematic analysis• 22 items were generated in the form of brief statements, with a 6 point Likert
Scale response format • “I find it easy to keep in contact with family and friends when I’m on the ward.”
Refining the Measure• 2 expert panels and the original reference group were asked for their comments
and the measure was adjusted accordingly.
Strongly Agree
Agree Slightly Agree
Slightly Disagree
Disagree Strongly Disagree
Psychometric Testing
Feasibility Study 1 (n=40)• Completed VOICE and the Service Satisfaction Scale: Residential (Greenfield, Atkinson &
Pascoe, 1985) for criterion validity • 5-15 minutes to complete VOICE, with minimum support required• 97.5 % found it easy to understand and complete
Feasibility Study 2 (n=105)• 82% found it an appropriate length• Only 2% disliked completing the measure and 93.5% did not find the questions distressing• Suitable for a wide range of diagnoses • Flesch-Kincaid grade level of 4.6 (approx age 10)
Test Retest (n=40)• Completed VOICE on 2 occasions to assess the stability of the measure over time
Factor Analysis (n=210)• Will be conducted using the baseline data from the randomised controlled trial
Validity and Reliability
Content and During the focus groups, service users identified a wide range of Face Validity: domains which they considered most important. Expert panels
confirmed that the measure reflected their own experiences and views
Criterion Validity: There is a strong, positive association between VOICE and SSS:RES (Pearson’s correlation coefficient, r = .881)
Internal Reliability: One item was removed from the scale, leaving a Cronbach’s Alpha of .938. This indicates that the items are measuring the same underlying construct
Test Retest Reliability: Lin’s concordance coefficient: total score, rho = 0.906 and mean score, rho = 0.886. This shows the measure has strong test retest reliability
Inter-Rater Reliability: Between 97 and 99% agreement
Main Issues in the Measure
• Admission (1)
• Care and Treatment (2)
• Medication (3)
• Staffing (5)
• Therapy and Activities (3)
• Environment (5)
• Diversity (2)
Admission
Fear and lack of understanding“I was plonked down in a chair and I was left there for nearly an hour, not knowing what was going on and watching the patients, you know, both aggressive and very, very withdrawn. You know, it literally frightened the living daylights out of me because I was there for nearly an hour, without anyone coming up to me.”
“When somebody is brought with a section, it’s like a big war. You’re trying to show them that you’re sane. You’re trying to show them that you’re … capable of doing what you want to do for yourself without hurting anybody. And they’re there trying to tell you … “well boy, we don’t believe and you’ve got to prove it to us by, by succumbing and … giving into whatever we want”. And that’s how it feels. I feel that the system … is making more schizophrenics violent than if they weren’t getting the treatment. I think that the system actually makes a lot of … the violence and the anger and the … sort of war against, from users … or survivors ... against the authorities.”
Qu: I was made to feel welcome when I arrived on this ward
Care, Treatment and Medication
Little involvement in care
“They keep saying that you’ve got the opportunity to but they never actually give, physically give you the opportunity to do anything about your care. They don’t have, it comes back to the time, this has been said. Ward rounds five, ten minutes at the most and then you’re kicked out so you can’t really get into a discussion about your care in that short a period of time.“
“They wouldn’t take the trouble to come and sit, just take one of the staff to sit in your bedroom with you and maybe talk over what’s making you feel like that. Yeah rather than do that, they just sit back and just let you go over the top and then they pound on you with the needle.”
Qu: I have a say in my care and treatment
Qu: Staff give me medication instead of talking to me
Staffing
Interaction and communication
“A lot of the time you had to go to them to say ‘I need to talk’ and they’d say ‘five minutes’ and half an hour later you’re getting worse and worse and worse. You’re spiralling down, you need that person to talk to and they finally decided to turn up.”
“They think because you’re mentally disturbed, you don’t deserve like to be treated like a, you’re treated like a subnormal human being because you’ve got the disturbance.”
“When I was feeling down, he picked me up you know. He told me the good things about myself, the positive things about myself you know.”
Qu: Staff are available to talk to when I need them
Activities and Therapies
Little to do, boredom, need for talking therapies
“I was just absolutely bored out of my nut and found it very difficult to motivate myself to get up, to eat, to do anything just because of the whole boredom issue.”
“Just dumped. Sleep, eat, poo. That’s it.”
“The situation where that girl … was trying to hang herself. If group therapy were able to listen to her and understand her because, you know, everyone’s walking around ‘oh she’s at it again. She’s trying to self harm. Whatever.’ But if they sat her down … in a therapeutic room and listened to her, what got her to that stage, then perhaps she would’ve understood, and then give positive thoughts back to her as to how she can deal with it.”
Qu: I think the activities on the ward meet my needs
Qu: I find sharing experiences in a group with other patients helpful
Environment
Feeling unsafe and unprotected
“I think acute wards are quite stressful places to be. Because like you’ve got people with all different kinds of illnesses. You might have someone with depression in with someone with paranoid schizophrenia or really aggressive and the two don’t match.”
“I was in with some very dangerous people that kept threatening me and making my life a hell and doing things to me and the staff done absolutely nothing to help me, you know. Absolutely nothing.”
“They were really violent. When I mean violent, they are the kind of people that one minute they are ok, everything is ok, then they just flip. Just minor little things.”
Qu: I feel safe on the ward
Diversity
Respect for difference“This particular nurse said, ‘do not trouble me, or any of you. This is Sunday, it’s Bible day, so don’t ask me for anything’. I’ve had another one say to me, ‘you lot are evil because you don’t believe in Jesus’. Now it’s fine for people to have their religion. I’m not against anybody’s religions, but do not use their religions on us because we’re not there for religion. We’re there to get ourselves well”.
“Well I presume most Black Caribbean … they label you don’t they, if they can’t find exactly what it is. Because all I had was a breakdown and now it’s gone to … schizophrenia and paranoia, paranoid schizophrenia. So how did they go from a breakdown to that?”
Qu: I feel able to practice my religion whilst I’m in hospital
In Summary
• VOICE is a brief self-report questionnaire measuring service users’ perceptions of acute care
• It is directly drawn from the experiences of service users and encompasses the issues that they consider most important
• It also has strong psychometric properties
• VOICE is currently being used as the primary service user measure evaluating health and social care interventions in a randomised controlled trial of acute inpatient care
VOICE
Developing a New Measure of Service User Perceptions of Inpatient Care,
using Participatory Research Methods
Jo Evans Service User Research Enterprise (SURE)
Institute of [email protected]
