Embed Size (px)
Citation preview
Using mixed methods to investigate effectiveness and costs of key worker services
for disabled children
Tricia Sloper, Social Policy Research Unit, University of York
Jeni Beecham, Personal Social Services Research Unit, University of Kent
Funders
Treasury Evidence Based Policy Fund
Department of Health
Department for Education and Skills
Welsh Assembly Government
Background: experiences of families with disabled children
Difficulties in finding out about and accessing services
Poor information provision Lack of coordination between services High levels of unmet need
Policy recommendations re key workers since 1976
Evidence on provision of and effectiveness of key workers Effects of key workers:
Better relationships with services Higher morale Less isolation and feelings of burden Improvements in receipt of information, access to
services and fewer unmet needs Partnership and family involvement Job satisfaction for staff
Recent increase in numbers of key worker services
Different models of service being implemented No information on the costs of these supports
Aims of the research
To compare the implementation and operation of different models of key worker services.
To assess the outcomes for parents and children of the provision of different models of key worker services.
To investigate sources of funding and costs of different models of key worker services.
To identify the features of the services that contribute to improved care for disabled children and their families.
To inform standards of good practice in services for disabled children and their families.
Why mixed methods
Information on existing key worker services – e.g. what agencies are involved, what model used, how many key workers, how many families, costs etc – quantitative data – survey.
More detail on different models – case studies of selected services looking at: Experiences of managers and staff – qualitative Outcomes for families – quantitative Experiences of parents and children – qualitative
Features of services related to better outcomes for families – quantitative – path analysis
Features of families and services associated with costs – quantitative – analyses of cost variations
Methods
1. Survey of key worker/care coordination schemes in UK.
2. In seven services: Interviews with staff – key workers, service
managers, steering group members (n=87); teachers and heads of schools (n=15)
Questionnaires completed by parents (n=205) and children (n=30) who received the services
Interviews with parents (n=68) and children (n=9)
How methods related to each other (1)
Survey – basic information on characteristics and costs. Informed selection of case study services
Interviews with staff – detail on the selected services, staff evaluations of important characteristics, advantages and disadvantages of their model. Informed selection of independent variables for
analysis of outcomes, and interpretation of quantitative analysis.
Informed estimation of the total and unit costs, and the analysis of variations in the contact costs.
How methods related to each other (2)
Questionnaires to parents and children – quantitative data on use of the service and outcomes. Path analysis for each outcome measure Calculation of key worker-family contact costs Analysis of the variations in contact costs
Interviews with parents and children – experiences of receiving the service. How do these fit with results of path analysis?
And the cost analysis? Experiences informed our interpretation.
Survey key findings
Differences in: Designated v non-designated key workers v both Management Funding Training for key workers Supervision of key workers Multi-agency involvement Length of time service in operation
Staff interviews
Important issues for staff Advantages and disadvantages of different types
of key workers Training Supervision Management Understanding of the key worker role - job
description Sharing of information and emotional support Funding Time to do the job
The outcomes we measured
Impact of key worker on parents’ quality of life
Parent satisfaction with the service
Parents’ unmet needs
Children’s unmet needs
Path analysis
A series of analyses based on assumptions of potential causal order reflected in the grouping of variables into blocks.
Context: characteristics of services (survey & interviews with staff) and of families (parent
questionnaires)
Mechanism: what key worker does with the family
Outcomes for families (parent questionnaires)
Blocks of variables for path analysis
Block 1: family and
service context
Block 2: service mechanisms
Block 3: process outcome
Family outcomes
Family and child context:-age of child-level of disability -social class Service context:-dedicated funding-length of time in operation-regular training, supervision and peer support-service manager -clear key worker job description-type of key worker-parent representation on steering group
Aspects of key worker role
Key worker contacts with family:
-frequency
-duration
-appropriate
-proactive
Measure of Process of Care scores
Impact on quality of life
Unmet parent needs
Unmet child needs
Satisfaction with key worker service
Path analysis of impact on parental quality of life (QOL)
BLOCK 1
Family and Service Context
BLOCK 2
Service Mechanisms
BLOCK 3
Service ProcessOutcome
Family Outcomes
Shorter length of time service in operation
Regular trainingsupervision andpeer support
Service managerand clear jobdescription
High aspects ofkey working
score
(0.37)
(0.36)
(0.55)
Positive impacton QOL
(0.18)
= paths from variables with no significant bivariate association with outcome measure(0.55) = beta coefficient values
Path analysis of satisfaction with key worker service
BLOCK 1
Family and Service Context
BLOCK 3
Service ProcessOutcome
Family Outcomes
Regular trainingsupervision and
peer support
Shorter length of time service
in operation
Dedicatedfunding
BLOCK 2
Service Mechanisms
Appropriate levelof contact with
key worker
High aspects ofkey working
scores
Longer telephonecontacts withkey workers
Higher child disability score
Higher satisfactionwith key worker
service
(0.36)
(0.36)
(0.37)
(0.23)
(0.20)
(0.13)(0.13)
(0.65)
(0.14)
= paths from variables with no significant bivariate association with outcome measure(0.50) = beta coefficient values
Path analysis of parent unmet need scores
BLOCK 1
Family and Service Context
BLOCK 3
ProcessOutcome
Family Outcomes
Olderchild
Longer length oftime servicein operation
No regular trainingsupervision,peer support
BLOCK 2
Service Mechanisms
Low aspects of key working
score
Not enoughcontact withkey worker
Lower childdisability
score
High parentunmet need
(0.34)
(0.36)Low measureof process ofcare scores
(0.43)
(0.17)
(0.19)
(0.51)
(0.42)
(0.20)
= paths from variables with no significant bivariate association with outcome measure(0.55) = beta coefficient values
Cost estimation in the seven sites
• Managers’ interview• Survey• Publicly available costs data
Total annual costs for each service
Contact cost: face-to-face visits and telephone (n=159)
• Family interviews: frequency and duration of key worker contact
Unit cost per ‘average’ worker hour: £24 - £42
• Staff interviews: Use of their time, 25-50% on face-to-face contact
Costs of providing key worker support
Contact costs over three months
13 families had no contact 38 families: between £1 and £49.99 36 families: between £50 and £99.99 36 families: between £100 and £199.99 27 families: between £200 and £499.99 7 families: between £500 and £999.99 2 families: greater than £1000.
Mean cost (£150) is a useful figure – but it is logical that if family's characteristics, needs and circumstances are different, so too will be the support cost.
Why contact costs might vary
Contact costs
Child and family characteristics and needs
Parents’ outcomes (n=4)
Parents’ attitudes to the service
Key worker attributes
Key worker activities
Scheme characteristics
Why contact costs vary (bivariate)
Child characteristics and needs Child continence difficulties (+) Child seen hospital doctor in
previous 3 months (+)* Total disability score (+)*
Parent outcomes Total quality of life score (+) Total satisfaction score (+)*
Scheme characteristics Dedicated funding (+)* Parent rep. Steering Group (-)*
* Results remaining valid when adjusted for ‘scheme membership’
Why contact costs vary (bivariate)
Child characteristics and needs Child continence difficulties (+) Child seen hospital doctor in
previous 3 months (+)* Total disability score (+)*
Parent outcomes Total quality of life score (+) Total satisfaction score (+)*
Scheme characteristics Dedicated funding (+)* Parent rep. Steering Group (-)*
Key-worker related Total number of role
‘aspects’ KW provides (+)*
KW makes contact with parent (+)*
Total MPOC score (+)
KW never positively affects way treated by services (-)*
Parent very satisfied with KW (+)
Parent wants to see KW more (-)*
* Results remaining valid when adjusted for ‘scheme membership’
Why contact costs vary (multi-variate)
Equation I Child’s total disability
score (+) Number of role ‘aspects’
KW provides (+) Parent representation
on Steering Group (-)
Why contact costs vary (multi-variate)
Equation I Child’s total disability
score (+) Number of role ‘aspects’
KW provides (+) Parent representation
on Steering Group (-)
Equation II Child’s total disability
score (+) Number of role ‘aspects’
KW provides (+)
Site E (+)
• Low Adjusted R2: c18% cost variation explained.• Only explored the cost variation in one of the services used.
On average families saw 4 types health and social care services (range 0-9) and 0.8 (range0-3) types of school- or education-based services in the 3 months prior to interview.
Path Analysis of Satisfaction with Key Worker Service - using costs?
BLOCK 1 BLOCK 2
BLOCK 3 3 Family and Service Service Process Family Service Context Mechanisms Outcomes Outcomes
(0.36) (0.13)
(0.65)
(0.23) (0.14)
(0.20)
(0.37)
Regular training, supervision and
peer support
Shorter length of time service in
operation
Dedicated funding
Higher child disability score
Appropriate level of contact with
key worker
High aspects of key working
scores
Longer telephone contacts with key
worker
High satisfaction with key worker
service
(0.13)
(0.36)
= paths from variables with no significant bivariate association with outcomes
(0.50) = beta coefficient values
Contact cost
Parent interviews
Understanding v confusion about key worker role
Whole family approach
Key worker being accessible and having time Proactive role KW spending time with child KW support in multi-agency meetings The last four have implications not just for
the way KW work, but also for costs/budgets.
Effective key worker services
Key workers carrying out different aspects of the role
A clear job description for key workers Appropriate levels of contact with families Regular training, supervision and support for key
workers in their key worker role Some dedicated funding for the service A service manager Time for key workers to carry out the role There is a direct link between many of these factors and cost but
the link between improving parent outcomes and costs is not straightforward but through the quality of the support provided.
The key worker role
Providing information and advice about services
Emotional support Identifying and addressing needs of all family
members Coordinating care Improving access to services Speaking on behalf of the family when
dealing with services Support in a crisis Providing information about child’s condition
Benefits of mixed methods approach
Triangulation views of different parties support from different types of data
Different methods inform each other – e.g. survey informed questions asked of staff; qualitative data informed quant analysis
Aid to interpretation Fun! Different types of data convince different
groups of people – putting it all together allows us to ‘persuade’ more people
Example Quantitative data show importance of job description,
qualitative data show why:
‘I suppose my main idea is that it’s my responsibility to communicate with all other professionals involved...and you’d like to feel that you’re the first person the family would turn to if they’ve got worries. And I’d also feel a responsibility in getting the problem sorted, even if it wasn’t in my area I would feel that was my job’
‘I wouldn’t be proactive I think is what I’m saying.. in looking for things that there might be difficulties with…I’m not checking up on other professionals and their involvement in families, that’s not my role, I do not see that as my role.’
(two key workers in the same service)
Some services consistently provided the families with leaflets and explained the key worker’s role to them – families understood the role: ‘to ensure that we had every possible service available to us that we were entitled to, to maintain the smooth running of these services once in place and also to provide emotional support if and when required’
In others, parent understanding varied: ‘well I'm confused about that and what it should be really.. and I don't really know I suppose, I don't really know what that role should be’.
Downsides
Time Finding researchers able to work with and
understand all methods Multiple funders!
