© Nuffield Trust Predictive Models for Health and Social Care: A Feasibility Study Authors:...

© Nuffield Trust

Predictive Models for Health and Social Care: A Feasibility Study

Authors: Bardsley M, Billings J, Dixon J, Georghiou T, Lewis GH, Steventon A (2011) ‘Predicting who will use intensive social care: case finding tools based on linked health and social care data’, Age and Ageing 40(2): 265-270

February 2011

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Background

Key points

Evidence that admission to a nursing home can be delayed or avoided by means of preventative ‘upstream’ interventions

Increasingly, public services will have to become more proactive in identifying and managing risk in older people, in order to mitigate them as much as possible

It will be important to identify and support not just those at highest risk of these costs, but also those currently at lower risk who might become higher risk in future

Current methods of assessing risk largely rely on face to face assessments.  In health care this approach has been shown to be less accurate at predicting predicting future events (hospitalisation) compared to statistical models.

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Predictive modelling

• BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher-quality healthcare than the NHS at a lower cost. Kaiser identify high risk people in their population and manage them intensively to avoid admissions

• Modelling aims to identify people at risk of high costs in future

• Use pseudonymous, person-level data

• In health sector a number of predictive models are available e.g. PARR++ and the combined model.

*Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Predictive Models Identify who will be where on next year’s Kaiser Pyramid

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Patterns in routine data to identify high-risk people next year

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Patterns in routine data to identify high-risk people

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Patterns in routine data to identify high-risk people

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Distribution of Future Utilisation is Exponential

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Can we predict costly social care events in the same way?

• Do the data exist in local systems?

• Can we extract individual, person-level data?

• Can we link a wide range of health and social care data at the person level?

• Are the data accurate and complete enough to use?

• Can we build a valid statistical model?

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Feasibility Project

• Funded by Department of Health Care Services Efficiency Delivery (CSED) programme over 18 months

• Worked with 5 sites to extract data sets

• Extracted person-level health & social care data sets

• Linked data from GP records (2 sites); GP register (all sites); Hospital (all sites); Social Care (all sites)

• Undertook exploratory analyses and ran a range of models

• Tested the impacts of different data sets on the models

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Using health and social care data to predict health and social care use

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Using routine data

• Less labour intensive so they can stratify the population systematically and repeatedly

• Avoid “non-response bias”

• Can identify people with lower, emerging, risk

• Important issues of confidentiality and consent to consider

• Linking data sources at individual level across health and social care is problematic where there is no NHS number in social care

• The tools are never 100% accurate

• Data may be missing from routine databases on certain groups

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Predictive factors – examples

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Information flows

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Data collected

• From four ‘sites’ (~ PCT areas)• Total seven organisations: 3 PCTs, 1 Care trust, 3 LAs • Total 1.4M pop (range 100,000-700,000)

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Data linkage - approach

First instance: NHS number (encrypted) from LA

In absence of NHS number:• Central ‘batch tracing’?• Shared PCT/LA databases?

Ultimately: •construction of ‘alternative IDs’

97% of individuals in one site (population ~400,000) were found to have unique ‘alternative ID’.

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Data linkage

Groups of people in social care data – how many are we able to match to GP register list (of ages 75+)?

Varies, but better for those with > service use

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Data linkage Social & secondary care interface

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Predictive value, sensitivity and specificity of the model incorporating a £5,000 threshold

Predicting social care costs: feasibility study

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Information on Social Care Needs

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Simplifying and sorting

Accommodation Adaptation Adult Placement Advice Given Carer Support Counselling DA1 Day Care Direct Payments Domiciliary Employment/Training Equipment Fostering Group Activity Holiday Payment Housing Advice Given ILS Independent Living Individual Informal Projects Information & Advice Lifeline 400 Lifeline 400 + Smoke Alarm Mobility Training (DAY) Mobility Training (NIGHT) Nursing Nursing (Block Contract) Nursing Block Beds Old Codes OT Advice Given OT Rehab Sessions Other Other Carer Support Services Outreach Permanent Placements with Parents Professional Support R.O. Allowance Referral to Carers Contact Line Referral to Carers Information Service Referral to Crossroads Registration Rehabilitation Residential Residential Block Beds Respite Respite (Carer) Secure Accommodation Sheltered Social Development Special Sheltered Specialist Assessment and/or Treatment Specialist Communication Equipment Specific Approval Support Supported Living Talking Book Machine Time Allocation Vouchers Training Transport Washer

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Individual histories

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Transitions in care (75+ in one site)

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Individual health and social care event timeline over a three-year period

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Predictive Modelling

Attempting to predict:

• For over 75s

a) Admission to care home (or receipt of high intensity home care) or £5,000 increase in social care costs in one year

b) Where person had no ‘significant’ costs in prior two years

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Original models predicting change at £5k threshold

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Diagram

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Distillation flask

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Trade off between PPV (blue line) and sensitivity (red line) according to different risk cut-offs

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Which variables are important in pooled £1k model?

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Models using lower £1k thresholds

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Impact of adding new datasets

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Iteration with Sites - Application

• Several sites said they would like to run the Combined Model and the social care model “side-by-side”

• Theographs thought to be very useful

• Differences between sites: some sites preferred small numbers of clients others wanted large numbers for mail-shots

• Regarded concerns over invasion of privacy as “a non-issue if the wording is right”

• Keen to use predictive models for existing or planned re-ablement services and multidisciplinary teams

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Personal observations on social care data

We could access user level social care data but...

• Different systems

• Issues of standardisation /coding etc and the absence of standard, structured coding schemes

• Some very detailed information collected but can be difficult to use in models

• We found it harder to obtain information on descriptions of needs

• Local concerns about data quality

• Note the history of comparative benchmarking

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study © Nuffield Trust

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

Headlines

• We have created linked health/social care data sets in five sites (overall populations about 2 million people aged 75+)

• We modelled intensive social care:

• Models had satisfactory PPV but low sensitivity at a risk score threshold of 50 (although can be traded-off against each other)

• Mainly social care driven

• Though accuracy worse than PARR model, accuracy was comparable with SPARRA and the combined model

• Discussion with sites – they were less concerned with model performance than we were

• We tried dozens of ways to improve the basic model (5K)

• We modelled change in social care costs with lower thresholds: 1K models are much better but usefulness may be diminished

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study

And the data have so much more to offer.....

• Cost modelling

• Iso-resource classification (individual budgets)

• Trajectories and transitions of care

• Evaluation (re-ablement)

• Information for

- Commissioners and managers

- Professionals

- Service users and carers

© Nuffield TrustFrom: Predictive Models for Health and Social Care: A Feasibility Study February 2011

www.nuffieldtrust.org.uk

Sign-up for our newsletterwww.nuffieldtrust.org.uk/newsletter

Follow us on Twitter(http://twitter.com/NuffieldTrust)

© Nuffield Trust