Performance management in the NHS:

going beyond the metrics

Dr. Mark ExworthyRoyal Holloway-University of London

M.Exworthy@rhul.ac.uk

NHS Confederation / LSE Health seminar, 27 April 2010

Performance: opening comments

• Performance has been a dominant narrative within English health policy reform in the last decade

• Focus is shifting over time* • Performance management shortcomings*• Research tends to…

– Focus on technical `performance products’– Lack conceptual development

• Wider perspective on performance required…– To go beyond the metrics

Feature Traditionally Increasingly

Unit of analysis Organisational level Individual level

Specificity Anonymous Named

Motivation Intrinsic Extrinsic

Focus Inputs & outputs Outcomes

Purpose Developmental / formative

Judgemental / summative

Reference group

Peer reviewed External

Performance management: a shifting focus over time

Exworthy et al, 2010

Performance: opening comments

• Performance has been a dominant narrative within English health policy reform in the last decade

• Focus is shifting over time* • Performance management shortcomings*• Research tends to…

– Focus on `performance products’– Lack conceptual foundations

• Wider perspective on performance required…

Performance management shortcomingsAdapted from Sheaff et al, 2004; Talbot, 2005

1. Incompleteness2. Over-complexity3. High transaction costs4. Attribution difficulties5. Quantity-quality imbalance6. Gaming 7. Short-term focus 8. Performance churn

Performance: opening comments

• Performance has been a dominant narrative within English health policy reform in the last decade

• Focus is shifting over time* • Performance management shortcomings*• Research tends to…

– Focus on `performance products’– Lack conceptual foundations

• Wider perspective on performance required…

Wider perspectives on performance

• The influence of subjective views of performance upon management

• Conceptual perspectives on performance

• Consequence (intended and otherwise) of performance management

1. Formal & informal performance

Formal performance– Hard information ~ official metrics– Quantitative measures; retrospective– Tends to focus on `poor’ performance rather than improving good

performance– Example: rankings, league tables, targets– `Safety net’ function

Informal performance– Soft information ~ perceptions, founded on subjective

judgements– Qualitative measures, can be prospective; reputation, trust,

goodwill, tacit knowledge, credibility – Eg. `safe pair of hands’, `keep an eye on them’, `what is really

happening?’– Substitution & complementary functions

Formal performance• Safety net: formal notions

of performance invariably used as a `safety net’ to address organizations with poor (formal) performance– Incentive? Formal

performance may offer `high performing’ organisations little incentive to improve

– High performance did not ensure freedom from centre

Informal performance• Substitute: informal

performance is often deemed more responsive, timely and useful than formal performance

• Complement: both formal and informal performance were seen as important in assessing organisations

• Formal performance is insufficient to explain existing performance patterns and to promote improvement

• Need to understanding better the interplay between formal and informal performance

Adapted from Goddard, Mannion & Smith, 1999; Exworthy et al, 2010a

Example #1. FTs exercising their autonomy

• FTs have not `performed’ as well as expected– Is autonomy such a panacea after all?

• FTs have technical ability to exercise autonomy • Many FTs lacked the willingness to do so:

– Some de facto autonomy already exists– Greater risk– Uncertain rules of the new game– Legitimacy– Fear of negative impact on local health economy

• Need to explore FT managers’ motivations, attitudes towards the award and use of autonomy

Exworthy et al, 2010a

Example #2. Mid-Staffordshire NHS Foundation Trust

• An FT and a high performing organisation?– “In the four years from 2002 until 2005 (the last year of the star

ratings system which ranked trusts from 0- to 3-star), Stafford had got, respectively, 2, 3, 0 and 1 star. Yet it was encouraged or "invited" to seek FT status ” (Paton, 2010 http://www.publicservice.co.uk/feature_story.asp?id=13870)

• Patients and staff knew about `poor’ performance:– “I remember at the time when our staffing levels were cut and we

were just literally running around. Our ward was known as Beirut from several other wards. I heard it nicknamed that. ITU used to call us Beirut… I remember saying: this will have repercussions, this can’t go on like this. Because relatives were regularly coming up to us and saying: my Mum has been buzzing for this long, there has been a buzzer going there for that long.” (p.197)

Francis Inquiry report, 24 February 2010http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/documents/digitalasset/dh_113447.pdf

Example #3. Performance management in local health economies

Director, PCT:

“We’ve got a wealth of informal knowledge about [Hospital X] because there are a number of people in the team who’ve been there before and the nature of [Hospital X] is it’s been very open to it because they’ve been so desperate for help.

[Hospital Y] is much more of a closed book to us. There are people that have come into different roles in the PCT who may have known something about it but it always feels quite antagonistic that relationship and the GPs are pretty disenfranchised lot as well. So there’s not a lot [of PCT staff] that has a kind of root of understanding and influencing this... isn’t there either.

[Hospital Z] was a fairly sort of collegiate comfortable sort of relationship and there’s a fair amount of sort of, traffic, you know, informal networking stuff through that, for a lot of people in the PCT or- there are in the Exec Team of the legacy PCTs”

http://www.sdo.nihr.ac.uk/projdetails.php?ref=08-1618-125 p.153Exworthy et al, 2010

2. Disclosure of clinical performance

• “The more closely we are watched, the better we behave?’ (Jeremy Bentham)

• “Sunlight is the best of disinfectant; electric light the most efficient policeman” (Brandeis)

Disclosure of clinical performance: unintended consequences

1. Quantification emphasis

2. Short-term objectives dominate

3. Manipulation of performance data and behaviour adjustment

4. “Misleading inferences” could be drawn from “raw performance data.”

5. Organisational inertia

Smith, 1995

Public disclosure of performance data

Problem identification Naming of

individualPublic

sanction Recipient response

Pawson et al, 2005, S1:23

Public disclosure of performance data(with unintended consequences)

Problem identification Naming of

individualPublic

sanction Recipient response

Culprit misidentification

Disseminationdissimulation

Sanction misapplication

Unintended outcome

Pawson et al, 2005, S1:23

Public disclosure of performance data

i. Identification• Initial focus on mortality rates of cardiac surgery• Disclosure supported by profession• 2-3% mortality rate neglects most patients• Attribution issue with 30 day mortality rate• Limited use of comparisons

ii. Naming• Average age of cardiac surgery patient = 68 years (and

rising)• Data accessibility and user literacy• Named consultant `hides’ clinical team

Public disclosure of performance data

iii. Public sanction• Strong normative pressure for clinicians to participate but not

compulsory• 25% cardiac surgeons do not participate

• Little evidence of `choice’ as sanction• Sanction mediated by user proxy (GP)• Episodic nature of care & emotional concerns

iv. Recipient response• Initially educational but increasingly judgemental• Professional ownership and promotion• Surgeons’ sense of autonomy threatened• Danger of gaming but little evidence so far

• Risk aversion: high risk patients avoided? • Some rejection of performance measures

• Junior surgeons less exposed to high risk cases

Disclosure and performance management

• Patients’ experiences:– PROMs– Financial incentive: “Hospital income will increasingly

be linked to patient satisfaction, rising to 10 per cent of their payments over time” (DH, NHS 2010-2015: From good to great. 10 December 2009)

• Danger that performance measures becomes too esoteric– Management remains loosely coupled with clinical

performance– Patients unable to interpret performance data

A final thought on performance

“There is a need to recognise the imperfections and limitations of

[performance] measures, and to use them as a means of supporting politically informed judgements”

Stewart and Walsh, 1994, p.45