Audit and SEA Made Easy GPST Teaching, Beardmore, November 2010 Mairi Jamieson GPST Course Organiser NHS Education for Scotland

Audit and SEA Made EasyAudit and SEA Made Easy

GPST Teaching, Beardmore, November 2010

Mairi JamiesonGPST Course OrganiserNHS Education for ScotlandNHS Education for Scotland

Definitely not rocket science

Aims of workshop

Refresh your knowledge of, or introduce you to, audit and SEA

Define criteria and standards

Increase confidence in performing Audit/SEA in your own practice

Appreciate the use of audit/SEA in quality assurance, also for revalidation, QOF etc.

Signpost to resources for further guidance

What is audit?

What is Audit?What is Audit?What is Audit?What is Audit?

Audit is about taking note of what we do, learning from it and changing if necessary

Audit is the improvement in the quality of care through standard-setting, peer review, implementation of change and re-evaluation

Quite simply, audit is a tool that enables you to monitor and then improve the quality of care you provide to your patients.

Systematic critical analysis of the quality of health care

A range of definitions exist:

The Audit CycleThe Audit CycleThe Audit CycleThe Audit Cycle

Define Criteria & StandardsDefine Criteria & Standardsoror

““What do you think you should be doing”What do you think you should be doing”

Collect the DataCollect the Datai.e. i.e.

The Information onThe Information onwhat you are doingwhat you are doing

Assess PerformanceAssess Performanceagainst criteria & standardsagainst criteria & standards

How are we doing in relation to How are we doing in relation to what we should be doing ? what we should be doing ?

Identify the ChangesIdentify the Changes Required & Required &

Implement Them !!Implement Them !!

Define Criteria & StandardsDefine Criteria & Standardsoror

What do you think you should be doing What do you think you should be doing

Choose the TopicChoose the Topic


Choosing The TopicChoosing The Topic

Eg. Clopidogrel Prescribing


Define Criteria & StandardsDefine Criteria & Standards

Criteria - what you want to measure (yard-stick)

eg. Patients should have clopidogrel prescribed only in accordance with NHSGGC guidelines (ie. aspirin contraindicated or postACS/angiography)

Standard - how well you should be doing

80% of patients should have clopidogrel prescribed in accordance with NHSGGC guidelines


Collect the DataCollect the Data

1. Identify patients on clopidogrel

2. Pharmacist review of notes – identify when started, by whom, indication, whether ever on aspirin +/- PPI

3. Determine whether in accordance with guidelines


Assess PerformanceAssess Performance

Compare our results with the

standard previously set

e.g. 21 of 116 patients on clopidogrel (18%) were prescribed according to NHSGGC guidelines – far below standard of 80%


Agree & Implement Agree & Implement

Changes RequiredChanges Required

1. Explore reasons for inappropriate use

2. Feedback to colleagues, discuss changes and implement them

Eg. Letters to patients/cardiologists, face-to-face medication review, raising

awareness of prescribers


Repeat the Audit!!!

Data Collection 2Data Collection 2

Repeat data collection once changes have had

a chance to take hold


Re-assess PerformanceRe-assess Performance

Compare the results with the

standards previously set and results

of data collection 1

Has the standard now been met?

e.g. Now find that 48 of 90 (53%) of patients on clopidogrel are prescribed within

guidelines

ie. Significant improvement but still below standard


Identify Further Changes RequiredIdentify Further Changes Required

Long term issues:

Determine if further change is required to sustain performance, and decide when

next to audit this topic (annually, every 2 years etc.)

Criteria Standard

Children under 2 years old shouldbe immunized against tetanus

and polio

90% of the registered 2 year oldsimmunized against tetanus and

polio

The notes of those patientssensitive to penicillin should

be clearly marked.

The notes of all (100%) patientssensitive to penicillin clearly marked.

Patients should wait no longerthan 30 minutes in the surgery

before consultation.

75% of patients should wait nolonger than 30 minutes in thesurgery before consultation.

Examples of Criteria & StandardsExamples of Criteria & Standards

CriteriaCriteria

Simple logical statements, used to describe a measurable item of quality health care

ie. What you want to measure

e.g. Patients with asthma should have their inhaler technique assessed at least once every 12 months.

Describes the ideal level of care to be achieved for each criterion

ie. How well you should be doing

e.g. 80% of patients with asthma should have their inhaler technique assessed at least once every 12 months.

StandardStandard

Arriving at StandardsArriving at Standards

Don’t get overly concerned - standard setting is flexible, can be revised upwards or down

Those involved decide on the level of care they find desirable - it is a professional issue.

Guidance can be derived from the literature/textbooks, but ultimately you decide.

Can be based on your own work and observations, varies between practices

Report format for audit

Report Section Guidance

Reason for Audit Explain why the topic was chosen

Potential benefits to patients/staff/practice

Criteria Set Try to limit to 1-3, relevant to topic

Short simple logical statements

Justify with reference to current evidence

Standards Set Agree a measureable standard for each criterion

Set a realistic timescale

Preparation and Planning

Who you discussed the audit with, and who assisted you

How you collected and analysed the data

Report format cont.

Report Section Guidance

Data Collection 1 Present using simple descriptive statistics/tables/graphs

Do not present irrelevant data

Comment on comparison with standard set

Description of Change

Describe changes agreed and implemented

Attach example as evidence if possible

Data Collection 2 Compare with standard and with data collection 1

If standard not reached speculate as to why

Conclusions What have you learned

How do you intend to take forward in future

Group Exercise 1Criteria & Standards

Group Exercise 2Implementing Change

Significant Event Analysis

What is a significant event?

“ Any event thought by anyone in the team to be significant in the care of patients or the conduct of the practice “

(Pringle et al, 1995)

What happened?

Record all of the facts relating to the identified significant event (including any relevant dates, times and people or organisations involved)

Data source: those directly and indirectly involved

Establish a clear and full picture of what happened

Impact or potential impact

Why did it happen?

Establish all of the main and underlying reasons why the event actually occurred.

Eg. A written telephone message about an important meeting was not passed to the practice manager because it had been lost.

But…. why was it lost?

Because it was written on a post-it and left on top of a report, which was subsequently filed away by an unsuspecting member of staff.

ie. Internal communication practices not up to scratch!

What have you learned?

Highlight any learning issues you and/or the practice experience.

For example it may be related to a training need or a lack of knowledge concerned with therapeutics, disease management or administrative procedures.

It could also reflect a learning experience (good or not so good) in dealing with patients, colleagues, or other organisations

Ensure that insight into the event has been established by the practice team or the individuals concerned

What have you changed?

Often a change in some aspect of care is required to improve the provision of care and/or minimise the risk that a similar event will occur.

If so, a description of the change actually implemented should be given rather than a “wish list” of thoughts

What have you changed? (cont.)

Sometimes it is not possible to implement change, either because the likelihood of the event happening again is so rare or because change is outwith the control of the individual/practice.

If this is the case, then reasons should be clearly documented.

Regardless of the type of significant event, change should at least be considered, then either implemented or justifiably ruled out

Important points

Doesn’t have to be an bad event Could explore example of excellent practice

Sharing Team activity

Blame-free Constructive learning not finger-pointing

Look beyond the superficial For underlying/systematic causes

“Nothing changes if nothing changes”

Group Exercise 3Analysis of SEAs

Useful links for further information

Guidance on Audit, RCGP Revalidation Toolkit (p28) http://www.rcgp.org.uk/PDF/

Scot_Complete_Revalidation_Toolkit_(Read_Only).pdf

Ideas for Audit, NES www.clinicalgovernance.scot.nhs.uk/.../ideasforauditandSEA.rtf

SEA – NPSA Guide 2008 http://www.npsa.nhs.uk/nrls/improvingpatientsafety/

primarycare/significant-event-audit/

Documents

Audit and SEA Made Easy GPST Teaching, Beardmore, November 2010 Mairi Jamieson GPST Course Organiser NHS Education for Scotland