Contents - ukmi.nhs.uk€¦ · Web viewUKMi. Audit standards and toolkit. for measuring quality in NHS Medicines Information Services

Centre:

Medicines Information manager:

Auditor:

Date of audit:

FINAL DRAFT UKMi Audit standards and toolkit July 2015 1

UKMiAudit standards and

toolkitfor measuring quality in NHS Medicines

Information Services

Contents

Contents......................................................................................................................................................2

Service outline.............................................................................................................................................3

Enquiry answering process and patient outcomes......................................................................................4

Risk management and governance.............................................................................................................6

Service management standards..................................................................................................................8

Training and staff development standards................................................................................................11

Publications and proactive information standards.....................................................................................12

Specialist service standards......................................................................................................................14

Actions from previous audit.......................................................................................................................15


Service outlineBackground enquiry answering service statistics

Current yearDate range:

Previous yearDate range:

Year beforeDate range:

n % change

n % change

n

Total number of enquiries

Number of enquiries/hospital bed or FCE

Mean time taken per enquiry (mins)

No. yellow cards submitted

Enquiry type n (%) % change

n (%) % change

Level 1

Level 2

Level 3

Specialist advisory service

Patient helpline

Reasons for significant changes in activity:

Staffing

Staff No. WTE* Other roles~

Pharmacists – band 8



Technicians

Other (state)

*Only for staff with MI in their job description. ~e.g. medicines management, clinical trials, clinical, dispensary

Service profile

Service hours

Service users for enquiries (Trust, primary care etc)

Details of proactive service provision

Specialist trust (e.g. mental health, paediatric) ORSpecialist service (e.g. drugs in pregnancy)?


1. Enquiry answering process and patient outcomes

Standard 1.1: Enquiry answers are of high quality.

Factors to consider during assessment:

Does the sample of enquiries assessed meet minimum standards for overall quality in terms of documentation, analysis, search coverage, answer (see appendix 1)

Do all enquiries sampled meet the minimum standard score of 15/20 (75%)?

Are procedures in place for checking answers?

How is quality in complex enquiries ensured? Are complex enquiries involving interpretation of evidence and subsequent advice (usually involving written answers) second checked for clarity? Are all answers involving calculations checked by an independent individual (e.g.MI or pharmacy staff or enquirer)?

What is the impact of enquiry answering on patient care and outcomes? (see appendix 3) Are there any negative impact scores?

What is the impact of enquiry answering on patient safety? (see appendix 4) Are there any negative impact scores?

How are enquiry answering standards maintained or improved?

Is appropriate supervision in place for trainees?

Is peer review of enquiries undertaken?

Evidence provided by auditee:

Further evidence obtained at audit:

Comments, advice where appropriate:


Comments, advice, commendations:

Action plan and review dates where appropriate:

Standard 1.2: Enquiry answering service meets the needs its users and patients.


Is a patient helpline in place? Are the patient helpline standards used?

Is the UKMi recommended survey tool, number of users surveyed and frequency of survey implemented? (see appendix 2 for details)

What is the mean user survey score achieved for the previous 12 months (or appropriate time period)? Are there any individual scores less than 4?

What percentage of enquiries were answered within agreed timescale over the previous 12 months (part of


national KPI dataset). Minimum standard >95%.

Are other KPIs collected? If so what are they? Who are the results reported to, how often and are there any individual targets set?

Regional centres only: Is there a process for stakeholder input to service. Describe. What reporting mechanisms to stakeholders are in place (attach any relevant).








2. Risk management and governance

Standard 2.1: The service is organised with a view to maximising safety


Are the latest editions of appropriate core resources (as outlined in the latest UKMi resource list) available to MI staff and appropriate resources accessible out of hours? If not, are there justifiable reasons for this?

Are SOPs in place that cover departmental working practices to ensure service safety e.g. second checking policy, and to address business continuity e.g. management in the absence of MI pharmacist?

How are IRMIS reports utilised?

Is there adequate supervision of trainees?

Is there adequate supervision of MI technicians and is there evidence of reaccreditation where applicable?







Standard 2.2: Risks are identified and managed accordingly


Has a service risk assessment been undertaken using UKMi or appropriate local tool?

Is there a risk management plan in place to address identified risks?

How are risk assessment plans monitored, escalated and communicated?

Is there a service business continuity plan?

Are there any environmental issues that have an impact on service delivery?

Are there any personnel issues that have an impact on service delivery?








Standard 2.3: The service contributes to initiatives to improve patient safety


Are Yellow Cards submitted?

Are IRMIS entries made?

Are issues identified through MI enquiries (individual or trends) fed back to the relevant pharmacists/ pharmacy departments and/or the relevant wards/ trust departments or committees in order to address issues and improve practice/safety in the department and/or trust?

Have any initiatives been implemented in response to patient safety issues that have come to light through the enquiry answering service?

Regional: what national patient safety work does the service contribute to?








3. Service management standards

Standard 3.1: Service is accessible and efficient.


How do customers access the service? Is there dedicated access by telephone for internal and external callers, during the advertised opening hours of the service, including answerphone & bleep if no MI staff available to answer the phone. Do MI service hours reflect departmental service hours? What happens in the absence of an MI pharmacist (short and long term)?

How is the service advertised?

Are incoming enquiries handled promptly? Post, telephone, email, web (tested prior to audit where relevant). Is there a generic e-mail address? How often are messages (e-mail & ansaphone) picked up?

Telephone number:

Date Number of rings

Is an MI Pharmacist or MI technician available?

If No, or time >5mins is another pharmacist available?

Answer machine.Bleep number available?

Time to reply to answer machine/ bleep

Y/N Y/N Y/N

Y/N Y/N Y/N

E-mail address:Date sent: Time sent: Date acknowledged: Response time:

Web address:

Date sent: Time sent: Date acknowledged: Response time:

Are acknowledgment procedures and means for negotiating deadlines in place?

Are adequate procedures in place for handling and prioritising workload?

What procedures are in place for records management? Is the latest version of Midatabank in use? Is there suitable back up facilities? Are there appropriate procedures in place to ensure enquiry records meet organisation standards for length of storage? Is MiDatabank accessible in absence of an MI pharmacist or event of MI centre closure for legal purposes?

Are completed enquiries appropriately dealt with? How many are pending in MiDatabank? Is there appropriate follow up with clinical pharmacists etc?

Are completed enquiries available outside MI? Are enquiries available to non-MI staff in the Pharmacy?

Are enquiries shared with other centres?








Standard 3.2: Service is responsive to highlighted problems


How are complaints handled? How is criticism and praise handled? What positive comments have been received? How are concerns dealt with? Is there a procedure for dealing with concerns/complaints?

What action is taken to negative feedback on user survey? (scores of less than 4)

What procedures are in place for recording errors that arise from enquiry answering? Are incidents reported to IRMIS and local incident reporting system? Are negative impact ratings reported to IRMIS and local incident reporting system?







Standard 3.3: Staff are appropriately managed


Is the service pharmacist led?

Are PDPs and objective setting exercises undertaken for all staff?

How is poor performance handled?








Standard 3.4: Development plans are in place for the service


Is a service delivery plan aligned to national and organisational strategic in place?

What long term plans (even if aspirational) are in place for the service?







Standard 3.5: Regional – infrastructure supports local MI centres and contributes to national issues


How are local MI staff supported?

Is there a process for dissemination national issues to local centres and vice versa?

How are stakeholder issues addressed (e.g. commissioning framework, stakeholder engagement, management oversight etc)?

Is every local MI centre audited every 3 years as a minimum?








4. Training and staff development standards

Standard 4.1: Service provides high quality training


Are sufficient facilities available for trainees?

Are appropriate UKMi training tools/support materials used for various staff groups trained? E.g MiCal, workbook?

Are learning outcomes clearly defined for all staff groups trained?

Is feedback from trainees formally collected? How are identified issues acted on?

Is there a record of all staff trained and an assessment of whether the learning outcomes have been achieved?







Standard 4.2: Staff providing the service are adequately trained


Are individual staff training needs identified?

Have all relevant staff attended the UKMi national training course?

Are there any problems accessing appropriate training?

Region: Does the centre provide appropriate support for newly appointed local MI staff?

Region: Does the centre support/facilitate training for local centres at a regional level (e.g. network meetings)









5. Publications and proactive information standards

Standard 5.1: Adequate systems are in place to ensure quality


Are procedures in place to guide production?

Is the ‘Project’ facility in MiDatabank used for publications and other proactive work?

Is all proactive information checked for readability and grammar?

Is all proactive information checked for accuracy?

Is documentation retained identifying authors and checkers?







Standard 5.2: Proactive information meets the needs of the target audience


Has the target audience had the opportunity to comment on whether the publication meets their needs?

Is there a stakeholder group that contributes to content of publication?

Is there a stakeholder group that addresses organisation needs in terms of proactive publications?








Standard 5.3: Regional: Service contributes to national UKMi publications?


Does service influence decisions about national UKMi outputs in line with SPS/national requirements?

Does service contribute to national UKMi outputs?

Does service meet the agreed commitment to national outputs?








6. Specialist service standards

Standard 6.1: High quality specialist service is provided


Is the specialist service adequately resourced in terms of access to specialist texts etc?

Do newly appointed staff have specific training in provision of specialist service?

Does the service have sufficient trained staff to ensure service continuity?

Does the specialist service contribute to national UKMi agenda in terms of outputs?

Is the specialist service proactive in terms of sharing their expertise?

Is there external clinician input/support that can be accessed by the specialist service?








7. Actions from previous audit

Standard 7.1: Service has responded to recommendations from previous audit


Have key recommendations from previous audits been addressed, especially those that directly impact on patient safety?

If not, and the recommendations are still valid, have barriers to addressing these been explored?

Has implementation been partially addressed but new barriers or issues emerged outwith direct MI control?








Appendix 1.

Quality of enquiry answering

Guidance notes for ranking enquiries and Criteria for grading answers to enquiries

*Selecting enquiries for audit from MiDatabank

1. Go into ‘reporter’ function in MiDatabank.2. To obtain the percentage of level 1, 2 and 3 enquiries handled by the centre select a time frame of 1 year (or since MiDatabank installation if less than 1 year).

Click on ‘Show report’. Note percentage of level 1, 2 and 3 enquiries listed in ‘Workload’ page. 3. Run a second report covering the time frame chosen for the audit. Click on ‘Show report’ then click on ‘Enquiries’ tab at the top of the page. NB. It may take some

time to build this report if there are lots of enquiries. If this is the case then it may be worth producing three separate reports by filtering for complexity level.4. Once the ‘Enquiries’ report has been built click on ‘Save’ tab and save as a Word document. The report will be saved as a table in this format. 5. Delete all columns except enquiry number and complexity level. The table can be highlighted and sorted by complexity level using the ‘sort’ function in Word. 6. This document, together with details of the yearly percentages of enquiry complexity level, should be sent to the auditor to select 30 enquiries that are

representative of the enquiries received e.g. if 50% of enquiries are level 1 then 15 level 1 enquiries should be selected for audit.

Sample of enquiries audited = 30. Ensure that proportions of Level 1, 2 and 3 enquiries* are representative of the mix of level 1, 2 and 3 enquiries answered.

Circle appropriate level based on outcome of enquiry assessment in spreadsheet below.

Enquiries audited Documentation Analysis of enquiry Search coverage Answer Mean impact score (outcome)

Mean impact score (safety)

Unsatisfactory <80% <80% <80% <80% Any negative scores

Any negative scores

Satisfactory 80-90% 80-95% 80-95% 80-90%

Commended >90% >95% >95% >90%


http://www.ukmi.nhs.uk/filestore/ukmiacg/Criteriaforgradinganswerstoenquiries.docx

http://www.ukmi.nhs.uk/filestore/ukmiacg/Enquirylevelguidance-quickreference_2.doc


Appendix 2.

User survey guidance

Satisfaction survey sample size.

Number of relevant enquiries answered/month % of enquiries surveyed No. of questionnaires sent

out every month

Number of questionnaires returned/year (assuming 50% return)

0-100 5% 1 - 5 30

100-200 4% 4 - 8 24-48

200-400 3% 6 - 12 36-72

400+ 2% 12 72

Survey results – Provide the average score for the ‘overall rating’ of the service, for surveys conducted in the year prior to the audit. You may wish to provide data from a longer period of time if they better reflect any issues caused by changes in staff/service provision. It is recommended that you collate this score, along with other results obtained from the user survey in a running log, to better highlight any particular strengths or weaknesses of the service. An editable spreadsheet with instructions for use is on the UKMi website. It also generates results in graph form for reporting purposes.


http://www.ukmi.nhs.uk/filestore/ukmiacg/UKMiusersurveyguidanceJan2013.doc

Appendix 3. Medicines Information Impact Rating Scale to assess the impact of enquiry answering on Patient Care and Outcomes

Impact rating of Medicines Information enquiries: guidanceIt is important for all NHS services to assess and improve their impact on patient care, outcomes and safety. It has been shown that Medicines Information (MI) enquiry answering services have a positive impact on patient care and outcomes, and medicines safety, based on the opinions of enquirers and independent healthcare professionals.¹̛ ² Questionnaires to healthcare professional enquirers were used in this research to determine the impact on specific outcome measures, and impact rating scales were used to assess the overall impact of MI advice. These tools provide a useful a means of assessing the impact of MI, and whilst the use of the questionnaires is a comprehensive way of collecting data on impact from enquirers, the impact rating of enquiries is a relatively easy way of incorporating an assessment of impact into more everyday clinical governance practices in MI, such as internal QA and peer review.

Tips for assessing impact of enquiries Usually only potential impact can be assessed (unless the outcome is known) Rating scales can be used to rate impact on a single patient OR a group of patients (where relevant) Initially, consider whether it was possible to have an impact (for example, if an event had already happened and MI advice could not change

anything for the patient, then the impact would be ‘no impact’; the reasons for this should be noted but no further consideration of impact would be required)

Do not spend too much time deliberating over the initial rating (a range of ratings can be noted initially if necessary & the issues that affect impact, which can then form the basis of the discussion)

Several issues often need to be considered to assess impact, which vary for each enquiry, but may include: Quality of advice (recommendations clear, answered question, comprehensive, appropriate, accurate, useful, evidence-based advice, tailored to

enquirer, contributed to enquirers decision-making process) Level of risk associated with the medicine(s) & clinical condition(s) Whether advice was followed by enquirer (&/or patient) Whether a change in therapy was made The extent to which advice contributed to patient care (consider advice in context of overall patient care but including: reassurance; ability to assess

risks & benefits; ability to participate in decisions about care) The extent to which the action taken as a result of advice contributed to the outcome Whether MI identified a problem that the enquirer was unaware of/didn’t ask about

1. Bramley D, Innes A, Duggan C, et al. The impact of Medicines Information enquiry answering on patient care and outcomes. IJPP 2013;21(6):393-404. 2. Innes A, Bramley D, Wills S. The impact of UK Medicines Information services on patient care, outcomes and medicines safety (submitted to EJHP March2014)


Impact Advice from MI resulted in Examples

-2 Adverse Impact

A detrimental effect on patient care or outcome

patient received inappropriate advice that had a negative impact on their health, wellbeing or outcome patient received inappropriate treatment that had a negative impact on their health, wellbeing or outcome major unnecessary inconvenience to patient

-1Some Adverse Impact

A detrimental effect on patient care but no effect

on patient outcome

minor adverse effect on patient care patient received inappropriate advice but this was not likely to have an impact on their health, wellbeing or outcome patient received inappropriate treatment but this was not likely to have an impact on their health, wellbeing or outcome minor unnecessary inconvenience to patient

0 No Impact No effect on patient care or outcome

no impact on patient in any way patient had already received treatment or care and MI advice was sought afterwards so no impact was possible advice given was purely for information only

1Some Positive Impact

Improved patient care but this was unlikely to lead to an improvement in patient

outcome

advice confirmed the correct course of action that enquirer was considering (even if therapy not changed) advice helped in the decision-making process but likelihood of this leading to an improved outcome unknown or unlikely patient received appropriate therapy but likelihood of this leading to an improved outcome unknown or unlikely identified medication enabling suitable alternative to be given (no impact on outcome expected) avoidance of missed dose but low risk of consequences if missed or delayed addressed concerns enabling medication to be given but low risk of consequences if missed or delayed patient reassured but no change to their outcome

2 Positive Impact

Improved patient careand this was likely to lead

to an improvement in patient outcome (no improvement in patient

outcome apparent)

longer term positive outcome expected but too early to judge direct impact advice helped in the decision-making process and this was likely to lead to an improved outcome patient received appropriate therapy likely to lead to an improved outcome patient received an improved standard of care patient received care in-line with current local or national guidance patient received care in-line with current available evidence identified medication enabling suitable alternative to be given (impact on outcome expected) avoidance of missed dose where moderate risk of consequences if missed or delayed

3Very Positive Impact

Improved patient outcomeOR

very positive effect on patient care

advice helped the health care professional in the decision-making process and this led to an improved outcome patient received appropriate therapy and this led to an improved outcome treatment resulted in improvement of symptoms treatment resulted in a cure of condition successful prophylaxis of a condition resolution of an adverse drug reaction / drug interaction longer term very positive outcome expected but too early to judge direct impact identified medication enabling suitable alternative to be given (impact on outcome) avoidance of missed dose where high risk of consequences if missed or delayed

4Extremely Positive Impact

Potentially life saving treatment or avoidance of a life threatening condition


Appendix 4. Medicines Information Impact Rating Scale to assess the impact of enquiry answering on Medicines Safety

Impact Advice from MI resulted in Examples

-2 Adverse Impact

Increased risk or reduced safety where harm was

caused

avoidable negative response occurred e.g. avoidable adverse effect, symptoms from a drug interaction

-1Some Adverse Impact

Increased risk or reduced safety where

NO harm was caused

inappropriate drug/dose/route/frequency was advised (increased the risk) condition not treated appropriately (increased risk) patient not referred to a clinician when it was appropriate to do so (increased risk) an adverse event was more likely to occur (increased risk)

0 No Impact No effect on patient safety or risk

no impact on patient safety in any way patient had already received treatment or care and MI advice was sought afterwards so no impact was possible advice given was purely for information only

1Some Positive Impact

Improved safety where alow risk to patient was

avoided

avoided an unlikely adverse event of minor consequence to the patient (if it had occurred). safety check on drug therapy already prescribed or administered (no or low risk) safety check on drug therapy prior to prescribing or administering (no or low risk) advice confirmed the safety of drug therapy unnecessary drug therapy avoided (low risk) avoidance of potential adverse effects (low risk) avoidance of potential drug interaction (low risk) avoidance of potential risk in pregnancy/ breastfeeding etc or safer drug advised (low risk)

2 Positive Impact

Improved safety where amoderate risk to patient

avoided

avoided an unlikely adverse event of moderate consequence to the patient (if it had occurred). avoided a likely adverse event of minor consequence to the patient (if it had occurred). safety check on drug therapy already prescribed or administered (moderate risk) safety check on drug therapy prior to prescribing or administering (moderate risk) unnecessary drug therapy avoided which is likely to have prevented an adverse drug reaction (moderate risk) avoidance of potential adverse effects of moderate risk avoidance of potential drug interaction of moderate risk avoidance of potential risk in pregnancy or breastfeeding or safer drug advised (moderate risk)

3Very Positive Impact

Improved safety where amajor risk to patient avoided

avoided an unlikely adverse event of major consequence to the patient (if it had occurred) avoided a likely adverse event of moderate to major consequence to patient (if it had occurred) safety check on drug therapy already prescribed or administered (high risk) safety check on drug therapy prior to prescribing or administering (high risk) unnecessary drug therapy avoided which is likely to have prevented an adverse drug reaction (high risk) avoidance of potential adverse effects of high risk avoidance of potential drug interaction of high risk avoidance of potential risk in pregnancy or breastfeeding or safer drug advised (high risk) avoidance of high risk contraindicated drug – when problem not identified by enquirer / actively identified by MI

4 Extremely Positive Impact

Improved safety where anextreme risk to patient

avoided an unlikely adverse event of life-threatening risk to patient (if it had occurred). avoided a likely adverse event of major or life-threatening risk to the patient (if it had occurred).


avoided avoidance of a life threatening condition avoidance of major detrimental effects on organ function avoidance of known teratogenic drug in pregnancy


Appendix 5 Risk assessment and grading – to be updated in line with risk assessment policy


Risk grading:

1 - 4 = Green, acceptable risk, regular review.

5 - 12 = Yellow, manageable risk, action required.

13 -25 = Red, serious risk, immediate action required.

Documents

Contents - ukmi.nhs.uk€¦ · Web viewUKMi. Audit standards and toolkit. for measuring quality in NHS Medicines Information Services