B12

CRI and COLA do not endorse, directly or indirectly, the presentations given at this conference or the products or services provided by the exhibiting vendors. Presentations are intended to be free of bias. The use of any particular product is for demonstration purposes only, and does not imply an endorsement of the product by the presenter or the sponsors of the symposium. © 2017 CRI

Basics of Proficiency Testing for Physicians

Verlin Janzen, MD, FAAFP Family Physician & Laboratory Director

John Daly, MD COLA Chief Medical Officer

Karen Dyer, MT(ASCP) DLM Director, CMS Division of Laboratory Services

DESCRIPTION:

In this session, Dr. Janzen and Dr. Daly will focus on the basics of Proficiency Testing (PT). Physicians will learn how to select a PT provider, manage the PT process, and perform the laboratory director responsibilities relating to PT. OBJECTIVES: At the end of the session, participants will be able to:

Assess the value of proficiency testing (PT) as a valuable, practical, and quality enhancing exercise in any laboratory

Participate in PT as required under CLIA ‘88 for all non-waived testing

Summarize the CLIA requirements for the POL as it pertains to PT

Evaluate and interpret PT results and reports; and, when problems occur, determine what actions should be taken to prevent an adverse effect on patient results

Thursday April 6, 2017

Verlin K Janzen, MD, FAAFP Family Physician & Laboratory Director

Hutchinson, KS

John Daly, MD COLA Chief Medical Officer

Karen Dyer, MT (ASCP) DLM Director, CMS Division of Laboratory Services

1

Level: Basic

Intended Audience

Physicians w/little laboratory experience

Laboratorians wanting to be LD of moderately complex laboratory

2

List 3 reasons why PT is a quality enhancing exercise in any laboratory.

Participate in PT as required under CLIA-88 for all non-waived testing.

Evaluate & interpret PT results and reports

When PT failures occur, determine what actions should be taken to prevent an adverse effect on patient results

3

External check that monitors the quality of testing done in your laboratory

CLIA requirement for some moderate & high complexity tests/analytes

4

External check that monitors the quality of testing done in your laboratory

CLIA requirement for some moderate & high complexity tests/analytes

Remember – QC is an internal quality check

5

6

7

8

9

10

11

12

Regulated analyte – analytes for which CLIA

requires proficiency testing

Moderate and high complexity tests only

Does NOT include waived version of test

Non-regulated analyte All analytes not listed as regulated

PT is NOT required, BUT …..

13

ROUTINE CHEMISTRY Alanine Aminotransferase (ALT)

Albumin

Alkaline Phosphatase

Amylase

Aspartate Aminotransferase (AST)

Bilirubin, Total

Blood Gases (pH/pO2 /PCO2 )

Calcium, Total

Chloride

Cholesterol, Total

Cholesterol, HDL

Creatine Kinase

Creatine Kinase, Isoenzyme (CKMB)

Creatinine

Glucose (excluding devices cleared by FDA for home use)

Iron, Total

Lactate Dehydrogenase (LDH)

LDH, Isoenzyme

Magnesium

Potassium

Sodium

Protein, Total

Triglycerides

Urea Nitrogen (BUN)

Uric Acid

14

ENDOCRINOLOGY Cortisol

Free Thyroxine (FT4)

Human Chorionic Gonadotropin (hCG) - (exluding urine qualitative hCG)

T3 Uptake

Triiodothyronine (T3)

Thyroid Stimulating Hormone (TSH)

Thyroxine (T4)

TOXICOLOGY Alcohol (Blood)

Lead (Blood)

Carbamazepine

Digoxin

Ethosuximide

Gentamicin

Lithium

Phenobarbital

Primidone

Procainamide

Quinidine

Theophylline

Vancomycin

Valproic Acid

15

HEMATOLOGY Cell ID/ White Blood Cell Differential

(manual and automated)

Erythrocyte (RBC) Count

Hematocrit (excluding spun

Microhematocrits)

Hemoglobin (excluding single-analyte instruments)

Leukocyte (WBC) Count

Platelet Count

Fibrinogen

Partial Thromboplastin Time

Prothrombin Time

BACTERIOLOGY All Cultures (including growth/no-

growth)

Susceptibility Testing

Gram Stain

Antigen Detection

Bacterial Identification

16

A1c

Lipase (amylase is

regulated)

Ferritin

D-dimer

Retic

ANA

hs-CRP

Troponin

BNP

Etc

17

1. Lab selects PT provider – purchase challenges for

tests done in your laboratory

2. PT provider sends unknown specimens 5 specimens (Event) - 3 times per year

Non-regulated: 2 specimens twice/year

3. You test, send in results

4. They grade and report back to you and CMS /

COLA

18

NO Not required for waived tests

Not required for UNREGULATED analytes

Regulated analytes PT required for these analytes

Listed in CLIA regulation

Also gives acceptable limits for grading

19

For NON-regulated analytes …

20

CLIA requires that the laboratory have a system for verifying the accuracy of its test results at least twice a year

Options Split sample testing: ± 10-15%

Lab Director sets acceptable limits

5 samples twice/year – ≥ 80% correct*

Proficiency testing (2 samples 2 times per year)

§ 493.1709 (b)

21

Must enroll for each regulated analyte may enroll for non-regulated to meet QA requirement

Required only for primary method in use Twice per year must compare results between methods

must be continuously enrolled

may not change programs for one year

test in same manner as patient samples

DO NOT communicate or send to another lab

document handling, processing, testing, reporting

22

Combinations of testing for your specialty

Specimens lyophilized vs liquid specimens

kodachromes vs printed photos for cell ID

Is there a peer group for your instrument

Result turn-around times

Web-based data entry / reporting

Offer off-cycle PT if you fail

CME - physicians, techs

Cost

23

AAB

AAFP

Accutest

API

Calif Thoracic Society

CAP

Excel (CAP)

Idaho

MLE (ACP prg)

New Jersey

Pennsylvania

Puerto Rico

Maryland

New York

Wisconsin

Physician specialty sponsored PT programs

Others commonly used in POLs

24

Put PT shipments on calendar !

Notify mail room or shipping/receiving staff to contact lab IMMED on arrival

Inspect for damage – notify PT provider for

replacements

Document date rec’d – and DUE DATE

Store properly

READ and follow ALL instructions

25

Use routine testing methods

Same staff that tests patients Recommend rotating between staff

No repeat testing to improve results Unless … you have policy to repeat certain

abnormal results – this is okay on PT

Incorporate in routine workflow

Retain leftover specimen

26

You MAY NOT send PT specimens to another laboratory If instrument down – notify PT provider

You MAY NOT share, compare, or discuss PT results with another laboratory (even another location

of your own practice) before the cut-off date for submitting the results to the provider.

You MAY NOT test PT samples for another lab CLIA requires that you report to CMS/AO if asked to do so

27

PT enrollment order form

Each event PT instructions

Worksheets / instrument printouts

Copy of report submission form w/ signed attestation statement

Report & Event summary w/evidence of review

Documentation of investigation & corrective actions for any PT failures

28

Clerical errors Instrument code / method

Correct units

Decimal in right place

Transposed numbers

Putting result under wrong specimen

Have another person check form

Certified mail – return receipt requested?

29

“We the undersigned, recognizing that some special handling may be required due to the nature of proficiency testing materials, have as closely as practical, performed the analyses of these specimens in the same manner as regularly performed on patient samples.”

Signed by Testing personnel

Laboratory director

30

Event 3 times per year, you will receive 5 unknown specimens (aka

challenges) for each regulated analyte

Each 5 specimens = event

Unsatisfactory performance Getting less than 4/5 specimens for an analyte (80%)

correct for an event

Unsuccessful Unsatisfactory performance for any 2 of 3 successive events

for an analyte

31

DEFINITION: Unsatisfactory performance (< 4/5) for any 2 of 3 successive events for an analyte

1st time – usually no sanctions, MUST show remedial

action CMS / COLA will usually contact lab

2nd time – probably have to cease testing Fail any “3 of 5 events” = CEASE testing

Reinstatement – requires 2 successful PT events

Cease testing is for minimum of 6 months (CFR 493.807 (b))

Off-cycle PT can be used for 2nd PT event if needed BUT this won’t count for

ongoing grading – so if fail next event – could be as cease testing again

32

PT G

radin

g

33

P - Passed, not at risk

F - Failed 2/3 - at risk for next 2 events

P - Passed - but at risk for next 2 events

*P - passed, but must pass 2 consec to resume testing

Failed 2 of last 3 AND 3/4 or 3/5 - cease testing

F - Failed & at risk if fail 2 of next 3 events

34

P - Passed, not at risk

F - Failed 2/3 - at risk for next 2 events

P - Passed - but at risk for next 2 events

*P - passed, but must pass 2 consec to resume testing

Failed 2 of last 3 AND 3/4 or 3/5 - cease testing

F - Failed & at risk if fail 2 of next 3 events

Event 2013 A 2013 B 2013 C 2014 A 2014 B 2014 C 2015 A

Grade F P F P F *P P

Test? Yes Yes Yes Yes No No Yes

35

P - Passed, not at risk

F - Failed 2/3 - at risk for next 2 events

P - Passed - but at risk for next 2 events

*P - passed, but must pass 2 consec to resume testing

Failed 2 of last 3 AND 3/4 or 3/5 - cease testing

F - Failed & at risk if fail 2 of next 3 events

Event 2013 A 2013 B 2013 C 2014 A 2014 B 2014 C 2015 A

Grade F F P F *P P P

Test? Yes Yes Yes No No Yes Yes

36

Event 2013 A 2013 B 2013 C 2014 A 2014 B 2014 C 2015 A

Grade F F P F *P P# P

Test? Yes Yes Yes No No Yes Yes

Event 2013 A 2013 B 2013 C 2014 A 2014 B 2014 C 2015 A

Grade F F P F *P F *P

Test? Yes Yes Yes No No No No

# - may test if at least 6 months @ cease testing (CFR 493.807 (b))

37

Pass two CONSECUTIVE events for that analyte 1 or more of the events may be off-cycle PT

At least 6 months @ cease testing

Off-cycle PT Specimens other than usual event specimens

REQUESTED/PURCHASED from PT provider

After pass 2 consecutive – ask CMS/COLA if you can resume testing

38

Individual report Your results, P/F

Report card – last 3 events

Participant summary booklet How all participants did

Educational material

Information on other methods / instruments

39

40

SDI = (Result – Mean) / SD CH-1: SDI = (28 – 34.5) / 2.9 = -2.2 CH-2 SDI = (237 – 211.8) / 10.4 = +2.4 … so essentially SDI is # SD’s from mean

41

[26] = educational challenge

42

43

Includes only REGULATED analytes

44

45

Summary booklet Summary of results of all labs

Grading criteria

Group statistics

Report codes

Other information to assist evaluating results

Educational material

46

47

48

Review all PT reports ASAP If errors – notify PT provider

Develop process & policy Who reviews

Who investigates failures

Steps in evaluation

Report to whom – when

Investigate ALL failed challenges NOT just if you fail event (i.e. < 4/5 correct)

49

Peer group – same instrument Must contain @ least 10 to be valid

Method group – all instruments using same method

All method group – all methods testing analytes

Referee group – subset of ‘expert’ laboratories

50

If you fail -- you MUST determine why, correct prob, document

1. Check for clerical errors

2. Calculations – check for accuracy

3. Specimen - ? Dilution error, near end of life?

4. Review QC for day PT was done – shifts, trends

5. Review maintenance records, calibration dates

6. Review reagent logs – check for expiration dates

7. Look at PT summary information – ? freq failures

Did you use correct specimen for your instrument?

8. Call

PT provider - ? Others having same problem

Manufacturer for help

Document, document, document, document

51

1. Checklist for receipt, processing & testing

2. PT Survey Exception Report

52

Instrument

Quality control

Personnel training

Processes & procedures

If you fail a second time – you may be required to CEASE testing

** ALL MUST BE INVESTIGATED **

53

Corrective Actions Re-train

Obtain technical assistance

New policy/procedure

Re-calibration / maintenance

Contact manufacturer

New instrument

Re-evaluate after corrective action Split sample testing

Request additional PT specimen

54

YES Are there any ungraded challenges? Self evaluate!

Are all results on one side of mean?

Are any SDI’s over 2.0

There is more value to PT reports than P/F

55

56

You are a moderately complex lab and do CBC’s w/manual diff. Your tech identifies a cell on a PT specimen that she believes is a blast, but your CBC testing policy doesn’t allow her to report a blast because you are moderate complexity. Your CBC policy says that slides with abnormal cells are to be sent to the hospital pathologist for review.

Your tech should (multiple choice):

A. Send the specimen to the hospital pathologist

B. Call the PT provider and chew them out for sending a cell she can’t identify

C. Indicate on the PT report – abnormal cell identified, referred for identification

D. Call the laboratory director

57

A. All tests done in moderate or high complexity laboratory

B. All moderate and high complexity tests

C. Only regulated analytes

D. All tests but waived tests

60

Your tech is running a PT specimen through the cell counter. The WBC result is high @ 22.1. She decides to run it again, and gets a 24.1.

1. Is this okay according to CLIA?

2. Which result should she report?

62

A. Proficiency testing is required

B. CLIA doesn’t require any proficiency testing or accuracy check

C. Twice a year, the laboratory must verify the accuracy of these tests

D. Proficiency testing may be done

65

A. Must be reviewed by the laboratory director

B. Should be reviewed by laboratory staff in a timely fashion

C. With all analytes scoring 4/5 or better should simply be filed in the PT notebook for CLIA inspectors if they want to review

D. Are a waste of time for the laboratory director

67

A. Instrument

B. Quality control procedures

C. Personnel training

D. The laboratory director

69

Your are named the new laboratory director after receiving 20-hr COLA certificate and are reviewing your first PT report. You look at the last page of the report under the column “Cumulative Performance” – and all you see is “successful”. You should:

A. Jump for joy, and invite your lab staff to your home for cocktails

B. Turn to the front page, date and initial, and give to your chief tech for filing

C. Look at the pages in the middle with reports on the individual analytes for challenges you may have missed or that were ungraded

D. You should never have looked at the report in the first place

71

Your are named the new lab director and are reviewing your first PT report. You look at the last page of the report under the column “Cumulative Performance” – and you see two analytes (sodium, alkaline phosphatase) that were “unsuccessful”. You should:

A. Go to the outskirts of town and jump off the bridge

B. Resign as laboratory director because the CLIA police will be over any time to shut your lab down

C. Investigate what happened (QC, PM, previous PT, etc) and implement corrective actions based on problems found

D. Immediately stop reporting any sodium and alk phos results and send all to the hospital lab

73