James O. Westgard

Westgard QC, Inc.

Madison, WI

www.westgard.com

1

Six Sigma Tools for Selection/Design of SQC

CLIA QC

Two levels QC per day!

ISO 15189: Assuring the quality of examination procedures

Performance specifications for each procedure used in an examination shall relate to the intended use of that procedure.

The laboratory shall design internal quality control systems that verify the attainment of the intended quality of results.

Intended use, intended quality of results are ISO phrases for quality goals or quality requirements

4

How accomplish this with practical approach/tools?

Apply Six Sigma concepts/principles

Define “tolerance specifications”

Evaluate analytic performance of method

Imprecision

Bias

Select/design SQC procedure

Sigma SQC Selection Tool

Chart of Operating Specifications

Assess comparability of test results

5

What does Six Sigma mean?

-6s -5s -4s -3s -2s -1s 0s 1s 2s 3s 4s 5s 6s

- Tolerance

Specification

+ Tolerance

Specification Target

+6 SDs should fit into spec

-6 SDs should fit into spec

6

How define “tolerance specification”?

Proficiency Testing (PT) or External Quality Assessment (EQA) criteria for acceptable performance

Format of “allowable Total Error”, TEa

US CLIA requirements

Glucose Target Value ± 10%

CAP requirements

HbA1c Target Value ± 7.0%

7

How determine the “sigma-quality” of analytical processes?

Sigma = (TEa – Bias)/SD

TEa, allowable Total Error

Bias, inaccuracy of the method

Initially from comparison of methods exp.

Ongoing from PT/EQA survey or Peer Comparison program

SD or CV, imprecision of the method

Initially from replication experiment

On-going from routine SQC data

All terms in either % or concentration units

8

9

Sigma-metric Calculation

-6s -5s -4s -3s -2s -1s 0s 1s 2s 3s 4s 5s 6s

Bias

-TEa +TEa

True

Value Lower

Tolerance

Specification

Upper

Tolerance

Specification

SD

Sigma = TEa-Bias

SD

Example calculations for HbA1c CAP TEa=7.0%

Sigma = (%TEa - %Bias)/%CV

TEa=7.0%, Bias=2%, CV=1%

Sigma metric is (7-2)/1 or 5 sigma

TEa=7.0%, Bias=1%, CV=1.5

Sigma metric is (7-1)/1.5 or 4 sigma

TEa = 7.0%, Bias = 2%, CV=2%

Sigma metric is (7-2)/2 or 2.5 sigma

10

How use sigma-metrics for selection/design of SQC

CLSI C24A3 QC selection tool

Power curves of different control rules and different numbers of control measurements

Aim for 90% detection of medically important errors, 5% or less false rejections

Charts of Operating Specifications

Prepared for TEa, 90% error detection

Format same as Method Decision Chart

11

CLSI C24A3 Sigma SQC Selection Graph

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0 1.0 2.0 3.0 4.0

1.65 2.65 3.65 4.65 5.65

13s

/2of32s

/R4s

/31s

/6x

0.07 ----- 6 1

13s

/22s

/R4s

/41s

/8x

0.03 ----- 4 2

13s

/22s

/R4s

/41s

0.03 ----- 4 1

12.5s

0.04 ----- 4 1

12.5s

0.03 ----- 2 1

13s

/22s

/R4s

0.01 ----- 2 1

13s

0.00 ----- 2 1

13.5s

0.00 ----- 2 1

Pfr

Ped

N R

Pro

ba

bil

ity f

or

Re

jecti

on

(P

)

Systematic Error (SE, multiples of s)

Sigma Scale 3s 4s 5s

12

Chart of Operating Specifications

13

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

0.0 1.0 2.0 3.0

13s

/2of32s

/R4s

/31s

/6x

0.07 6 1

13s

/22s

/R4s

/41s

0.03 4 1

12.5s

0.04 4 1

13s

0.01 4 1

12.5s

0.03 2 1

13s

/22s

/R4s

0.01 2 1

13s

0.00 2 1

Pfr

N R

3.0

0 sigm

a p

erformance

Ob

serv

ed I

na

ccu

racy

, %

Bia

s

Observed Imprecision, %CV

OPSpecs Chart TEa=7.0% w/0.90Ped

What SQC is required? “Rules of Thumb”

6 sigma process - single rule QC with 3.0s limits and N of 2

5 sigma process - single rule QC with 2.5s limits or multirule QC with Ns of 2-3

4-sigma process – single rule QC with 2.5s limits or multirule QC with Ns of 4-6

3-sigma process – need Ns of 6-8 or even higher, not practical for most laboratories Can’t afford to run enough controls to detect

medically important errors!

14

Practical SQC Design Process

1. Define TEa for test of interest

2. Estimate SD or CV from replication experiment or from routine SQC data

3. Estimate Bias from comparison of methods, PT survey, Peer Comparison

4. Calculate sigma = (%TEa-%Bias)/%CV

5. Use quality-planning tool to translate sigma into SQC (control rules, N)

15

Practical Sigma SQC Planning Tools

Manual application

CLSI C24A3 Sigma SQC Selection graph

Normalized Method Decision and OPSpecs Charts - www.westgard.com

Computer support

Bio-Rad Unity w/Westgard Advisor

Westgard QC EZ Rules 3

Reference: Assuring the Right Quality Right. Westgard QC, 2007.

16

A process problem!

Can only be solved by upper management