Quality control in clinical laboratories

1 of 27

Notes on Quality control in Clinical laboratories.. By Dr. Ashish V. Jawarkar Contact: [email protected]... Facebook: www.facebook.com/pathologybasics

Web: pathologybasics.wix.com/notes

2 of 27



OVERVIEW

1. Introduction 2. Terms and Definitions

1. Statistical quality control 2. Errors and mistakes 3. Preanalytic, analytic and post analytic stage 4. Precision and accuracy 5. Types of analytic errors

1.Random error 2.Systematic error 3.Total analytical error 4.Allowable total analytical error

6. Internal and external SQC 7. Control materials 8. Calibrators

3. Internal Quality control 1. Normal distribution 2. Calculation of control limits 3. Levy Jennings chart 4. The Westgard rules 5. The average of Normals method 6. Bull’s algorithm 7. Delta check method

4. External Quality control 1. Basics 2. EQAS charts and statistics 3. Precision index and coefficient of variation ratio 4. EQC normal distribution charts 5. Youden plots 6. Yundt chart

5. Quality specifications 6. Criteria for acceptable performance 7. Appendix

3 of 27



Introduction

1. The purpose of a clinical laboratory is to evaluate the patho-physiologic condition of an individual patient to assist with the diagnosis and / or to monitor therapy.

2. To have value for clinical decision making, an individual laboratory test result must have total error small enough to reflect the biological condition being evaluated.

3. Moreover nowadays, the overwhelming majority of laboratory results are being generated by automated analysers.

4. These analysers are developed by integration of technologies; analytical chemistry, computer science and robotics.

5. This has diminished the routine laboratory work significantly; the role of technologists and pathologists has been shifted to ensuring that the results that these machines give are accurate.

6. This is achieved by doing proper maintenance, quality control and calibration and data management.

4 of 27



Terms and definitions

1. Statistical quality control / Statistical process control 1. For centuries, manufacturers have checked the quality of their products to

find out defects. At that time, every product was checked one by one, without exception.

2. But with large scale production, it became impossible to check each and every product manufactured.

3. Modern quality control aims to check the quality of a minimum number of samples from the total production. This procedure is called Statistical quality control.

4. It would also be wiser to define SQC as the process that focuses on revealing any deviation from well defined standards.

2. Errors and mistakes Errors: All “wrong” laboratory measurements due to “non human” actions. They have a statistical significance. Mistakes: All “wrong laboratory measurements due to “human” actions. They have no statistical significance.

3. Pre analytical / analytical and post analytical stage

1. Errors and mistakes can be classified according to the time and stage which they occur in laboratory practice –

Pre analytical stage – encompasses all procedures that occur before the analysis of patients samples on automated analyzers (e.g. Blood drawing, sample transportation, centrifugation, dilution etc.

Analytical stage – Includes analytical methods

5 of 27



Post-analytical stage – refers to all procedures after the analysis of patient’s samples (e.g. Transmission of data from analysers to LIS, validation of results, printing of results, and communication of results to clinicians/patients etc.)

MAJORITY OF PRE AND POST ANALYTICAL OUTLIERS ARE ‘MISTAKES’

MAJORITY OF ANALYTICAL STAGE OUTLIERS ARE ‘ERRORS’

2. Most of the laboratory QC processes are directed towards these analytical stage errors – this is because

a. The analytical errors can be attributed to the laboratory staff b. These errors can be detected by statistical methods c. Statistical limits for analytical errors can be established

3. Examples

Preanalytical stage Analytical stage Post analytical stage 1. Inappropriate specimen (wrong anticoag, wrong tube, insufficient specimen) 2. Improper preservation 3. Inappropriate patient preparation 4. Mistake in patient identification

1. Expired/denatured reagents 2. Expired/denatured control/calibrators 3. Calibration curve time-out elapsed 4. Failure in sampling 5. Failure in reagent aspiration 6. Change in analyser photometric unit/flow cell 7. Analyser failure

1. Wrong matching between lab result and patient 2. Wrong copy from anlyser to laboratory report 3. Delay in delivering result to patient/clinician 4. Loss of results

4. Precision and accuracy

1. Precision 1. Precision means repeatability or reproducibility of test results 2. In other words it is the closeness of agreement between repeated

measurements of the same sample (in a very short time, usually on the same day)

3. PRECISION = xi - x̅ Where xi is a single measurement x̅ Is average of successive measurements

6 of 27



2. Accuracy a. Closeness of agreement between the value obtained by analyser and true

value of the sample 5. Types of analytical errors

1. Random error a. Result of measurement minus the mean that would result from infinite

measurements of the same sample (see precision – infinite is random error)

b. RE = xi - x̅ I , where x̅ i is mean of infinite number of measurements c. Random error affects precision d. RE is always greater than zero e. RE can be decreased by increasing the number of measurements f. It can be attributed to undetermined reasons (inherent error)

2. Systematic error

a. Result of mean that would result from infinite number of measurements minus the true value of the sample

b. SE = x̅ I - µ , where µ is the true value of the sample c. Systematic error cannot be decreased by increasing the number of

measurements. d. It can be attributed to certain reasons and therefore can be eliminated

much easier than random error i.e. it can be zero.

7 of 27



3. Total analytical error

TE = RE + SE a. As we have seen from definition of RE, it can never be zero, hence TE can

never be zero. 4. Total allowable analytical error (aTE)

a. Since TE>0 is unavoidable, TE of every single determination must be lower than a specified limit. This limit is called aTE.

6. Internal and external SQC

a. Random and systematic errors should be detected at an early stage b. Two statistical methods can help in detection of these errors

1. Internal SQC

a. Performed every day by the laboratory personel with control materials b. It detects basically the precision

2. External SQC

a. Performed periodically, usually by a third party b. It checks primarily the accuracy

7. Control materials

a. Control samples are pools of biological fluids b. They contain analytes which are determined by the laboratory in

concentrations which are close to decision limits where medical action is required

c. They are usually prepared by the equipment manufacturer / reagent manufacturers

d. They are available in different levels (concentrations) – these check performance of laboratory methods over their entire range

8 of 27



8. Calibrators

1. Calibration is the process of evaluating and adjusting the precision and

accuracy of measurement equipment. 2. For this purpose, reference standards with known values for selected

points covering the range of interest are measured with the instrument in question. Then a functional relationship is established between the values of the standards and the corresponding measurements.

3. Enlisting exact steps of calibration is beyond the scope of this document.

9 of 27



Internal Quality control 1. Normal Distribution

1. Normal or Gaussian distribution (N) is the basis of SQC theory. Distribution chart is a biaxial diagram (x/y).

2. X-axis represents the values of a variable’s observations and y-axis the frequency of

each value (the number of each value’s appearance). 3. It has a bell-shaped form with its two edges approaching asymptomatically the x-axis. 4. The highest point of normal distribution corresponds to the value with the higher

frequency (mode value). It is always on the top of every distribution curve.

5. Median value (M) is the value which divides the variable’s observations in two equal parts. It represents the “center” of the distribution.

6. Mean value or average value (μ or x is equal to the value which all the observations should have if they were equal. The mean value (µ) or average can be calculated by the next formula:

Where: xi = Single value, Σxi = Sum of values, N = Total number of values

7. In a normal distribution, mean, median and mode coincide.

10 of 27



8. Variance – The length of distribution curve defines the variance of the variable. The most common measure of variance is Standard deviation (SD)(s). Standard deviation can be calculated by the following formula –

The distance between the upper (UL) and lower limit (LL) of a normal distribution is six standard deviations (6s). Since mean value is in the center of normal distribution, the total range of a normal distribution is μ ± 3s (to be more exact, not all, but nearly all (99.73%) of the values lie within 3 standard deviations (3SD) of the mean). 9. Z Score - Mean and SD allow for calculation of distance of each observation from the centre (mean). The distance is called the Z score. It can be calculated by the following formula:

For example we are looking at a distance of value xi = 80 from the mean of the normal distribution N~(100,5) Here Z score = 80-100 = -4 5 10. Every normal distribution can defined as N~(μ, s).

For instance N~(76, 2.3) means a normal distribution with mean value = 76 and standard deviation = 2.3. 11. The distance between upper limit and lower limit of a normal distribution is six standard deviations (6s). Since mean value is in the center of normal distribution, the total range of normal distribution is (µ +/- 3s).

11 of 27



12. The empirical rule of normal distribution µ+/- s Contains 68.26% of observations µ+/- 2s Contains 95.46% of observations µ+/- 3s Contains 99.73% of observations 13. Coefficient of variation (Cv) Standard deviation depicts variation in the same units as the mean. Hence it cannot be used to compare distributions with different mean. In such cases Cv can be used. It is the ratio of standard deviation to mean expressed as a percentage.

12 of 27



2. Calculation of normal limits

1. The automated analyzer is calibrated with reference material (calibrator) and validation of calibration is done with External quality control / Inter Lab comparison.

2. In internal SQC two or more control level samples are assayed every day at least once per day before the patient’s samples. Then laboratory checks if all control values lie within the control limits. If at least one of the two control limits is outside of one of the two control limits, then further actions may be required until random or systematic errors are under control.

3. The laboratory staff collects 20-30 successive measurements from any control level.

4. Standard deviation (s) and mean value (µ) are calculated. Range (µ+/- 3s) is considered as trial limits (any outlier is rejected).

5. This calculated mean value is then taken as true value of daily controls. Their standard deviation is the inherent error of the system.

3. Levy Jennings Chart 1. Rotate the normal distribution curve clock wise 2. Draw seven lines from the points µ+ 3s, µ+ 2s, µ+ s, µ, µ - s, µ- 2s, µ- 3s. 3. Values obtained are plotted in the chart date wise. 4. For every different parameter and different level, different LJ chart is plotted.

Random and systematic error in LJ chart

1. If any of the daily control values exceeds 3SD limits, it is a random error. 2. Detection of systematic error is more complicated. 3. In systematic errors two or more successive control values exceed the control limits

which can respectively be 3SD, 2SD or 1SD.

13 of 27



14 of 27



4. The Westgard Rules 1. Westgard rules are a type of quality criteria used for error detection. 2. They are denoted as AL where A is the number of control values and L is the

control limit.

RULE GRAPH INTERPRETATION 13S

One control value lies over or under 3SD - Random error Patient results should be blocked, and root cause analysis should be done.

22S

Two successive control values lie between 2SD and 3SD - Systematic error Patient results should be blocked, and root cause analysis should be done.

12S

One control values lie between 2SD and 3SD - Random error Patient results need not be blocked, only caution is warranted.

101S

Ten consecutive control values are on the same side of the mean - Systematic error Patient results should be blocked, and root cause analysis should be done.

15 of 27



5. Average of normals method (AON) 1. LJ chart and Westgard methods are based on analysis of control samples, but

even control samples determination has some disadvantages like – a. It is costly b. It is time consuming

2. These disadvantages can be minimized by some other methods – like AON method.

3. AON method is based on the principle that mean value of all normal results fluctuate between well defined limits. LJ and Westgard detect random and systematic errors. AON method detects only systematic error.

4. This method is mostly used for biochemistry analysers. Method 1. The laboratory collects data for an anaylate from a fixed number of healthy

persons. Its mean value and standard deviation is calculated. This value will be used as control value.

2. The standard error of these normal samples run daily is calculated with the following formula.

s = standard deviation, N is number of samples

3. The confidence interval is calculated as follows

4. This confidence interval will be used for definition of control limits of the

method. 5. Every day laboratory calculates the mean value of N normal results, this mean

value is symbolized as AON and is calculated by the following formula –

6. If AON exceeds the control limits, the anaylate’s determination has a systematic

error.

16 of 27



7. In daily practice AON method has its own control chart which detects only systematic errors. In AON chart each dot represents a daily mean value of the same anylate.

6. Bull’s Algorithm

1. The statistical quality control carried out in hematology analyzers has many

important differences from the corresponding techniques in the clinical chemistry analyzers.

2. These differences are due to reasons such as the high stability of cytometry technology, the small biological variation of some hematology parameters, the big reagent vials and the small time lasting of the hematology controls.

3. Because of the above reasons Levey-Jennings charts in hematology analyzers are different from corresponding charts in clinical chemistry. For instance the hematology Levey-Jennings charts have only three lines (upper and lower limits and central line). The reason is that these Levey-Jennings charts are not created statistically from a normal distribution of former quality control data, which is not possible because of the very small variation of hematology quality control values. In hematology analyzers the upper and lower control limits act as the “specification’s limits” in industry quality control.

4. The small biology variation of many hematology parameters made many researchers to established quality control methods based only on patients results. Such suitable parameters are the erythrocyte indexes (MCV, MCHC, MCV) with the smaller biological variation (due not only to biology but mostly to the hematology analyzer’s technology).

5. These attributes of them inspired Brian Bull (an American Hematologist) to establish a new quality control method widely known as “Bull’s algorithm”.

6. Bull’s algorithm (also known as method) detects systematic errors in MCV, MCHC and MCV and consequently in HgB, Hct and RBC. His method is a kind of moving average. Its main idea is to estimate the mean value of the last twenty patients’ values, including in them the mean value of the batch of the previous twenty values.

7. The algorithm itself is a quite complicated equation which eliminates the outliers and estimates the moving average of the last twenty values. Bull’s algorithm has

17 of 27



been proved quite effective in detecting small systematic errors (almost 1%) not only in erythrocyte indexes but also in almost all the hematology parameters. It uses all patients’ data without exception. The last fact made Bull’s algorithm the cheapest quality control method in laboratory medicine.

8. Hematology quality control samples last only 20 – 30 days and are very expensive, when, on the other hand, whole blood samples are stable in the refrigerator for 24 hours.

9. These facts led some researchers to find methods which are based on the repetitive analysis of patient samples. These methods are known as “retained patient specimens”.

10. In 1988 Cembrowski (Canadian clinical chemist) established the most effective “retained patient specimens” method. It was based on the repetitive analysis of the same patient samples between two successive days. His method is known as “m/nlim”

- “Lim” stands for the quality control limit. It is equal to the double of the standard deviation of the repetitive analysis (2 x SD). - “n” stands for the number of patients’ samples which will be analyzed twice. - “m” stands for the portion of “n” number of samples which is permitted to be out of limits (“lim”). Statistical simulations created by Cembrowski proved the effectiveness of his method. According to him the best combination of “m”, “n” and “lim” is 2, 3, 2 or 2/32s. Concluding, three different methods are in the disposal of the laboratory in order to detect the analytical errors in hematology laboratory. Levey-Jennings detects systematic and random errors. On the contrary, Bull’s algorithm and “retained patient specimens” detect only systematic errors, but they have the advantage of the low cost. Laboratory can choose the best combination of the three

7. Delta check method 1. The AON and Bull’s algorithm detect systematic errors using a group of

patients. 2. Delta check method is used for detecting random errors using previous values of

individual patients. Delta check = (current value) - ( previous value) Delta check % = (Current value – Previous value) * 100 Current value

18 of 27



3. The delta check values should vary between two limits which are called ‘Delta check limits’. In order to calculate them we must take into consideration the reasons for delta differences –

a. Intra individual biological variation of the analyate CV1 b. The analytical variation (Smeas) can be easily estimated by the control

values c. The pre-analytical variation (CVpre-analytical)

19 of 27



External Quality Control 1. Basics

1. EQC refers to the process of controlling the accuracy of an analytical method

by interlaboratory comparisons. 2. The EQAS co-coordinator prepares a sample pool and sends to the

participants of the scheme one or two samples from the same pool. 3. The samples are assayed by the laboratories using the same equipment and

reagents as they do in routine for patient determinations. 4. The EQAs coordinator gathers all the results and they group them (peer

groupas) according to laboratory analytical methods, analyzers or any other criteria.

5. Then the coordinator calculates the target value (consensus mean) and its total variation (standard deviation) of the laboratories results.

6. If any of the laboratories have values outside of the control limits, then this laboratory is considered ‘out of control’.

7. The out of control laboratories get an indication that there is some problem with their analysis.

2. Understanding EQAS charts and statistics

a. Standard Deviation Index 1. Standard Deviation Index calculates the distance of laboratory

results from the consensus mean. 2. It quantifies the inaccuracy of the analytical method. 3. It is similar to Z score and calculated by the following formula –

SDI = laboratory result – Mean value of peer group Standard deviation of peer group

4. The SDI value of each laboratory can be located on proper SDI chart as follows:

20 of 27



5. Control limits of SDI are zero+/- 2SDI 6. Four rules are usually employed for SDI evaluation:

a. 2/51SDI – Two from five successive control limits exceed 1SDI, it is a warning rule

b. x̅1.5SDI – The mean value of five SDI values exceeds the limits +/- 1.5SDI. It reveals a lasting systematic error.

c. 13SDI – One value exceeds 3SDI d. R4SDI – The range between the lower and higher SDI values

exceeds +/- 4SDI

b. Precision index (PI) and coefficient of variation ratio (CVR) Precision index = Standard deviation of laboratory Standard deviation of peer group CVR = CV of laboratory/month CV of peer group / month

21 of 27



c. EQC normal distribution charts 1. More often than not, EQAS coordinators represent graphically the

total performance of all the laboratories in a normal distribution chart.

2. This chart is usually a histogram.

3. The EQAS coordinators usually group the laboratories according to

their analytical methods and their automated analyzer. Histograms containing two or more peer groups have bars with two or more different colors respectively.

22 of 27



d. Youden plot 1. Many EQAS schemes use control samples of different levels in

order to check the performance of the analytical methods. 2. Youden plot is a rectangular chart of which the four angles

correspond to the control limits of the two control levels (-4SD and +4SD).

3. The acceptable part is the gray zone and the rejected part has different colors.

4. Each dot represents a different laboratory and therefore Youden plot describes the whole EQAS scheme.

e. Yundt chart

1. Yundt chart helps to illustrate performance of an analytical method

across all its measuring range. 2. It needs atleast three control levels to be plotted. 3. If the line across the dots of three levels is a straight one, then

laboratory has very good linearity.

23 of 27



4. If the line across the dots is not straight, the linearity of the method

has several issues.

24 of 27



Quality Specifications 1. SQCs goal is to detect random and systematic errors or better, to ensure that

total error is lower than total allowable error. 2. aTE depends on certain characteristics of analyate and analytical method like

imprecision, bias, biological variation etc. 3. According to these characteristics some analyates need more or less rigorous

SQC rules than others. There are two common practices: United states: depending of analytical performance of the method. This practice is followed in US. The laboratories follow CLIA regulations. Europe: aTE depends biological variation of the analyate.

25 of 27



Criteria for acceptable performance CV1 - intra individual variation CV2 – inter individual variation CVw-day – within day variation CVb-day – between days variation CV2

total = CV2wday + CV2

bday 1st Criterion of acceptable performance CVtotal </= 0.5 CV1

2nd Criteria for acceptable performance CVwday </= 0.25TE% 3rd criteria for acceptable performance CV total </= 0.33 TE% 4th criteria for acceptable performance Bias% < 0.25 CV2

1 + CV2w

5th criteria for acceptable performance TE% </= k 0.5 CVw + 0.25 CV2

1 + CV2w

For k =1.65 TE% </= 1.65 *RE* SE

26 of 27



APPENDICIES

27 of 27



Health & Medicine

Quality control in clinical laboratories