ELSEVIER Clinica Chimica Acta 260 (1997) 207-216

ez Assuring quality in laboratory testing at the point

of care

Charles R. Handorf

Methodist Hospitals of Memphis, 1265 Union Avenue, Memphis, Tennessee 38104, USA

Received 25 May 1996; received in revised form 7 September 1996; accepted 8 September 1996

Abstract

The science of laboratory medicine has undergone much change during recent years. Despite more recent emphasis on quality improvement, there has not been sufficient attention paid to effective quality management of new approaches to laboratory testing such as point of care testing. It is important that appropriate resources be allocated to quality management, so that waste is minimized and that resources which are expended may be demonstrated to affect the quality of patient care in a positive way. Older quality manage- ment tools such as process quality control and proficiency testing are vital to the success of point of care testing programs, however, new ways of looking at the use of these tools are required. Newer approaches such as electronic quality control of point of care devices and an expanded role of total quality management strategies will enhance rather than supplant the more traditional quality improvement mechanisms. © 1997 Elsevier Science B.V.

Keywords: Laboratory testing; Point of care; Quality

1. Introduction

Medical laboratory science has made remarkable progress over the past 40 years. These four decades span a progression from largely manual testing of only a few possible analytes to the development of large, complex, multichannel devices capable of performing literally dozens of different analyses on very small specimens at a tremendous rate of speed. Organiza- tionally, the advent of large analytical devices helped to solidify the preem- inence of the central hospital laboratory as the primary testing engine for

0009-8981/97/$17.00 © 1997 Elsevier Science B.V. All rights reserved. PII S0009-898 1 (96)06497-2

208 C.R. Handorf / Clinica Chimica Acta 260 (1997) 207-216

laboratory medicine. Within the last 10 years, however, there have been further technological advances which have led to development of miniatur- ized, easily portable testing devices [1]. For certain common analytes such as glucose, electrolytes, hemoglobin, blood gases, and for certain coagulation tests these devices have carried the ability to test right to the patient bedside. Unfortunately, performance monitoring and quality improvement techniques have not progressed in synchrony with the technological ad- vances which have made point of care testing a possibility. If we are to make further significant strides in delivering timely, cost effective, and high quality care to our patients, it is essential that we critically reevaluate and modernize our approach to quality management in all laboratory testing.

In 1990 Westgard predicted, "Current quality assurance approaches will not be adequate to satisfy the needs for quality in the next decade." [2] Alternative site/point of care laboratory testing provides an ideal study case for the modernization of laboratory quality management strategies precisely because it is so structurally different from the centralized laboratory model. Table 1 offers a contrast between testing in the central laboratory and point of care testing. These striking differences help to point out the weaknesses in trying to apply old quality models to new test systems. As has been previously pointed out, it is because point of care testing impacts so many different constituencies in so many new ways that unique challenges abound [3]. Nurses, clinical physicians, laboratorians, administrators, and instru- ment manufacturers approach point of care testing from very different

Table 1

Characteristics of central laboratory Characteristics of point of care laboratory testing testing

Many tests, few sites, few instruments Large runs, 'factory environment' Few highly trained analysts with

extensive logistical support (procedure manuals, on-site troubleshooting, comptetency evaluation, supervision)

Analysts with restricted tasks in the testing cycle

Longer turn around times Controlled physical environment for

reagents and instruments Complex instrumentation with specially

trained analysts required

Few tests, many sites, many instruments Small runs, 'boutique environment' Many analysts, not highly trained in laboratory procedures, and very possibly without significant timely logistical support

Analysts with broader responsibilities in the testing cycle Shorter turn around times Less controlled physical environment for reagents and instruments Less complex instrumentation with simpler operator training, instrumentation not foolproof

C.R. Handor['/ Clinica Chimica Acta 260 (1997) 207 216 209

points of view; these differences may lead to conflicts which take a great deal of time and thoughtful understanding to defuse.

In his keynote address at the College of American Pathologists Confer- ence XXVIII: Alternate Site Testing, O'Leary outlined five basic elements of performance measurement to which we should refer in our efforts to redefine quality measurement in point of care testing: (1) we must know what is important to measure; (2) we must have good measuring tools and know how to use them; (3) we must have indicators, the actual performance measurements which ideally should tell us if we are getting the results that we want; (4) we must have reference databases so that individually and as a laboratory community we may have quantitative frames of reference in which to work to refine the quality of our services; and (5) to measure well, we must have the will to measure [4]. This all implies a much more broad based approach to quality measurement in laboratory science; quality measurement is not just a rote adherence to quality control schemes that have been developed over time without any demonstrated relationship to good clinical outcomes. Lambird has coined the term 'pseudoquality' to apply to the mechanical devotion to such schemes [5]; in an era of diminishing resources we cannot afford the unthinking luxury of pseudo- quality.

2. What is important to measure?

In point of care testing, then, which quality parameters are important to measure? Before we can answer that question we must take one step back and ask ourselves what we hope to achieve in the performance of a laboratory test. What clinical outcome do we expect as a result of our laboratory testing? We know very little about the actual clinical outcome enhancements provided by point of care testing, and such information may continue to be somewhat hard to come by. One such study by Despotis did demonstrate that near patient coagulation testing in patients undergoing cardiopulmonary bypass could significantly decrease blood bank costs associated with microvascular bleeding and could improve patient care results [6]. Most laboratory tests, however, are merely a transitory event in the continuum of a patient's care, although any given testing event may be absolutely critical to the proper care of a patient. Judging the impact of a laboratory test on the ultimate outcome of a patient's experience may be a little like trying to judge the quality of a tapestry by pulling out and examining a single thread of that tapestry. We may have to satisfy ourselves with the examination of intermediate outcomes. To that end a definition of a good quality laboratory test has been offered: "...the most appropriate

210 C.R. Handorf / Clinica Chimica Acta 260 (1997) 207-216

laboratory test, correctly performed, reported, and utilized within a clini- cally optimal timeframe to produce the proper patient diagnosis and optimal patient management result."[3]

In 1986 the College of American Pathologists sponsored a conference entitled Quality Assurance in Physician Office, Bedside, and Home Testing [7]. The working group of that portion of the conference which dealt with analytical performance criteria published their recommendations; these bear repeating, since they provide a performance based bedrock for further examination of quality management practice in point of care testing. Their recommendations were: 1. The required analytical accuracy of a test is determined by its intended

clinical use and not by the location where the testing occurs. 2. Goals should be specified for the allowable inaccuracy of an individual

analytic result, the allowable inaccuracy of the analytical method, and the allowable imprecision of the analytical method. Where appropriate and feasible, goals may be specified for comparability of methods.

3. Sources of information which may serve as a basis for deriving analytical accuracy requirements include the opinion of expert clinicians, consensus of clinical opinions derived from clinical surveys, and models of analyti- cal performance based on estimates of analyte biologic variability.

4. The goal setting process for tests of substance identification differs from that for tests of substance quantitation. The goals for both types of testing should be based on the intended clinical use of the test results for a given analyte. Laboratories should select methods and instruments which best meet the analytical demands of intended clinical use.

5. Data must be collected to determine the current levels of analytical performance at various testing environments.

6. Instrument and reagent manufacturers' performance claims should be based on data derived from realistic field experience and expressed in terms meaningful to clinicians.

7. Clinicians must be made aware of the limits of analytical accuracy for a given testing procedure. Instruments and analytical methods should be selected that will meet the expected clinical demands for analytical accuracy.

8. Improvement toward desired analytical performance goals will require continual educational efforts that are targeted at physicians and pro- moted by laboratory professionals in cooperation with industry.

In the process of point of care testing, then, what should we measure to insure the quality of analysis and the reaching of analytical performance goals? In order to answer this question properly, we must look at the whole testing process, reduce it to a series of steps, and examine each of those steps to define potential weaknesses in the process which might allow for

C.R. Handorf / Clinica Chimica Acta 260 (1997) 207-216 211

failure and the production of a poor quality result. The total quality assessment of a laboratory testing event must evaluate the preanalytical, analytical, and postanalytical steps of the process. The preanalytical portion of each point of care test is generally brief and, by comparison with central laboratory testing, is less prone to error simply because of the lack of specimen transport or storage; errors may occur during this stage, however (e.g. specimen collected on the wrong patient). The analytical portion of each point of care test is contributed to by three components: (1) the analyst, who may be a very highly trained laboratory specialist or a highly trained professional of some other sort or a relatively lesser trained individ- ual such as a phlebotomist or aide; (2) the analytical hardware, which is the non disposable portion of the analytical system; and (3) the analytical software, which is the disposable portion of the analytical system such as reagent strips or cartridges. Each of these components is critical in the production of a correct result and each must be subjected to quality analysis. The postanalytical phase of the testing cycle is the portion of the cycle in which the patient result is recorded in the patient record and the information thus derived is integrated into the care of the patient. Typically, this phase is briefer in point of care testing than in centralized testing, and indeed this improvement in speed may shorten the time in which a result is made available for patient care; however, it is also a point in the cycle where the information may become lost or, for other reasons, may not be appropriately acted upon. This phase, too, must be subjected to quality review.

3. What are our measuring tools?

Traditionally, the laboratory has depended upon two basic quality man- agement approaches to the evaluation of the analytical segment of test performance. Quality control, originally deriving its statistical basis from the principles of factory production line control, has for years been the basic quality mechanism upon which we have relied. Built into a back- ground of acceptance/rejection rules, quality control of laboratory processes using frequent challenges with pseudo specimens of known concentrations and compositions has been the most significant mechanism used to see that analytical systems remained intact from day to day. Proficiency testing, on the other hand, has added the dimension of analysis of occasional speci- mens of unknown concentration or composition and also has typically included a feature of inter laboratory comparison of results. In the world of point of care testing, so different in structure and function from centralized laboratory testing, what is the place of these older quality management tools and what new tools are needed to insure and improve quality?

212 C.R. Handorf / Clinica Chimica Acta 260 (1997) 207-216

The main error that has been made in attempting to apply older quality management paradigms to point of care testing has been the failure to appreciate the basic functional differences between centralized and point of care testing. Because these testing approaches are so different, it would seem natural to conclude that different quality management approaches are required. Frequent pseudo sample quality control is not so critical in a point of care testing device designed for durability and simplicity of operation as it is in a large, complex multichannel instrument; indeed, it may make almost no sense at all in a device which consists of a hand held analyzer and a disposable reagent cartridge designed for unit use. In this sort of a system, the disposable cartridge is actually an integral part of the analytical device; disposing of the used cartridge and replacing it with another for the next analysis produces a new analytical system, one which can either be subjected to a quality control pseudo sample or to an actual patient sample but not to both. Does quality control by use of pseudo samples have a place in point of care testing? It absolutely does, however, the emphasis should be shifted somewhat. For instance one might employ such testing for lot validation of test cartridges at the point of use. In the example of quality control testing of glucose reflectance meters, frequent use of pseudo samples has the added advantage of allowing relatively untrained analysts the ability to practice their technique without the pressure of having to produce a patient result.

Manufacturers of unit use point of care devices have introduced the concept of electronic quality control (EQC), primarily for the purpose of circumventing the need for the expensive use of testing cartridges with pseudo samples. In EQC, a device which simulates the form and function of a test cartridge is introduced into the analyzer and one or more electronic function checks are performed. Although the use of EQC has been contro- versial among laboratorians and the regulators of laboratory medicine such as the Health Care Financing Administration, EQC should be adopted as a part of the overall scheme of quality management for appropriate point of care devices. It would seem, however, that EQC may only be used to monitor the performance integrity of the analytical device, and that it is no better than quality control pseudo samples at evaluating actual or expected performance of the cartridge-analytical device-patient sample complex.

Proficiency testing, a time honored tool in the laboratory quality manage- ment armamentarium, must be continued and enhanced in point of care testing. Precisely because point of care testing typically involves many more analysts than central laboratory testing, the monitoring of their perfor- mance is ideally suited to proficiency testing schemes in which peer compari- son is a strong component. Analysts who do not come from a traditional laboratory background may not fully appreciate the importance of profi-

C.R. Handorf / Clinica Chimica Acta 260 (1997) 207-216 213

ciency testing, and indeed may be somewhat threatened by the evaluative nature of this activity. The central laboratory must provide leadership and education regarding the importance and proper use of proficiency testing, so that it is seen as a vital tool for performance enhancement rather than as drudgery or frivolity.

An absolutely essential extension of the use of proficiency testing in the quality system of point of care testing is in the development of operator competency evaluation techniques. New techniques must be developed for operator proficiency evaluation, precisely because point of care testing approaches are so likely to involve less sophisticated analysts who may require closer observation of their analytical abilities. These approaches may include such features as real time monitoring of operator competency rather than retrospective analysis as is now the case with traditional proficiency testing. Also included may be a variable frequency approach, in which analysts who test patients less frequently are required to perform relatively more proficiency testing challenges. Provisions must be made for timely reeducation and remediation of analysts who do not meet established performance expectations.

Certainly another tool for the evaluation and maintenance of overall laboratory quality has been the peer driven educational and accreditation activities of such organizations as the College of American Pathologists. Biannual inspections by the CAP provide an opportunity for any hospital involved in point of care testing to challenge its own performance against a set of criteria. As important as these activities are, however, Laposata has clearly pointed out that for point of care testing, inspection of performance against a set of criteria must be done much more frequently than every 2 years to insure maintenance of performance [8]. This is probably related to the distribution of point of care testing among a large number of sites and an even larger number of analysts. Monthly internal inspections with a relatively limited checklist of required performance would appear to be ideal for maintenance of point of care testing programs.

Finally, the quality management of the total point of care testing process including the preanalytical and postanalytical portions requires use of total quality management approaches; many of the quality issues in this area are non quantitative and may span many areas of responsibility in a typical hospital setting. Fortunately, quality initiatives of the Joint Commission of Healthcare Organizations have made total quality management an integral part of every hospital culture so that addressing issues such as proper test utilization, result turnaround time, and test reporting have become easier. Proper problem identification, solution development, and monitoring re- quire the cooperation of all individuals involved in point of care testing.

214 C,R. Handorf / Clinica Chimica Acta 260 (1997)207-216

4. What are the indicators of performance?

Traditionally, many of the indicators of laboratory quality performance have been based in comparisons with biologic variability [9]. There has been a tendency to seek an analytical holy grail, to continually redefine quality of performance as our ability to perform to higher analytical standards has improved. This approach tends to ignore the relevance of results to what is required for patient care, and also ignores the enormous effort and cost expended on trying to reach this ever moving target. Unfortunately, we do not know very much about what is required in terms of quality control and proficiency testing to reach, at a defined level of certainty, the clinically relevant analytical goals that patients require for good care. Much of our setting of indicators for performance has been little more than empirical.

Watts has emphasized that from a clinician's point of view the impor- tance of analytical precision is somewhat variable, and depends upon the intended use of the test (such as group screening, individual single point testing, serial testing to monitor disease progression, or therapeutic moni- toring) [10]. In certain cases, precision may be compromised for another aspect of the test such as cost, availability, or timeliness of the result. In the opinion of some, results of blood glucose within 20% of the true value are acceptable for most decisions regarding insulin administration. It is clear that for meaningful indicators of analytical performance to be established, there needs to be a much closer attention paid to actual clinical needs. Based upon clinical relevance, indicators may be developed which are neither too rigorous nor to loose to gain statistical confidence that clinical needs are being met.

5. What are the reference databases?

As experience develops with point of care testing, it is vitally important that we make use of the mountains of quality data that become available to us. Only through the collection, classification, reduction, and analysis of these data may we extract information that will help us each to benchmark our performance against that of others and that will help us all ultimately to improve performance. An early analysis of such data has been the College of American Pathologists Q-probes study of bedside glucose moni- toring, first performed in 1991 and repeated in 1994 [11]. Participants in each study were asked to answer an exhaustive questionnaire regarding performance of bedside glucose testing within their institutions; there were 605 participants in the 1991 study and 544 participants in the 1994 study. Many aspects of the surveyed programs showed improvement during the

C.R. Handorf / Clinica Chimica Acta 260 (1997) 207 216 215

two studies including the use of more than one QC level, the availability of a written procedure manual, periodic training and performance review of personnel, scheduled preventive maintenance, external proficiency testing, and regular comparison of bedside glucose results with clinical laboratory glucose results. A troubling observation of these two studies was that despite improvement in these surrogates of quality, the accuracy of bedside glucose results compared with results generated in the central laboratory did not improve between the two studies. This suggests that we have quite a lot to learn about quality in bedside monitoring programs; much of this will be learned by collection and evaluation of database information.

6. Conclusion: do we have the will to measure?

O'Leary cautioned that for quality initiatives to be meaningful, we must have the will to measure and act upon our measurements; this is as true for point of care laboratory testing as for any other type of testing. In order to measure effectively and appropriately and to derive meaningful conclusions from those measurements, however, several difficult tasks must be per- formed. First, we must have clearly established, clinically relevant analytical performance goals; these must be based in our knowledge of pathophysiol- ogy and must have demonstrable relevance to good patient care. Second, we must carefully evaluate the entire testing process - -no t just the analytical portion, to determine potential weak points which may result in failure to reach the analytical performance goal. Third, we must select tools and, if necessary, develop new tools or new approaches to the use of quality management tool in point of care testing; we must be able to show, with a statistical degree of certainty, that we are doing the right amount of the right type of quality management to assure the quality which we desire. We can no longer afford blind adherence to antiquated quality control ap- proaches which have not been shown to be relevant to new laboratory testing such as testing at the point of care.

References

[1] Woo J, Henry JB. The advance of technology as a prelude to the laboratory of the twenty-first century. Clin Lab Med 1994;14:459-471.

[2] Westgard JO, Burnett RW, Bowers GN. Quality management science in clinical chemistry: a dynamic framework for continuous improvement of quality. Clin Chem 1990;36:1712-1716.

[3] Handorf CR. Quality control and quality management of alternate site testing. Clin Lab Med 1994;14:539-557.

216 C.R. Handorf / Clinica Chimica Acta 260 (1997) 207 216

[4] O'Leary D. Global view of how alternate site testing fits in with medical care. Arch Pathol Lab Med 1995;119:877-880.

[5] Lambird PA. Resource allocation and the cost of quality. Arch Pathol Lab Med 1990;114:1168-1172.

[6] Despotis GJ, Santoro SA, Kater KM, et al. Prospective evaluation and clinical utility of on-site coagulation monitoring in patients undergoing cardiac operation. J Thorac Cardiovasc Surg 1994;107:271-279.

[7] Howanitz PJ. Quality assurance in physician office, bedside, and home testing. Skokie, IL: College of American Pathologists, 1986.

[8] Laposata M, Lewandrowski KB. Near patient blood glucose monitoring. Arch Pathol Lab Med 1995;119:926-928.

[9] Fraser CG, Petersen, PH. Desirable performance standards for imprecision and bias in alternate sites: The views of laboratory professionals. Arch Pathol Lab Med 1995;119:909-913.

[10] Watts NB. Reproducibility (precision) in alternate site testing: A clinician's perspective. Arch Pathol Lab Med 1995;914-917.

[11] Howanitz PJ, Jones BA. Bedside glucose monitoring: Comparison of performance as studies by the College of American Pathologists Q-probes program. Arch Pathol Lab Med 1996;120:333-338.