Proceedings of the 2012 Industrial Engineering Research Conference

G. Lim and J.W. Hermann, eds.

Analysis of Uncertainty due to Calibration in Clinical Laboratory

Measurement Processes

Varun Ramamohan, Yuehwern Yih

Purdue University

West Lafayette, IN, USA

Jim Abbott

Roche Diagnostics Corporation

Indianapolis, IN, USA

George Klee

Mayo Clinic

Rochester, MN, USA

Abstract

The calibration of the instrument used in the analytical stage of a clinical laboratory testing process has a significant

contribution to the uncertainty of the final measurement result. In this paper, we present the development of a mathematical

model of the analytical stage of the serum cholesterol laboratory test that quantifies and integrates the uncertainties of the

various components of the laboratory measurement system. Monte Carlo simulation is used to estimate the net uncertainty

of associated with the measurement system. The model is further used to quantify the effect of uncertainty in calibration

parameters on both the location and spread of the distribution of the final laboratory measurement result. In addition, the

effect of the choice of calibrator concentration levels on the uncertainty of the measurement is quantified using Monte

Carlo simulation. The simulation is also used to identify the optimal calibrator concentration levels that minimize

uncertainty at medical decision points as well as across the patient concentration level range. The Monte Carlo

methodology of optimal calibrator selection presented in this paper can be extended to linear calibration measurement

systems in general.

Keywords: Calibration analysis, Uncertainty estimation, Clinical laboratory measurements, Monte Carlo simulation

1. IntroductionClinical laboratory testing plays a crucial role in the medical decision making process. A statement of uncertainty about the

result of a laboratory test provides information about the standard behavior of the laboratory measurement system.

Additionally, given that accreditation to ISO/IEC 17025 requires testing laboratories to provide estimates of measurement

uncertainty [1], and the passage of the Clinical Laboratory Improvement Amendments in 1988 requires clinical laboratories

to validate laboratory procedures and establish valid quality control systems, it becomes important to develop

methodologies that facilitate a.) the estimation of the uncertainty associated with a clinical laboratory measurement process,

and b.) analysis of the estimated uncertainty; that is, acquire an understanding of different contributors to the estimated

uncertainty. In this study, the measurement of cholesterol concentration in human blood serum is used as a test case to

study the effect of instrument calibration on the distribution of the measurement result (also referred to as the measurand).

This is accomplished by developing a mathematical model of serum cholesterol assay analysis procedure, and using the

Monte Carlo method to characterize the long-term behavior of such a model with uncertainty in its parameters. The

simulation model is used to study two aspects of the effect of calibration on the measurand distribution: one, the average

effect of calibration on the measurand distribution is understood by means of estimating the correlation introduced between

calibration function parameters when the parameters are estimated from the same set of data points, and the effect of this

correlation on the mean and standard deviation of the measurand distribution. The implications of this correlation

introduced by the calibration process for clinical laboratory policy are also explored. Secondly the effect of calibrator

concentration levels on the measurand distribution is quantified, and the simulation model is employed to choose the

Ramamohan, Abbott, Klee & Yih

combination of calibrator concentration levels that minimizes the total error at medical decision points as well as across a

given range of possible patient sample concentration levels. This study was carried out in collaboration with Roche

Diagnostics based in Indianapolis, IN (USA) and the Mayo Clinic at Rochester, MN (USA).

The concept of uncertainty and general guidelines for the development of uncertainty models were first introduced in the

ISO/BIPM/OIML/IUPAC Guide to the Expression of Uncertainty in Measurement in 1993 [2] and revised in its subsequent

editions and companion publications [3, 4]. The ‘uncertainty’ associated with the quantity to be measured or measurand

refers to a parameter used to characterize the dispersion of values that can be attributed to the measurand [1]. This concept

of uncertainty is used to characterize the variation in the components and parameters of the system. In this paper, any

component that is subject to variability is characterized by a probability distribution with the expected value or mean and

the standard deviation as parameters. Additionally, in this paper, the effect of calibration on the location or mean of the

measurand distribution is also explored in addition to the uncertainty.

There have been several attempts to model specific analytical laboratory measurement processes [5-9], and in particular,

some attempts to estimate the measurement uncertainty associated with the serum cholesterol laboratory measurement

process [10-12]. Fuentes Arderiu et al. [10] compare the measurement uncertainty associated with the direct plasma low-

density lipoprotein (LDL) cholesterol method of measurement with that estimated indirectly via the Friedewald equation.

However, the methods involved in the estimation of measurement uncertainty described in this paper involve either direct

estimation from laboratory test results or the use of the Friedewald equation. The methodology used by Kouri et al. [11] and

Chen et al. [12] to estimate measurement uncertainty for the serum cholesterol assay involved the use of the rules of

uncertainty propagation described in [2] and [13]. In this paper, a mathematical model of the serum cholesterol laboratory

assay analysis procedure is developed and the Monte Carlo method is used to estimate the uncertainty associated with such

a mathematical model. To the best of our knowledge, there is no literature in the field on the use of a Monte Carlo model to

estimate and analyze the uncertainty associated with a clinical laboratory measurement process.

However, the concept of modeling analytical methods in chemical laboratories from a systems engineering perspective has

previously been suggested by Aronsson, de Verdier and Groth [14], and Krouwer [15]. In this paper, the different stages of

the serum cholesterol assay analysis procedure – from collecting the patient sample to reporting the results – are

conceptualized as a self-contained system. The input to the system - represented by the patient sample - is processed by the

system components, and a property of the sample is quantified and expressed as the output quantity, represented by the

result of the measurement process. The principal components of the serum cholesterol laboratory measurement process are

identified as being the following: the patient sample, the calibrators, the measuring instrument and the reagents. In addition,

the assay analysis procedure is divided into two phases: the calibration phase, wherein the parameters of the calibration

function relating the measured property and the quantity of interest or measurand are estimated; and the measurement

phase, wherein the patient sample with unknown cholesterol concentration is analyzed by the calibrated instrument.

The Monte Carlo method is used to characterize the long-term behavior of the system under conditions of uncertainty in its

components. As stated in the GUM Annex 1 [16], the law of uncertainty propagation proposed in the GUM becomes

unsuitable in the following situations: a.) the mathematical model involved is non-linear in nature; b.) the behavior of the

measurand of the system is not Gaussian in nature; and c.) estimating the degrees of freedom for the sources of uncertainty

is not possible, especially for Type B characterizations [2], which involve non-statistical characterizations of the variation

of a parameter. The third case is particularly relevant here, as the statistical characterization (Type A) of the various sources

of uncertainty using relevant data is often not practically possible in the laboratory. The Kragten spreadsheet method [17]

also becomes unsuitable for precisely the above reasons, as it is an automation of the GUM law of propagation of

uncertainty. The principal advantage of using the Monte Carlo method is that it requires the development of a simulation

model of the process. This in turn enables conducting simulation experiments to analyze various aspects of the laboratory

measurement process, such as estimating the contribution of the different components and phases of the system. In addition,

it also enables the identification of optimal/near-optimal values for the parameters of the system, illustrated in this paper by

identifying combinations of calibrator concentration levels from a set of possible calibrator combinations that minimize the

uncertainty at medical decision points and across the patient sample concentration range.

2. Model Development

The system approach is used to develop an uncertainty model of the analysis procedure of the serum cholesterol assay. The

assay is analyzed on the Roche Diagnostics Corporation P-Modular measurement system. As discussed in section 2, the

system consists of the following main components: a.) the measuring instrument, b.) calibrators, c.) reagents and d.) patient

Ramamohan, Abbott, Klee & Yih

samples. The measurand of the system is the cholesterol concentration of the patient sample, and it is an indirect

measurand. The instrument quantifies the optical absorbance of the reaction of the sample and added reagents. A calibration

function then converts the measured absorbance value into the desired concentration value. Therefore, for this particular

system, the calibration function serves as the starting point upon which the mathematical model of the system is built. The

relationship between absorbance and cholesterol concentration is linear; that is, it can be written as: C = mA + I. Here C

denotes the cholesterol concentration; m and I are parameters of the calibration function and A denotes the absorbance

reading obtained from the instrument. An average reference line is established in order to generate random values of

absorbance for the simulation. P-modular system calibration data obtained from Mayo Clinic is used to establish the

average line. The data consists of absorbance readings generated from calibrator solutions with cholesterol concentrations

of 0 mg/L (water blank) and 1650 mg/L obtained over 7 days. The means of these absorbance values at both concentration

levels are computed. The average reference line is then characterized using these two data points.

2.1 Calibration Phase

In this phase, the linear relationship between absorbance and cholesterol concentration is characterized by estimating the

parameters of the function. The calibration line is characterized by measuring the absorbance of two calibrator solutions;

one with a relatively high level of cholesterol concentration and other a water blank (a cholesterol concentration of zero).

Two principal components of the system are involved in this process: the calibrator solutions and the measuring instrument.

Consequently, there are two principal sources of uncertainty associated with these components, calibrator uncertainty and

instrument uncertainty.

Three factors contribute to the uncertainty of the calibrator solution. The first refers to the uncertainty that occurs while

preparing the calibrator solution; that is, it is the uncertainty in assigning a particular concentration to the solution. It is

called calibrator set-point uncertainty (denoted as uc1). The second source represents the difference in concentration that

occurs when different vials of the calibrator solution are prepared from the master batch. This is referred to as vial-to-vial

variability (uc2). The third source describes the deterioration of the calibrator solution over time as the calibrator is

reconstituted each time it is used. This time-dependent source of uncertainty is known as calibrator reconstituted stability

(uc3(t)).

Since data for the above sources of uncertainty was not available, it was decided to use specifications provided by the

manufacturer to characterize the uncertainties of these sources. These specifications were expressed in terms of percentage

error. For example, the specification for calibrator set-point uncertainty was obtained as ± 2%. After discussion with both

the clinicians and the instrument manufacturer, it was decided to characterize the uncertainties as Gaussian distributions,

and the specifications are used to set the parameters of the distribution. As an illustration, for calibrator set-point

uncertainty the mean of the distribution was set at 0%; and the specification limit of 2% is used to represent the Gaussian

distribution at two standard deviations from the mean, thereby resulting in a standard deviation of 1%. Thus, due to

calibrator set-point uncertainty, the actual calibrator concentration of a calibrator solution with a desired concentration of

1650 mg/L will vary according to the Gaussian distribution described by a mean of 1650 mg/L and a standard deviation of

16.50 mg/L. The other sources of uncertainty were also characterized in a similar manner. The notation used for each

uncertainty component is also given in the table. This method of characterizing the uncertainty associated with a

contributing component is known as ‘Type B’ estimation, as opposed to the ‘Type A method’ [1], which involves

characterizing the variation of a source of uncertainty by fitting a suitable distribution to data available for that particular

source.

These sources of calibrator uncertainty are combined serially in the order in which they are introduced into the calibrator

preparation process. Let C denote the target value to be assigned to the calibrator. Calibrator reconstituted stability is a time

dependent quantity, so the corresponding distribution has to be sampled for each time period. Therefore, the equation for

calibrator concentration when these sources of uncertainty are introduced one after the other becomes:

(1)

Here C’ denotes the calibrator concentration after all components of calibrator uncertainty have been introduced. The

absorbance reading A’, which corresponds to a cholesterol concentration of C

’, is calculated from the average line. The

absorbance of this calibrator, with calibrator uncertainty incorporated into it, is then measured by the P-module instrument.

The measurement of the absorbance of the calibrator C’ will introduce instrument uncertainty into the model. This process

is described below.

Ramamohan, Abbott, Klee & Yih

The P-module measuring instrument measures the absorbance of the reaction of the calibrator solution with the reagents.

The P-module instrument contains a reaction disk consisting of multiple reaction cells. A sampling mechanism pipettes the

sample into the reaction cell and a reagent mechanism pipettes the reagent into the reaction cell. A stirrer paddle facilitates

the chemical reaction which results in the production of a chromophore molecule that absorbs the light that the photometer

passes through the reaction cell.

There are three sources of uncertainty associated with the measuring instrument, representing the sample, reagent and the

photometer. The first is that associated with the sample pipetting mechanism, and it represents the uncertainty in the

volume of sample pipetted out into the reaction cell. It is referred to as sample pipetting uncertainty (um1). The absorbance

of the final reaction mixture is linearly proportional to the concentration of the chromophore, and therefore to the sample

volume – an x% change in the sample volume added produces the same (x)% change in the optical absorbance. The second

uncertainty is that associated with the reagent pipetting mechanism, and it corresponds to the uncertainty in the volume of

reagent pipetted out into the reaction cell. It is referred to as reagent pipetting uncertainty (um2). Since this chemical

reaction is an end-point reaction, the reagent is present far in excess of what is required. Therefore, a change in the volume

of the reagent would not have a significant impact on the amount of chromophore produced, and therefore would not

substantially affect the absorbance. This is an example of a system component that, while possessing significant uncertainty

in itself, does not contribute to the uncertainty associated with the measurand. The third source of uncertainty is that

associated with the photometer. This is referred to as photometer uncertainty (um3). These sources of uncertainty are

characterized in much the same way as the components of calibrator uncertainty.

These components are combined serially in the order that they affect the desired absorbance value. Following the reasoning

used to determine calibrator uncertainty, we have the following expression for absorbance after photometer uncertainty is

introduced.

(2)

The above expression denotes the final value for absorbance after all components of uncertainty involved in the calibration

process have been incorporated into the model. This process is repeated for the two different calibrator concentration levels.

The corresponding concentration values and absorbance readings are used to estimate the values of the calibration line

parameters m and I. Thus these parameters incorporate the uncertainty due to all the different components of calibrator and

instrument uncertainty. In this particular case, this process was implemented for calibrator concentration levels of 0 mg/L

(water blank) and 1650 mg/L. Calibrator uncertainty and sample pipetting uncertainty is meaningless in case of the water

blank, so only the photometer uncertainty contributes to the calibration uncertainty.

2.2 Measurement Phase

Once the calibration line C = mA + I has been established, the process moves into the next phase; that is, the measurement

phase. In terms of the calibration line, the uncertainty associated with this phase is that associated with the independent

variable A; or that associated with the sample absorbance reading. There are three major components of uncertainty

associated with this phase, the patient sample uncertainty, reagent uncertainty and the instrument uncertainty. Patient

sample uncertainty typically relates to the pre-analytical stage of the clinical laboratory testing process, and is therefore out

of the scope of this paper. However, in some cases where the desired concentration of the sample is known beforehand,

such as for quality control processes, the uncertainty associated with the sample can be estimated. It is characterized by a

Gaussian distribution with the mean being the desired or ‘true’ absorbance reading, and the standard deviation depending

upon the specifications obtained from the sample manufacturer.

Instrument uncertainty has been dealt with previously in section 3.1. If we denote the ‘true’ concentration of the sample as

Cs, and the corresponding absorbance as As, the absorbance obtained after the incorporating sample and instrument

uncertainty is expressed as:

(3)

Here As’ denotes the absorbance after going through the measurement phase, and us denotes the random realization of the

sample uncertainty distribution. When this absorbance value is input into the calibration line, we get the system output- the

cholesterol concentration of the sample- as:

(4)

The value of Cs’ is recorded for a large number of realizations of all the uncertainty component distributions and then the

net system uncertainty is calculated as the standard deviation of these recorded values of Cs’.

Ramamohan, Abbott, Klee & Yih

3. Calibration Analysis & Optimization

The calibration phase has a significant impact on both the location and spread of the distribution of the measurand, and in

this section, we quantify this change in measurand distribution due to calibration. In this section, two aspects of the effect

of calibration on the measurand distribution are analyzed: a.) the first case involves understanding the mechanism of

calibration; that is, the correlation between the parameters of the calibration function is estimated and the effect of this

correlation on the measurand distribution is quantified, and b.) a methodology is proposed for identifying calibrator

concentration levels that minimize uncertainty at medical decision points as well as across the range of patient sample

concentration levels.

3.1 Calibration Correlation Analysis

In the case of the cholesterol assay, the parameters of the linear relationship – the slope and the intercept – between analyte

concentration and optical absorbance are characterized during the calibration phase. When the parameters of the calibration

function are estimated from the same set of data points, the calibration process introduces a correlation between the slope

and the intercept. This correlation is estimated using the simulation model in the following way: random vectors of length

100 were recorded for the slope and intercept and the correlation coefficient between these vectors was estimated. The

correlation coefficient was estimated to be -0.97. The slope and the intercept of the calibration function were found to be

normally distributed. The means and the standard deviations (denoted by μ and σ, respectively) of the slope and intercept

were computed.

In order to estimate the effect of the correlation introduced by the calibration process, the slope and the intercept were

varied from (μ - 3σ) to (μ + 3σ) in increments of 1σ, thereby resulting in 7 different values for both the slope and the

intercept. This range represents 99.7% of the possible variation of the slope and the intercept under random simulation

conditions. Therefore, 49 calibration lines are generated for each combination of the 7 slopes and intercepts, and using the

simulation model developed, 60 measurements from patient samples with a known cholesterol concentration of 105 mg/dL

are recorded for each such calibration line, representing the measurand distribution for each calibration line. The mean and

standard deviation of the measurand distribution are computed from the 60 measurements recorded for each calibration line.

Therefore, this represents a system wherein the slope and the intercept of the calibration line are independent of each other.

The means and standard deviations of the measurand distribution recorded for the calibration lines are summarized in Table

1(a) and Table 1(b) respectively. As can be seen from Table 1(a), when both the slope and the intercept are at a distance of -

3 standard deviations from their respective mean values, the mean and the standard deviation of the measurand distribution

assume values of approximately 87.5 mg/dL and 2.52 mg/dL. The corresponding values when the calibration line

parameters are at a distance of +3 standard deviations from their respective mean values are 122.4 mg/dL and 2.82 mg/dL

respectively. As is evident, these are significantly different (approximately 16.67%) from the target value of 105 mg/dL,

and can lead to significantly different diagnoses at medical decision points.

Table 1(a): Effect of uncorrelated calibration parameters on mean of measurand distribution

μI - 3σI μI - 2σI μI - σI μI μI + σI μI + 2σI μI + 3σI

μS - 3σS 87.47

89.59

91.26

92.62

94.57

96.58

98.13

μS - 2σS 91.89

93.50

94.88

97.31

98.43

100.66

102.03

μS - σS 96.09

97.62

99.33

101.06

102.55

104.27

106.37

μS 100.21

101.90

103.12

105.15

106.78

109.20

110.30

μS + σS 104.02

106.18 107.83

109.30

110.75

113.31

114.81

μS + 2σS 108.04

110.10

111.26

113.68

115.32

116.67

118.50

μS + 3σS 112.49

114.34

115.78

116.84

119.16

121.32

122.40

Ramamohan, Abbott, Klee & Yih

Table 1(b): Effect of uncorrelated calibration parameters on spread of measurand distribution

μI - 3σI μI - 2σI μI - σI μI μI + σI μI + 2σI μI + 3σI

μS - 3σS 2.52

2.31

2.52

2.25

2.69

2.71

2.64

μS - 2σS 2.49

2.69

2.31

2.60

2.72

2.67

2.84

μS - σS 2.98

2.68

2.69

2.51

2.93

2.75

2.69

μS 3.07

2.84

2.44

2.83

2.97

2.78

2.71

μS + σS 2.97

2.63

3.37

3.45

2.68

2.68

2.53

μS + 2σS 2.94

3.08

3.16

2.75

2.59

2.77

2.95

μS + 3σS 2.92

3.30

3.03

2.81

2.86

2.88

2.82

The correlation introduced by calibration restricts the possible combinations of the slope and the intercept from 49 to only

seven possible combinations; and this process of restriction is described below. The effect of this correlation is estimated by

modeling the variation of the slope and the intercept as jointly distributed Gaussian random variables. The individual

distributions of both the slope and the intercept have earlier been found to be Gaussian; and the random vector with the

slope and the intercept as components satisfy all the conditions to have a bivariate Gaussian distribution. If S ~ N(μS, σS)

and I ~ N(μI, σI) represent the parameters of the individual marginal distributions of the slope and the intercept respectively,

then the probability density function of the joint probability distribution of both random variables is given by:

Here ρ represents the coefficient of correlation between the slope and the intercept of the calibration line. Given the joint

distribution of the slope and the intercept, the conditional distribution of the intercept for a particular value s of the slope is

described by the parameters below:

(6)

The effect of this correlation is estimated by varying the slope from (μS - 3σS) to (μS + 3σS) in increments of σS, and then

using the conditional distribution of the intercept to compute the expected value of the intercept at each value of the slope.

For instance, when the slope is fixed at a value of (μS - 2σS), the expected value of the intercept is computed from its

conditional distribution defined above, as shown below:

(7)

That is,

(8)

60 measurements of samples with a target (known) concentration of 105 mg/dL were recorded for each of the seven

calibration lines generated by computing the slope and intercept as described above. The mean and standard deviation of

these measurements were computed for each calibration line, and are summarized in Table 2. The off-diagonal elements of

the 7X7 matrices contained in Tables 1(a) and 1(b) can be conceived of as representing the measurand distribution

parameters recorded for calibration lines with the slope and the intercept having a correlation coefficient of -1. Therefore,

the estimated coefficient of correlation of -0.97 between the slope and the intercept implies that the values of the measurand

distribution parameters recorded in Table 2 should not be significantly different from the off-diagonal elements in Tables

1(a) and Table 1(b). As can be seen from Table 2, this is indeed the case. Therefore, the process of estimating the

parameters of the calibration function from the same set of data points induces a correlation that restricts the possible

combinations of the slope and the intercept to the off-diagonal elements, and limits the effect of calibration on the location

Ramamohan, Abbott, Klee & Yih

and spread of the measurand distribution to that described in Table 2. The maximum deviation (bias) from the target

concentration of 105 mg/dL observed in Table 2 is 7.1%, as opposed to a maximum deviation of 16.67% observed in Table

1(a).

Table 2: Effect of correlated calibration parameters on measurand distribution

Position of slope

distribution

Slope Avg. Intercept Measurand mean

(mg/dL)

Measurand std.

deviation (mg/dL)

μ - 3*σ 0.0491 -69.26 97.54 2.78

μ - 2.5*σ 0.0497 -70.11 99.45 2.39

μ - 2*σ 0.0503 -70.96 99.93 2.55

μ - 1.5*σ 0.0509 -71.82 101.65 3.08

μ - 1*σ 0.0515 -72.67 103.00 2.99

μ - 0.5*σ 0.0521 -73.53 103.49 2.99

μ 0.0527 -74.38 105.05 2.76

μ + 0.5*σ 0.0533 -75.23 106.15 2.65

μ + 1*σ 0.0539 -76.09 107.91 3.01

μ + 1.5*σ 0.0545 -76.94 108.84 2.98

μ + 2*σ 0.0551 -77.79 110.70 3.04

μ + 2.5*σ 0.0557 -78.65 111.01 2.53

μ + 3*σ 0.0563 -79.50 112.44 2.94

3.2 Calibrator Optimization

In this section, we explore the effect of choice of calibrator concentration levels on the measurand distribution. In the case

of the serum cholesterol assay, the straight line calibration function is established by measuring the optical absorbance of

two calibrator solutions – samples of known cholesterol concentration. In this section, the concentration of each calibrator

is varied from 0 – 250 mg/dL in increments of 10 mg/dL, resulting in 26 possible concentration levels for each calibrator,

and 676 possible calibration lines generated for each combination of calibrator concentration levels. The effect of these

calibration lines is studied on the parameters of the measurand distribution of patient samples with a target cholesterol

concentration of 200 mg/dL, the concentration level above which the patient is judged as having ‘borderline high’

cholesterol levels [18]. Therefore, for each of the 676 calibration lines generated, 60 measurements from patient samples

with a known (target) cholesterol concentration level of 200 mg/dL are recorded, and the mean and standard deviation for

each set of 60 measurements for each calibrator combination are computed. Since it is not possible to display results for all

676 calibration lines, the results from the above simulation experiments are summarized in the Tables 3(a) and 3(b) in the

following manner: in Table 3(a), one of the calibrators is varied from 0 to 240 mg/dL in increments of 30 mg/dL, and for

each value of the first calibrator the corresponding value of the second calibrator that minimizes the absolute deviation from

the desired concentration of 200 mg/dL is listed. In other words, for each value of the first calibrator, the value of the

second calibrator that minimizes the displacement of the mean of the measurand distribution from its desired value is

chosen. Additionally, the mean and standard deviation corresponding to the calibrator combination are also listed. The total

error [19] corresponding to a given calibrator combination is calculated as:

Ramamohan, Abbott, Klee & Yih

Total Error = Bias (%) + 1.96*Imprecision (%) (9)

The National Cholesterol Education Program has defined the maximum allowable total error that prevents misdiagnosis for

the serum cholesterol assay as 8.9%, and hence the use of the total error criterion to judge the effect of a given calibrator

combination on the measurand distribution. The results are organized in a similar manner in Table 3(b); however, in this

case, for each value of the first calibrator, the corresponding value of the second calibrator that minimizes the standard

deviation is listed.

Table 3(a): Effect of calibrator concentrations on mean of measurand distribution

Calibrator 1

(mg/dL)

Calibrator 2

(mg/dL)

Measurand mean

(mg/dL)

Measurand spread (mg/dL) Total Error (%)

0 200 200.02 4.27 4.19

30 0 200.04 4.24 4.18

60 190 200.06 4.33 4.27

90 250 199.92 4.24 4.2

120 240 199.90 4.22 4.19

150 0 199.94 4.28 4.22

180 250 199.99 4.31 4.23

210 140 199.95 4.26 4.2

240 50 200.05 4.22 4.16

Table 3(b): Effect of calibrator concentrations on standard deviation of measurand distribution

Calibrator 1

(mg/dL)

Calibrator 2

(mg/dL)

Measurand spread

(mg/dL)

Corresponding measurand

mean (mg/dL)

Total Error (%)

0 50 4.16 199.17 4.49

30 160 4.19 200.59 4.4

60 170 4.18 199.89 4.15

90 60 4.21 199.36 4.45

120 240 4.21 199.89 4.18

150 40 4.19 199.48 4.37

180 60 4.13 199.78 4.16

210 120 4.15 200.46 4.3

240 120 4.18 200.33 4.26

The calibrator combination that minimizes the total error at the 200 mg/dL decision point can be chosen from Tables 3(a)

and 3(b). From the above tables, this calibrator combination is 60/170 mg/dL, with a total error of 4.15%. It should be noted

here that the calibrator combinations of 50/240 and 60/180 mg/dL respectively result in total errors of 4.16% at 200 mg/dL;

which are not significantly different from the total error of 4.15%. Therefore, this methodology identifies multiple feasible

calibrator combinations that result in low total errors around the medical decision point. Further, it is desirable that the

calibrator combination chosen above results in low total errors across the range of possible patient sample concentrations.

The simulation methodology can be used to verify that the optimal calibrator combination chosen above results in

medically insignificant total errors across the range of possible patient sample concentration. This is illustrated by

estimating the total error due to the optimal calibrator combination – 60/170 mg/dL – across a patient sample range of 50 –

300 mg/dL, covered in increments of 25 mg/dL. These results are summarized in Table 4 below. The National Cholesterol

Education Program has defined the maximum allowable total error that prevents misdiagnosis for the serum cholesterol

Ramamohan, Abbott, Klee & Yih

assay as 8.9% [19], and as can be seen from the table below, the optimal calibrator combination chosen above results in

total errors less than 8.9% across the patient sample concentration range studied below. Therefore, this study illustrates the

use of the modeling/simulation methodology to choose calibrator combinations that result in low total errors at medical

decision points as well as across the range of possible patient sample concentrations.

Table 4: Effect of optimal calibrator concentration (60/170 mg/dL) on patient sample spectrum

Patient Sample

Concentration

(mg/dL)

Measurand spread

(mg/dL)

Measurand mean

(mg/dL)

Total Error (%)

50 1.95 50.25 8.14

75 2.3 74.88 6.18

100 2.73 99.97 5.38

125 3.11 125.62 5.38

150 3.57 149.68 4.87

175 3.88 175.37 4.56

200 4.22 199.78 4.25

225 4.7 226.71 4.86

250 5.04 250.04 3.98

275 5.46 274.8 3.97

300 5.87 300.35 3.96

4. Conclusions The primary implication from the analysis of the effect of correlation on the measurand distribution relates to the necessity

of using a fully determined calibration function in order to convert the measured property into the desired measurand value.

As shown in the previous section, if the parameters of the calibration function are estimated in such a manner that they are

independent random variables, they result in significantly higher total errors as compared to the case wherein they are

estimated using the same set of data points. While the correlation introduced due to estimation of the calibration parameters

from the same set of data points is a natural consequence of the nature of the calibration function, this paper presents a

methodology for quantifying said correlation and its effect on the measurand distribution. A possible avenue for further

research involves investigating the generalizability of the calibration correlation analysis results described in this paper to

linear calibration systems as a whole.

The methodology employed in this paper to identify calibrator concentration levels that minimize uncertainty around

medical decision points and across the possible range of patient sample concentration levels can be extended to linear

calibration systems in general, and can be particularly useful in high precision applications. It should be noted that this

methodology does not identify strictly optimal calibrator combinations that minimize uncertainty around medical decision

points. However, it provides a convenient alternative that chooses the calibrator combination that minimizes uncertainty

around a target measurand value from a set of possible calibrator combinations.

There are a few important points that need to be kept in mind while developing the uncertainty model. First, developing a

mathematical model that is accurate and adequately representative of the clinical laboratory testing system is a vital part of

the process of modeling such a system. A close working relationship with both the clinicians and the instrument

manufacturers is required. These requirements prescribe that this simulation approach would be useful for clinicians and

instrument manufacturers as a design tool to set quality specifications and to estimate and analyze the contribution of the

different sources of uncertainty to the net system uncertainty. Some possible avenues for future research are discussed here.

There are some potential sources of uncertainty that, although not included currently in the study, require investigation;

some examples being the contribution of reagent uncertainty to the uncertainty of the chemical reaction itself, and the

uncertainty associated with the average reference line. Additionally, the use of the model to simulate and evaluate the effect

of existing and new quality control protocols on the net system uncertainty can be explored.

Ramamohan, Abbott, Klee & Yih

References 1. ISO/IEC 17025, 1999, “General requirements for the competence of calibration and testing laboratories”, Geneva:

ISO.

2. BIPM, IEC, IFCC, ISO, IUPAC, IUPAP and OIML, 1993, “Guide to the expression of uncertainty in

measurement”, Geneva: ISO.

3. Joint Committee for the Guides in Metrology 100, 2008, “Evaluation of measurement data – Guide to the

Expression of Uncertainty in Measurement”,

http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf.

4. Joint Committee for the Guides in Metrology 104, 2009, “Evaluation of measurement data – An introduction to the

‘Guide to the Expression of Uncertainty in Measurement’ and related documents”,

http://www.bipm.org/utils/common/documents/jcgm/JCGM_104_2009_E.pdf.

5. Kallner, A. and Waldenström, J, 1999, “Does the uncertainty of commonly performed glucose measurements

allow identification of individuals at high risk for diabetes?” Clinical Chemistry and Laboratory Medicine, 37,

907–912.

6. Petersen, P.H., Brandslund, I., Jorgensen, L., Stahl, L., Olivarius, N. de Fine and Borch-Johnsen, K., 2001,

“Evaluation of systematic and random factors measurements of fasting plasma glucose as the basis for analytical

quality specifications in the diagnosis of diabetes. 3. Impact of the new WHO and ADA recommendations on

diagnosis of diabetes mellitus”, Scandinavian Journal of Clinical and Laboratory Investigation, 61, 191-204.

7. Patriarca, M., Castelli, M., Corsetti, F. and Menditto, A, 2004, “Estimate of uncertainty of measurement from a

single-laboratory validation study: Application to the determination of lead in blood”, Clinical Chemistry, 50,

1396-1405.

8. Fuentes-Arderiu, X. and Dot-Bach, D, 2009, “Measurement uncertainty in manual differential leukocyte

counting”, Clinical Chemistry and Laboratory Medicine, 47, 112-1155.

9. Ramamohan, V., Chandrasekar, V., Abbott, J., Klee, G.G. and Yih, Y., 2011, “A Simulation-Based Methodology

for Uncertainty Modeling and Analysis of Clinical Laboratory Measurement Processes”, Proceedings of the 2011

Industrial Engineering Research Conference, Reno, NV, USA.

10. Fuentes-Arderiu, X., Braga-Fernandez, S., Freire-Campo, L., Garcia-Lario, J.V., Garcia-Martin, M.I., Jorde-

Andres, J.L., Largo-Caballerizo, E., Lugo-Arocena, J., Pardo-Laseca, Cristina., Villanueva-Curto, S. and Juve-

Cuxart, S., 2009, “Comparison of measurement uncertainties in direct plasma low-density lipoprotein cholesterol

method of measurement and indirect estimation according to Friedewald equation”, Accreditation and Quality

Assurance, 14, 179-183.

11. Kouri, T., Siloaho, M., Pohjavaara, S., Koskinen, P., Malminiemi, O., Pohja-Nylander, P. and Puukka, R., 2005,

“Preanalytical factors and measurement uncertainty”, Scandinavian Journal of Clinical and Laboratory

Investigation, 65, 463-476.

12. Chen, H., Deng, X-L., Bi, X-Y., Yang, P., Zhang, L-P. and Chen H-C, 2010, “Study on measurement uncertainty

for total cholesterol in routine clinical laboratory”, Chinese Journal of Clinical Laboratory Science, 5.

13. EURACHEM/CITAC Guide, 2000, “Quantifying uncertainty in analytical measurements”, 2nd

Edition.

14. Aronsson, T., de Verdier, C-H. and Groth, T., 1974, “Factors influencing the quality of analytical methods – A

Systems Analysis, with Use of Computer Simulation”, Clinical Chemistry, 20(7), 738 – 748.

15. Krouwer, J, 2002, “Setting Performance and Evaluating Total Analytical Error for Diagnostic Assays”, Clinical

Chemistry and Laboratory Medicine, 48(6), 919-927.

16. Joint Committee for the Guides in Metrology 101, 2008, “Evaluation of measurement data – Supplement 1 to the

“Guide to the Expression of Uncertainty in Measurement” – Propagation of distributions using a Monte Carlo

method.”, http://www.bipm.org/utils/common/documents/jcgm/JCGM_101_2008_E.pdf.

17. Kragten, J, 1994, “Calculating standard deviations and confidence intervals with a universally applicable

spreadsheet technique”, Analyst, 119, 2161–2165.

18. National Heart Lung and Blood Institute, National Institutes of Health website:

http://www.nhlbi.nih.gov/health/health-topics/topics/hbc/diagnosis.html

19. Bachorik, P.S. and Ross, J.W., 1995, “National Cholesterol Education Program Recommendations for

measurement of low-desnity lipoprotein cholesterol: Executive summary” Clinical Chemistry, 41(10), 1414-1420.