STATISTICS IN MEDICINE, VOL. 10, 341-349 (1991)

A STATISTICAL METHOD FOR ASSESSING A THRESHOLD IN EPIDEMIOLOGICAL STUDIES

K. ULM lnsiiiute for Medical Statistics and Epidemiology, Technical University Munich, Ismaningerstrasse 22, 8000-Munich 80,

Federal Republic of Germany

SUMMARY I describe a method for estimating and testing a threshold value in epidemiological studies. A threshold effect indicates an association between a risk factor and a defined outcome above the threshold value but none below it. An important field of application is occupational medicine where, for a lot of chemical compounds and other agents which are non-carcinogenic health hazards, so-called threshold limit values or TLVs are specified. The method is presented within the framework of the logistic regression model, which is widely used in the analysis of the relationship between some explanatory variables and a dependent dichotomous outcome. In most available programs for this and also for other models the concept of a threshold is disregarded. The method for assessing a threshold consists of an estimation procedure using the maximum- likelihood technique and a test procedure based on the likelihood-ratio statistic R, following under the null hypothesis (no threshold) a quasi one-sided x 2 distribution with one degree of freedom. This use of this distribution is supported by a simulation study. The method is applied to data from an epidemiological study of the relationship between occupational dust exposure and chronic bronchitic reactions. The results are confirmed by bootstrap resampling.

1. INTRODUCTION

The goal of many epidemiological studies in occupational medicine is to clarify whether a specific agent is a health hazard. For those agents which constitute a hazard, one of the main problems is to establish the existence of a threshold concentration. In occupational medicine such a concentration is called a threshold limit value or TLV. In Germany the TLV is defined as the maximum permissible concentration of a chemical compound present in the air within a working area (as gas, vapour or particulate matter) which, according to current knowledge, generally does not impair the health of the employee or cause undue annoyance.' This implies that below the TLV there should be no risk related to the agent, and only above this concentration does the risk of disease or health impairment increase. About 70 years ago Flury and Heuber showed that inhalation of prussic acid below a certain concentration, independently of the duration of inhalation, did not cause death by poisoning.'

It should be noted that the concept of a TLV applies only to non-carcinogenic agents. With carcinogenic agents it is assumed that there is no threshold and that only a so-called virtually safe dose or VSD can be estimated. There is much controversy about how to assess a VSD,' but this is not the topic of this paper; I focus on non-carcinogenic agents. In Figure 1 situations both with and without a threshold are shown; if there is a background response without any exposure then both curves are shifted upwards. The mathematical aspects of the concept of thresholds are

0277-6715/91/030341-09$05.00 0 1991 by John Wiley & Sons, Ltd.

Received July 1988 Revised July 1990

342 K. ULM

I without threshold

threshold dose

Figure 1. Dose-response curves with and without a threshold value

described by Brown.3 From the statistical point of view the question is whether one or two segmented regression lines are appropriate and, in the case of two segmented regression lines (the lower has to be horizontal), the location of the change point.

Some approaches to this problem are presented in the literature:-" and an overview has been given by Shaban.'' Most are based on the method of ordinary least squares and are therefore suitable if the dependent variable is measured on a continuous scale. In epidemiological studies the dependent variable is often dichotomous (like disease absent or present) so that, in all models used for the analysis of such studies, the estimation procedure is based on the theory of maximum likelihood (ML), which is equivalent to iteratively reweighted least squares.

This paper presents a method for estimating a threshold (TLV) based on an ML procedure, testing hypotheses about the threshold and calculating a P per cent confidence interval for the threshold concentration.

2. METHOD

The method for estimating a threshold is presented within the framework of the logistic regression model, but it can easily be transferred to other models as well. The dependent variable Y has two possible outcomes, 0 for individuals without disease and 1 for those with disease, and the explanatory variables are denoted by xlr . . . , xp. To simplify the notation, a model with just one explanatory variable x is considered first. The logistic model is given by

= Po + PlX, p (4 1 - P(x) logit P(x) = In

where P(x) = P ( Y = 1 I X = x) and Po and B1 are constants. If the explanatory variable x has a threshold, denoted by T, model (1) will be modified to

for x < z

Po + Pl(x - z) for x > z, logit P(x) =

or equivalently

0 for z G O 1 for z > 0.

logitP(x) = Po + Pl(x - z)Z+(x - T) with I + ( z ) =

Usually the parameters Po, and T are unknown and have to be estimated. But by contrast to Po and fll the parameter T is constrained by min x < T < max x. A threshold value T below min x can

ASSESSING EPIDEMIOLOGICAL STUDIES 343

be estimated only under an additional assumption about the background risk P(O), a situation which is disregarded in the following.

Model (2) can be extended to more than one explanatory variable and also to more than one variable with a threshold. The model for considering p variables where only variable x1 has a threshold (zl) is given by

Estimation procedure

To estimate the parameters Po, B1 and T of model (2), the likelihood function L(P,, B1, T) or more commonly its logarithm LL(/?,, /I1, z) is maximized:

where

0 if subject j does not have the disease 1 if subject j has the disease,

d j = {

xi is the value of the explanatory variable x for subject j and n is the sample size. In almost all available programs (for example BMDP or GLIM) a threshold cannot be

estimated directly. Following an approach of Hawkins' one can calculate the log-likelihood values LL, = LL(P,, PI , z(j)) assuming the threshold has value x(j):

, ( j = 1, . . . , 4, z(j) = x(j)

where x(l) = minxj, x(") = maxxj and

If LLj is the maximum of all these values then the estimate f will be between x(j- ') and x(j+ l). To obtain a more precise estimate z*, the log-likelihood for some values of z* between x(j- ') and x(J+') can be calculated. This approach can be applied easily in GLIMI3 within a macro and can be shortened by using a constrained optimization routine.

Test procedure

The null hypothesis to be tested is whether the variable x bas a threshold z:

H,:T < x(l) = minxj.

H, : z > x(,),

The alternative hypothesis is

The test of a parameter fl (H, : fl = 0 against H 0 # 0) is usually based on the likelihood-ratio statistic R = - 2(lnL(H,) - InL(H,)). Under the null hypothesis, R has a chi-square distribu- tion with one degree of freedom. This procedure is valid if the parameter tested is not con~trained.'~ But this is not the case for parameters with constraints like 7. Feder' reports that the log-likelihood ratio in segmented regression does not appear to be x2 distributed with one degree of freedom. In testing a threshold estimated by a sum of squares the same F-statistics as

344 K. ULM

testing a parameter without constraints are propo~ed,~ . but the statistical tests are approximate rather than exact. Cox" uses a modified value of the likelihood-ratio statistic G2 (the deviance) to describe the performance of various threshold models. The same statistic is also used by McCullagh and Nelder15 for a similar situation. To derive an appropriate distribution for testing the null hypothesis we have to take into account that a one-sided test is considered. In testing a parameter f l without constraints ( H , : B = 0) against an one-sided alternative ( H , : P > 0), H , is to be rejected at a significance level of a if R exceeds c = ~ t , , - ~ = , the (1 - 2a) fractile of the x : distribution, or equivalently by J R > z1 the (1 - a) fractile of the standard normal distribu- tion. To investigate whether this test statistic also holds for testing a parameter with constraints, a simulation study was performed. The application of this method to data from a study in occupational medicine (see Section 3) is verified by the bootstrap resampling technique. First the simulation study is described briefly.

Simulation

To investigate whether R under H o (no threshold) is distributed in the proposed way a simulation study was performed. A random variable x was generated from three different distributions (uniform, normal, log-normal). Based on the value of x, the probability of disease P ( x ) was calculated using model 1 (see below). Then a Bernoulli trial was performed with P(x) the probability of a success. The result, called Y, had two possible outcomes (1 for success and 0 otherwise). One sample contains 1000 pairs (x, y), which were analysed with two different models:

Model 1: logit P(x) = Do + D, x (no threshold) Model 2:

Po was set to - 3.5, and to three different values (0,1,03,0.5) according to the result of the application considered in Section 3. The range for the values of x was set between 3 and 12. One thousand different samples were generated for each combination of the distribution of x and of B,. Figure 2 shows the empirical distribution functions of R under H , for 8, = 05. Similar results were obtained for the other values of 8,. There is little difference between the curves for the three distributions. The fit of model 2 (assuming a threshold value) was better ( R > 0) in about 50 per cent of all simulations, in the other 50 per cent R was equal to zero, and the estimation procedure leads to .r* = min x.

The x: distribution function H , ( R ) is also plotted in Figure 2 and lies below the three curves, indicating that their use for testing H , is too conservative. In Figure 2 also the quasi one-sided x: distribution function F,(R) is plotted:

logit P(x) = Po + Bl(x - z ) l + (x - z) (threshold).

F , ( R ) = f(x)dx + 05, R 2 0,

withf(x) the density of the standard normal distribution function. F , ( R ) shows better agreement with the three empirical curves than H , (R) and confirms the test statistic proposed.

The simulation study was extended to two explanatory variables, both with possible thresh- olds. Here of course the relationship between the explanatory variables is of interest. Even with no relationship the likelihood ratio for testing the overall null hypothesis (both variables without thresholds) does not follow a x2 distribution with 2 degrees of freedom. To test two or more variables with possible thresholds I propose to use different models with and without assuming a threshold for one variable. For an example, see Section 3.

An approximate P per cent confidence interval can be based on the likelihood ratio statistic^.^.'^ A two-sided P per cent confidence interval includes all values of z fulfilling the

loR

ASSESSING EPIDEMIOLOGICAL STUDIES 345

99

98 97 96 95

90

80

60

50

/ i

L

0 1 2 3 4 5 6 0 7 1 .R

Figure 2. Empirical distribution functions of the log-likelihood ratio R under H , for different assumptions about the distribution of x; also plotted are the x : distribution function ( H , ( R ) ) and the quasi one-sided x : distribution

function F , ( R )

Table I. Characteristics of the sample of coal miners who were smokers

Chronic bronchitic reactions No Yes Total

Number 448 375 823 Duration of exposure (years) 24 27 25 Average dust concentration (mg/m3) 8- 1 8.7 8.3

following condition:

3. OCCUPATIONAL DUST EXPOSURE AND CHRONIC BRONCHITIS

As an example I use an epidemiological study17 of the association between occupational dust exposure and chronic bronchitic reactions. About 13,000 employees from different dust-burdened plants in Germany were examined medically twice within an interval of 5 years between 1966 and 1977. The definition of a chronic bronchitic reaction was based on a decision tree depending on an anamnestic questionnaire together with the result of a lung function test. The method for assessing a threshold value is applied to the data for the coal miners since they have the highest dust concentrations, which were also measured more routinely than in the other industries. To reduce the number of covariates, only the subsample is analysed who started their job at the mines (no previous exposure) and who also had been smokers. We have two explanatory variables, namely average dust concentration and duration of exposure. Figure 3 and Table I summarize this set of data.

346 K. ULM

10 l 5

o no bronchitic reactiod with bronchitic reaction

0 10 20 30 40 50 o J

duration of exposure (years)

Figure 3. Dust concentration and duration of exposure for the smoking coal miners

Table 11. Results of four applications of the logistic regression model in assessing threshold values for the sample of smoking coal miners: estimated threshold, coefficient and standard deviation (S.D.)

Model 1 2 3 4

No threshold Threshold only for Threshold only for Threshold for dust concentration duration of exposure both variables

Dust concentration: Coefficient (S. D.) 0.058 (0.02) 0.10 (0.03) 0.058 (002) 0.10 (0.03) Threshold 7.58 7.58 - -

Duration of exposure: Coefficient (S.D.) 0.041 (0.009) 0.041 (0.009) 0.041 (0.009) 0.042 (0009) Threshold - - 8.0 10.12

In L - 552.18 - 550.08 - 552.09 - 550.00 R (compared with

model 1) 4.20 0.18 4.36 -

Four different models were used in analysing the data: Model 1: Model 2: Model 3: Model 4: .Model 1 is without any threshold; models 2 and 3 both have one variable with a threshold; and model 4 incorporates thresholds for both variables. Table I1 gives the results of all four analyses. The comparison of model 1 with model 2 for testing the hypothesis about a possible threshold for dust concentration leads to a log-likelihood ratio R of 4.20, which is statistically significant (p -= 005). The estimate of the threshold value is 3, = 7.58 mg/m3.

logitP,(d, t ) = Po + P l d + j12t

logitP2(d, t ) = P o + Bl(d - zl)Z+(d - t l ) + P z t

logitP,(d, t ) = Po + P l d + P2( t - z,)Z+(t - z2 )

logitP,(d, t ) = P o + B,(d - zl)Z+(d - zl) + P2(t - z 2 ) l + ( t - tZ).

ASSESSING EPIDEMIOLOGICAL STUDIES 347

'

Xt.,95 = 3.84 - 3.

2.

1.

0, . . , , , . , . , . , . , . , . , . . 7.58 , 9 ( m g / m 3 )

3 4 5 6 7 x

/ 95% confidence-interval

Figure 4. Difference between the log-likelihood functions D(t) = 2(ln L(f) - In L( t ) ) for various values of t compared with the estimated value f = 7.58 mg/m'

To test for a possible threshold for 'duration of exposure', model 1 has to be compared with model 3. The difference between both models is fairly small ( R = 0.18) and not statistically significant. Model 4 finally reflects these results again. The difference in the log-likelihood values between model 1 and model 4 is only due to the threshold for dust concentration.

The differences D ( T , ) for various values of T , according to formula (5) are plotted in Figure 4. Hereby model 2 is compared with model 1. The 95 per cent confidence interval for T, is from 3.7 to 9.5 mg/m3.

Bootstrap resampling technique

To verify these results the bootstrap resampling technique can be used. l 8 From the observed set of data a number of samples of the same size n as the original sample are drawn at random with replacement. Each subject has probability l/n of selection. Within each of these samples the threshold value is estimated and the value of the likelihood-ratio statistic is calculated. If the test statistic under H , ( R = 0) is within the 95 per cent range of the observed results of the bootstrap replications, the log-likelihood ratio obtained in analysing the observed data may well be attributed to chance and H, cannot be rejected at a significance level of 5 per cent. This means that in more than 95 per cent of all bootstrap replications a threshold different from the smallest observed value of x must be estimated. Additionally the 95 per cent range of estimated threshold values can be used to approximate the 95 per cent confidence interval.

Using the bootstrap technique with 500 replications, 95 per cent of values for .El are within the range 54-94 (median 7.54 mg/m3) (see Figure 5); only in 5 replications (1 per cent) is R equal to zero and no threshold can be estimated.

There is good agreement between both methods. The results of the bootstrap replications are clustered between 6.5 and 8 mg/m3 and around the estimated threshold at 7.58 mg/m3. The confidence interval obtained by bootstrap resampling is somewhat smaller than the interval based on the likelihood-ratio statistic.

In addition to the logistic model, probit and complementary log-log models have been applied. In both models the estimated threshold value was 7-55 mg/m3. The values of the log-likelihood function for these two models (- 550.15 and - 55085) are also very close to that of the logistic model (- 550.08). The results obtained by the logistic model are therefore confirmed by these other two models.

348

60. 50 . 40 . 30 . 20 . 10 -

K. ULM

median 7.54

t 0-.

Figure 5. Sample distribution of the estimated threshold values in 500 bootstrap replications; the median is at Q = 7.54 mg/m3 with an approximate 95 per cent confidence interval from 5.6 to 9.8 mg/m3

4. DISCUSSION

In this paper a method for assessing a threshold value within the framework of the logistic model is described. An important example is taken from occupational medicine, where for many compounds threshold limit values are specified. But the assumption of a threshold is not restricted to the field of occupational medicine. For example, considering the relationship between serum cholesterol and blood pressure as risk factors for coronary heart disease, a threshold effect can also be assumed.

Demonstration of a dose-response relationship is an important criterion to establish causality between an explanatory variable and a specified response. Therefore the possibility of a threshold should nearly always be considered. One possible exception to this requirement may occur when the response is cancer. Programs to analyse studies of this kind should therefore be extended to sllow threshold values.

The test of the hypothesis about the existence of a threshold value can be based on the likelihood-ratio statistic R. According to the one-sided alternative, the null hypothesis is to be rejected at a significance level of a if R exceeds x: , the (1 - 2a) fractile of the xz distribution with 1 d.f. A simulation study supported this result. Another method proposed to test the corresponding hypothesis is the bootstrap resampling technique. Both methods were applied to the same study and yield nearly identical results. The only difference is that the confidence interval based on the likelihood-ratio statistic is somewhat larger than that obtained by the bootstrap technique. For practical purposes, to test hypotheses about threshold values and to calculate confidence intervals, it seems easier to use the likelihood-ratio statistic than the bootstrap technique.

The main problem in assessing a threshold value may be the dependence of the model selected for the analysis. In another application of this method the data from an animal study have been analysed with various models.’ 9 ,20 The animals were grouped into eight exposure categories. The logistic model without a threshold leads to a deviance of 11.12 with 6 d.f., which could be improved to 6.32 with 5 d.f. by assuming a threshold value. The corresponding log-likelihood ratio of R = 4.91 is statistically significant (p < 0.05). Nearly the same result is obtained by the probit model. On the other hand the complementary log-log model, even without a threshold, yields a much better fit with a deviance of 3.45 with 6 d.f. The difference from the logistic model is striking. With the complementary log-log model, no threshold could be estimated. A slightly better fit was obtained by Prentice” in using a three-parameter power logistic model without a

ASSESSING EPIDEMIOLOGICAL STUDIES 349

threshold, which gives a deviance of 3.24 with 5 d.f. Overall it is somewhat uncertain whether a threshold exists for this set of data. The problem is that these models are suitable for describing certain phenomena but not for explaining them.

Therefore in assessing a threshold value, more than one model should be considered. As a preliminary step in the analysis of epidemiological studies I suggest using a non-parametric approach, for example isotonic regression.22 In connection with a scatterplot of the distribution of the data, the results of some parametric models can be confirmed. But again, owing to the importance of a dose-response relationship, the possibility of a threshold should always be taken into account in the parametric models.

To assess the TLV for dust concentration, results for the non-smoking coal miners and for all the other dust-burdened plants must be considered along with results published in the literature.

ACKNOWLEDGEMENTS

The author thanks Professors Norman Breslow, Richard H. Jonas, Charles C. Brown and Gerhard Arminger for their valuable comments on an earlier draft of this paper. The referees also gave helpful advice to improve the paper. This work was supported in part by a grant from the German Research Foundation.

REFERENCES

1. Woitowitz, H.-J. ‘Maximum concentrations at the workplace in the Federal Republic of Germany’,

2. Armitage, P. ‘The assessment of low-dose carcinogenicity’, Biometrics, Supplement: Current Topics in

3. Brown, C. C. ‘Mathematical aspects of dose-response studies in carcinogenesis- the concept of

4. Gallant, A. R. and Fuller, W. A. ‘Fitting segmented polynomial regression models whose join point have

5. Feder, P. I. ‘The log likelihood ratio in segmented regression’, Annals ofstatistics, 3, 84-97 (1975). 6. Schulze, U. ‘A method of estimation of change points in multiphasic growth models’, Biometrical

7. Jones, R. H. and Molitoris, B. A. ‘A statistical method for determining the breakpoint of two lines’,

8. Edler, L. and Berger, J. ‘Computational methods to determine a break point in linear regression’, ED V in

9. Freeman, J. M. ‘An unknown change point and goodness of fit’, The Statistician, 35, 335-344 (1986). 10. Cox, Ch. ‘Threshold dose-response models in toxicology’, Biometrics, 43, 51 1-523 (1987). 1 1 . Shaban, S. A. ‘Change point problem and two-phase regression: an annotated bibliography’, Interna-

12. Hawkins, D. M. ‘Testing a sequence of observations for a shift in location’, Journal ,of the American

13. Baker, R. J. and Nelder, J. A. The GLlM System, Release 3, Numerical Algorithms Group, Oxford, 1978. 14. Kalbfleisch, J. and Prentice, R. The Statistical Analysis ofFailure Time Data, Wiley, New York, 1980. 15. McCullagh, P. and Nelder, F. R. S . Generalized Linear Models, Chapman and Hall, London, 1988. 16. Cox, D. R. and Hinkley, D. V. Theoretical Statistics, Chapman and Hall, London, 1974. 17. DFG-Forschungsbericht. Chronische Bronchitis, Teil 2, Harald Boldt, Boppard, 1981. 18. Wahrendorf, J., Becker, H. and Brown, C. C. ‘Bootstrap comparison of non-nested generalized linear

models: applications in survival analysis and epidemiology’, Applied Statistics, 36, 72-8 1 (1987). 19. Ulm, K. ‘On the estimation of threshold values (correspondence)’, Biometrics, 45, 1324-1326 (1989). 20. Cox, Ch. ‘On the estimation of threshold values (reply)’, Biometrics, 45, 1327-1328 (1989). 21. Prentice, R. L. ‘A generalization of the probit and Logit methods for dose-response curves’, Biometrics,

22. Ulm, K. ‘Nonparametric analysis of dose-response relations in epidemiology’, Mathematical Modelling,

American Journal of Industrial Medicine, 14, 223-229 (1988).

Biostatistics and Epidemiology, 37, 119-129 (1982).

thresholds’, Oncology, 33, 62-65 (1976).

to be estimated‘, Journal of the American Statistical Association, 68, 144-147 (1973).

Journal, 26, 495-504 (1984).

Analytical Biochemistry, 141, 287-290 (1984).

Medizin und Biologie, 16, 128-134 (1985).

tional Statistical Reoiew, 48, 83-93 (1980).

Statistical Association, 72, 180-186 (1977).

32, 761-768 (1976).

7, 777-783 (1986).