J Clin Epidemiol Vol. 42. No. 2, pp. 119-126, 1989 Prmted in Great Bntain. All rights reserved

0895-4356:89 $3.00 + 0.00 CopyrIght r‘ 1989 Pergamon Press plc

EFFECTS OF FRAMING AND LEVEL OF PROBABILITY ON PATIENTS’ PREFERENCES FOR CANCER CHEMOTHERAPY

ANNETTE M. O’CONNOR

University of Ottawa. Faculty of Health Sciences, School of Nursing. 451 Smyth Road, Ottawa, Ontario. Canada KIH 8M5

(Receiwd in ret,ired f&-m I9 July 1988)

Abstract-Although most cliniclam agree that patients should be informed about treatment alternatives, little is known about the way patients perceive probabilistic information about treatment outcomes and hou it influences the choices they make. The purpose of this study was to examine the influence of level and framing of probability on preferences for cancer treatment alternatives in which tradeoffs between quantity and quality of life are made. 129 healthy volunteers and 154 cancer patients indicated their preferences for a toxic treatment over a non-toxic treatment at varying survival probabilities. Subjects responded to questions in one of three randomly assigned conditions: (1) a positive frame in which the probability of survival was given; (2) a negative frame in which the probability of dying was given; and (3) a mixed frame in which the probability of surviving and dying were both given. The cancer patients’ preferences for the more effective toxic treatment was significantly stronger than the healthy volunteers. Both groups were significantly influenced by the level of probability that was presented. Preferences for the toxic treatment were weaker when the chance of survival dropped below 50%. This weakening preference below 50~~ survival was enhanced for subjects who responded in the negative frame. A negative frame or probability level below 0.5 would seem to stimulate a “dying mode” type of value system in which quality of life becomes more salient in decision making than quantity of life. The implications in eliciting informed consent from patients are discussed.

Decision making Values Choice behavlour

INTRODUCTION

During the process of treatment decision making, patients usually receive information about treatment alternatives and the likely out- comes of treatment. It is encouraging to note that the majority of patients not only want the opportunity to give informed consent but also to become actively involved in the decision making process [l. 21. However, little is known about the way patients perceive probabilistic information about treatment alternatives and how it influences the choices they make. This problem is of crucial significance when illness is life threatening, treatment outcomes are uncer- tain, and decisions involve making tradeoffs between quantity and quality of life.

Treatment choices involving uncertain out- comes are known to be influenced by the way probabilities of outcomes are framed [3%5] and the feaef of probability [6-81. However, several

Oncology

questions remain about the nature of these effects and the points at which they occur. Firstly, how do preferences shift across the entire spectrum of probability? Most studies of treatment decisions have focussed on the extreme ends or have sampled only a few levels. It is therefore difficult to judge the points at which the shifts actually occur. Secondly, to what extent are cancer patients who actually face life-threatening illness influenced by level and framing of probability? Previous studies have elicited preferences from healthy volun- teers or chronically ill patients and inferred that cancer patients would respond in the same way. Cancer patients’ preferences for toxic treatment differ from those of healthy volunteers or health professionals who are responding hypothetically [9-l 11. Their responses to framing and levels may also differ. Lastly, to what extent do level and framing of probability interact? This ques- tion has not been studied. In view of the fact

119

120 ANNETI-E M. O'CONNOR

that clinicians who present probabilistic infor- mation to patients use both a level and frame to present the information, it would be important to know how they work together to influence preferences.

METHODS

The general plan of the study was to develop a treatment choice questionnaire and test it for probability and framing effects with two groups of subjects: healthy volunteers and cancer patients. The subjects were asked to choose between two hypothetical treatments (A or B). Treatment A produced mild but persistant nausea, diarrhea and fatigue; treatment B had no side effects. Subjects were asked to choose between the two treatments when the l-year probabilities of survival for both were equal. When the less toxic treatment B was invariably chosen, the probability level for A remained the same but was lowered for B by 1% decrements until subjects switched their preference to A. Preferences were assessed in this manner for each of the following levels of probability: 100, 90, 75, 50, 40, 25 and 10% survival.

Subjects responded to the questions in one of three randomly assigned framing conditions: (a) a positive frame in which the probability of survival was given, (b) a negative frame in which the probability of dying was given, or (c) a mixed frame in which the probability of surviving and dying were both given. Therefore, preferences for toxic treatment A were elicited at each of the following levels of probability: (a) positive frame-100, 90, 75, 50,40, 25, 10% survival rates; (b) negative frame-O, 10, 25, 50, 60, 75, 90% death rates, and (c) mixed frame- 100/O, 90/10, 75125, SO/SO, 40160, 25175 and 10/90% survival/death rates.

The elicitation techniques were tested for reliability and position order effects. Interrater and test-retest reliability had been established previously with similar questions [7, 141 but was confirmed again with the specific questions used here. Twenty volunteers were tested by one interviewer (ET) and retested by the other inter- viewer (LA). There were no statically significant differences between test and retest scores and correlation coefficients between scores ranged from 0.88 to 0.95.

A preference score for toxic treatment A was calculated by dividing the probability at which patients switched preferences from B to A by the baseline probability at which the patient started. For example, a subject whose preferences were assessed at 100% baseline survival for both treatments and switched preferences from B to A when survival for B was 90% would have a preference score for A of 90/100 or 0.90. If the subject switched preferences from B to A at 45% when baseline survival was 50%, the preference score for A would also be 0.90 (45/50 = 0.90).

The position order of the mixed frames was also tested. 40 volunteers in the main sample who were randomly assigned to respond in the positive/negative mixed frame were compared to 40 volunteers who were randomly assigned to a negative/positive mixed frame. There were no statistically significant differences between the preference scores of the two groups, confirming our previous results with nursing students [5].

If subjects’ scores remained stable across the entire spectrum of probability, the data would support the substitution axiom of expected util- ity theory. This axiom states that a utility or preference elicited at one level of probability can be substituted for decisions at other levels of probability [12] and is invoked each time a utility is used in a decision tree. If weights assigned to probabilities are subproportional, the preference profile of patients supports Kahneman and Tversky’s Prospect Theory [12]. In a previous study of cancer patients’ prefer- ences for chemotherapy. O’Connor et al. [13] found preferences were stable when survival rates for toxic treatments ranged from 60 to 100%; stability below 60% was unknown. were found.

The last position order test focussed on the order with which probability levels were presented. We tested the effects with patients and volunteers in our main sample. Subjects were randomly assigned either: (1) starting at one end of the probability spectrum and going to the opposite end; or (2) starting at the opposite end of the spectrum and moving in the reverse direction to subjects in (1). Thus patients in the positive frame would move from 100 to 10% or 10 to 100%. Similarly, subjects in the negative frame would move from 0 to 90% or 90 to 0% and subjects in the mixed frame would move from 100/O to 10/900/, or IO/90 to 100/O”/, . There were no differences in scores between subjects in different position orders and no interaction with framing.

The tests of the elicitation techniques indi- cated that the measures were reasonably re- liable, and no significant position order effects

Effects of Framing 12’

Sample

Healthy volunteers were obtained from the faculty, students, and staff of the Health Sci- ences Faculty at the University of Ottawa and the regional cancer clinic. Neighbours of the volunteers were also solicited for their par- ticipation. Interviews lasted for 1 &I 5 min in an office or quiet area of the health sciences build- ing, clinic, or in the volunteer’s home. None of the volunteers had a life-threatening illness.

Patients were selected from an outpatient regional cancer treatment center. The following inclusion criteria were used. Each patient must have been (a) diagnosed as having cancer, (b) ambulatory, (c) able to speak English or French, and (d) suitable for evaluating hypo- thetical treatments. This last criterion was a judgement call made by clinicians to exclude patients who were unable to respond in an interview due to severe physical deterioration, communication problems, or severe psycho- logical distress or depression. This criterion excluded many patients in the terminal phase of illness.

The clinic staff nurses screened all patients listed for scheduled appointments according to the inclusion criteria and the resulting list of suitable patients was presented to the two inter- viewers. Interviews took place before or follow- ing regularly scheduled clinic visits. Patients were attending the clinics for radiation therapy, chemotherapy, or follow-up visits. Those who consented were interviewed for l&l 5 min in an office or quiet area of the clinic.

The patient interviews took place during a 3 week period. There were 512 patients with scheduled appointments. 20% of the patients (101/S 12) were considered inappropriate to interview due to emotional, physical or com- munication problems, leaving 41 1 potential subjects. The interviewers approached eligible subjects according to their order on the sched- uled appointment list. When a patient was not available or was missed, the next patient on the list was selected. Several patients were missed because of multiple appointments at the same time and patients leaving while the interviewer was interviewing another patient. It was difficult to predict whether patients would have time to speak with the interviewer before or after the scheduled treatment. Due to the high volume of subjects (411) and the low volume of data collectors (2), interviewers were only able to approach 175 subjects. Of these, 18 patients declined the interview due to lack of time or

interest and 157 agreed to participate. Par- ticipants were randomized into one of the three framing conditions. Three of the 157 patients interviewed chose not to complete the interview. Thus the available sampling procedures reduced the external validity of the results. However., the internal validity of the results was not jeopardized. The random allocation of available subjects to one of three framing conditions proceeded without difficulty.

RESULTS

Group churacteristics and preferences

A total of 286 people were interviewed for this study: 129 volunteers and 154 cancer patients. The majority of the volunteers were health professionals (70%) and female (67%). The volunteers had a mean age of 35 yr with a range of 15--68. The 154 cancer patients included 87 (57%) females and 67 (44%) males. Their mean age was 54 years (range 20-85). Half the patients attended the radio- therapy clinic for treatment or follow-up and the other half attended the chemotherapy clinic.

Group preferences for the toxic treatment A are presented in Table 1. Mean preference scores for both groups were high; those of patients were significantly (p < 0.05) higher compared to the healthy volunteers. The distri- bution of scores also differed significantly (p < 0.05). A higher proportion of patients had scores of 0.99 compared to volunteers. A score of 0.99 indicated that subjects preferred to non-toxic treatment B when survival for both A and B treatments were equal, but preferred the toxic treatment A whenever survival for B was lower than A. 5 1% of patients were unwilling to trade off even 1% of survival to avoid the more toxic treatment A compared to 16% of healthy subjects. This pattern continued across the spec- trum of probability (Table 2) where 40% of patients and 6% of volunteers did not make tradeoffs.

Ir$uence qf probability level

Data were analysed using a two factor (frame and probability level) repeated measures ANOVA. F values were calculated using normal error for the probability level effect and subject (frame) error for the framing effect. When the raw preference scores were used, framing was not statistically significant and level of proba- bility was. However, the slopes of the preference curves for each of the frames appeared differ-

122 ANNETTE M. O'CONNOR

Table 1. Preference scores by group

Statistic Patients Volunteers

Sample size 154 128 Mean preference score* 0.91 0.80 Standard deviation 0.17 0.21 Frequency distribution of scores? n (%) n (%)

0.99 78 (51%) 0.750.98

20 (16%) 67 (43.5%) 79 (62%)

0.50-0.74 5 (3%) 22 (17%) 0.25-0.49 0.00-0.24 -4 (2.5%)

3 (2%) 4 (3%)

*The mean differences in patients’ and volunteers’ preference scores were statistically significantly different using the r-test (t = 4.89, p = 0.01).

tThe proportion of volunteers and patients in each of the categories were statistically significantly different (x2 = 44.82, p = 0.0001).

Table 2. Absolute tradeoff shifts with declining probability of survival by group

Patients Volunteers Pattern n = 154 n = 128

Never tradeoff 40% 6% Tradeoff decreases 23% 40% Tradeoff unchanged 21% 25% Tradeoff increases 16% 29%

x2 = 17.3, p = 0.0006.

ent. Therefore, the raw preference scores were transformed into deviation scores from 100% baseline survival preference scores. This trans- formation was theoretically sensible since the mean deviation scores would be expected to be zero using the substitution axiom of expected utility theory. Using deviation scores, both framing, probability level and their interaction were statistically significant (p < 0.05). The re- sults are presented in Table 3. The levels at which the differences occurred were determined using Scheffe’s multiple comparison test [15]. The influence of probability level on preference scores is illustrated in Fig. 1. Patients’ prefer-

ence scores for toxic treatment remained signifi- cantly (p < 0.05) higher than the volunteers at all levels of probability. However, both groups responded in a similar pattern to changes in probability levels. When baseline survival was greater than 50%, preferences for toxic treat- ment A remained stable. There were no statisti-

0.2

0.1 1 I I II 1 ““‘I

0 10 20 30 40 SO 60 70 60 SO 100 Probability of survival 1%)

Fig. 1. Preference scores of patients and healthy subjects across the spectrum of probability. Error bars represent the standard error of the mean. Differences between groups occurred at all levels of probability (p < 0.05). Differences within subjects occurred at probability levels below 50%

survival (p c 0.05).

Table 3. Effects of probability level and framing on preference scores for toxic treatment

Source DF Sum of squares F Value p Value

Patients Frame 2 1.00 3.68 0.03 Subject (Frame) 151 20.54 Prob Level 5 6.90 52.14 0.0001 Prob x Frame 12 0.70 2.64 0.0036 Error 755 19.97

volunteera Frame 2 1.04 4.78 0.01 Subject (Frame) 106 11.57 Prob Level 4 7.92 54.00 0.0001 Prob x Frame 8 0.52 1.77 0.08 Error 424 15.55

NOW: F values have been calculated using subject (frame) error for the framing effect and normal error for the probability level effect.

Effects of Framing 123

01 r

0.0

5 -0.1 - z B $ -02-

E a -0.3-

z

.-

0 Poshe frame . Negative frame h Mixed frame

-05’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ 0 10 20 30 40 50 60 70 90 90 100

Probabhty of survival (%I

Fig. 2. Patients’ mean deviation in preference scores. A zero deviation means that, on average. preference scores at a given probability did not differ from the score at 100% baseline survival. A negative deviative means that on aver- age, subjects’ preference scores were lower than those at 100 percent baseline survival. Scores in the negative frame differed significantly from those in the positive or mixed

frames (p < 0.05).

tally significant differences in preference scores at 100, 90. 75 and 50% baseline survival for either group. Preference scores ranged from 0.91 to 0.88 for the patients and 0.80 to 0.77 for volunteers. The preference scores below 50% survival were significantly lower than those above 50%. For baseline levels ranging from 40 to 10% survival, preference scores dropped from 0.85 to 0.61 for patients and 0.73 to 0.46 for volunteers.

Framing irzfiuences

Framing effects are illustrated in Figs 2 and 3. For volunteers, a statistically signifi- cant framing effect occurred at 50% survival (p = 0.0001). A marginal effect (p = 0.09)

0. 1

00 I- /Y+-- /- .- .-

6 . Negative frame

f -0 3 h Mwed frame

-04

-0 5

iL / . I 1 1 I 1 1 ’ ’ 1

0 10 20 30 40 50 60 70 80 90 100

Probabhty of survival (%I

Fig. 3. Volunteers’ mean deviation m preference scores. A zero devration means that, on average, preference scores at a given probability did not differ from the score at 100% baseline survival. A negative deviation means that on average, subjects’ preference scores were lower than those at 100% baseline survival. Scores in the negative frame differed significantly from those in the positive or mixed

frames (p i 0.05).

occurred at 10% survival. For patients, framing was statistically significant at 10% survival (p = 0.01) with a marginal effect at 25% survival (p = 0.12). Subjects responding in the negative frame had significantly lower prefer- ence scores than those responding in the positive or mixed frames.

There was a significant interaction between framing and level of probability for patients (p = 0.003) with a marginal effect for volunteers (p = 0.07). At lower levels of probability, the decline in patient and volunteer preference scores was enhanced in the negative frame compared to other frames.

DISCUSSION We found that cancer patients’ preferences

for toxic treatment differed from those of healthy subjects and preferences for both groups were influenced by level and framing or proba- bility at levels below 50%. The higher prefer- ence scores of cancer patients confirms previous findings reported by Sackett and Torrance [9], Boyd et al. [lo] and O’Connor et al. [ 11, 7, 131. For example, the mean preference score of 54 cancer patients about to receive chemotherapy for the first time [ 131 was 0.96 compared to 0.9 I in this study. The mean preference score of 140 nursing students [7] ranged from 0.77 to 0.81 compared to the volunteers’ mean score of 0.80 in this study.

The higher patient preference scores may mean that they are less cavalier about trading off chance of survival to avoid treatment tox- icity compared to healthy volunteers who are only responding hypothetically. However, dif- ferences in risk aversion cannot account for all of the disparity between healthy subjects and patients. In previous work [I 11, preferences elic- ited via riskless measures such as linear ana- logue scales also have been higher for patients. An alternative explanation may be that patients’ values or utilities have shifted in the process of adapting to their illness. They have changed their reference points upon which to base their judgements. A treatment with unpleasant side effects which was considered undesirable when patients were healthy is now considered more desirable now that they have cancer, particu- larly since the treatment is linked to improving their chances of survival. Some may view this shift in values as “cognitive dissonance reduc- tion” in that patients who have had the treat- ment make their values consonant with their decisions. A more appropriate label may be

124 ANNETTE M. O’CONNOR

“adaptation”, for without a shift in values, more patients may refuse life-saving treatment. These conjectures need to be tested in a prospec- tive study of subjects. How do preferences shift with change in health state and to what extent do they predict future decisions to refuse, delay, drop out of or follow toxic but lifesaving treat- ment regimens? Another question requiring study is the influence of treatment intent on preferences. Preferences may vary depending on whether patients are on active or palliative treatment. Preferences of some subgroups (cured, active treatment, palliative treatment) may be similar to those of healthy volunteers.

The preference stability at higher levels of probability confirms our previous work [ 131 and supports the substitution axiom of expected utility theory. A utility elicited using the proba- bility equivalence technique at 100% baseline survival can be used in a decision tree where probability levels range from 50 to 100%.

The substitution axiom is not supported at lower probability levels. The shift in preferences below 50% survival can be interpreted in at least two ways. Firstly, preferences may actually be stable, but subjects may not be able to respond consistently to variations in probability using the elicitation technique. They may be poorly calibrated when trying to keep trade- offs proportionate at all probability levels (e.g. adjusting tradeoffs from 20% at 100% baseline to 10% at 50% baseline to 5% at 25% baseline). This may have been the case in the 21% of patients and 25% of volunteers who traded off the same absolute amount of survival points across the entire spectrum (Table 2). They may or may not have realized that a tradeoff of 5 survival points when baseline sur- vival was 50% (0.10 = 0.90 preference score) was not the same as a tradeoff of 5 survival points at 10% baseline (0.50 = 0.50 preference score). However, 16% of patients and 29% of volunteers made absolute increases in tradeoffs as probability levels dropped (e.g. 5 survival points at 90% baseline and 15 survival points at 40% baseline). These subjects probably realised that they were increasing tradeoffs as survival rates dropped.

The alternative interpretation of the prefer- ence patterns is that preference for the toxic treatment decreased as the probability of sur- vival dropped below 50%. This interpretation was supported by comments patients made when levels went below 50%. Patients would state “what’s the use, the odds are against you”

or “you are probably going to die anyway, so why bother making yourself sick.” Both these comments indicate that individuals may have adopted a “dying mode” type value system in which quality of remaining life, rather than quantity of life, is a more salient factor in decision making. This type of patient may give up hope of survival and opt for treatment affording better quality of life. One of the clinicians who participated in the study also confirmed this view stating: “I never tell patients they have less than a 50: 50 chance because then they give up hope”. If one defines hope as the perceived likelihood of achieving a desired goal [16], the shift in tradeoffs may be explained. At lower probabilities of survival, people may literally “give up hope” (trading off the chance of survival) to avoid toxic treatment.

The preference pattern supports the sub- proportionality axiom of Prospect Theory. Kahneman and Tversky observed that most people choose the monetary prospect with the highest probability when probability is high, but when probability is low, they switch preferences to the monetary prospect with the greatest financial gain. Thus, when hope is high, the option affording the greatest hope is chosen. As hope declines, the option affording the greatest gain is chosen. With monetary prospects, the gain is financial, with health prospects, the gain is quality of life.

Tradeoffs are also influenced by framing. As we found in previous work with nursing students [5], subjects responding in the nega- tive frame considered the toxic treatment less desirable than those responding in the positive or mixed frames. The negative frame may have imposed a dying mode type value system in which quality of life is more important than quantity of life. The word “survive” is absent in the negative frame and present in the other two frames. This word may represent hope. In its absence, subjects may literally give up hope (trading off chance of survival) to a greater extent than in its presence. This effect occurs only when probability levels are low.

It is also interesting to note that the responses to levels and frames varies within the patient population. Some patients never tradeoff re- gardless of the probability level and frame. Many stated anecdotally that they would prefer a treatment without side effects, but would always choose the treatment affording the greatest chance of survival. For others, tradeoffs were made at high levels of probability, how-

Effects of Framing 125

ever, at lower levels, the tradeoffs were tapered or stopped. Subjects would state: “there’s not much chance, so I’m keeping the chances I’ve got [with the toxic treatment]“. The last and predominant pattern was one of increasing tradeoffs, either by maintaining or increasing the absolute tradeoffs made at higher levels when levels dropped. It is the latter group who are most troublesome, for their preferences are sensitive to probability levels which are often the same or higher than actual survival rates of cancer patients.

If these preference sensitivities occur with actual cancer treatment decisions, important ethical questions are raised in eliciting informed consent. How are clinicians to maintain the delicate balance between conveying accurate information and preserving hope? Two approaches are possible. Clinicians may take a laissez-fuire approach by maintaining that framing and probability levels are contexts which can and should change preferences. From this point of view, assessment of patients prefer- ences, based on reliable elicitation techniques like those used in this study, provides an appro- priate route for patients to participate in decision making affecting their care.

An alterative interventionist approach is to view framing and probability effects as cognitive illusions which should be corrected. Preferences at 50% chance of survival should be equal to those at a 50% chance of dying. Similarly. why should a treatment which improves baseline survival by 15% be acceptable to patients when survival is greater than 50% but not less than 50%? With this interventionist view, methods which help patients resolve preference sensi- tivities need to be developed. One approach involves presenting patients with alternatives using several frames and several levels of proba- bility, pointing out the pattern of increasing tradeoffs, and helping patients interpret and resolve these preference changes. One of the problems in using this approach is determining which preference is the illusion and which one is real. A second approach is to avoid placing patients in a dying mode unless they truly are dying. In this scenario, information is presented in a positive or mixed frame and preferences are elicited at high and low levels of probability. When preferences are sensitive to non-extreme probability levels below 50%, those above 50% are selected.

Appropriate methods of informing patients about treatment options require empirical eval-

uation. Further research is also needed to exam- ine the extent to which patients’ preferences correspond to actual decisions about accepting and adhering to treatment.

In conclusion, this research has verified that cancer patients’ preferences are influenced by frames and levels of probability. Our under- standing of how preferences are influenced and at what points they occur has also been improved. The “dying mode” hypothesis has been presented as a possible explanation of these effects. Further research may improve our knowledge about how different groups of patients perceive and use information about treatment alternatives and how perceptions correspond to actual decision behaviour. How- ever, the answers about what to do when fram- ing and probability effects occur cannot be established by empirical methods alone. They are also matters for the philosopher, the ethicist, and ultimately, the individual clinician eliciting patients’ preferences.

Ackno,l/edKL,menrs~This study was funded by the Registered Nurses Association of Ontario Foundation. Thanks arc ex- tended to the followmg persons for their advice. support, and assistance: Dr James Till (Ontario Cancer Institute), Elaine Thiel, Lawrence Durham, Lomse Alarie, Suzie Joanisse. Dr Marlan McGee, (University of Ottawa). Professor Ann Mohide (McMaster University), Dr Hilary Llewellyn-Thomas (Ontario Cancer Institute). patients of the Ontario Cancer Foundation Ottawa Civic Division, Cora MacMillan, Diane Panetta. Marie Dafoe. Dr Danielle Perrault. Dr Gordon Catton, and nursing and medical staff of the Ontario Cancer Foundation Ottawa Clinic (Civic Division).

I.

2.

3.

4.

5.

6.

7.

REFERENCES

Degner LF, Russell CA. Preferences for treatment control among adults with cancer. Res Nurs Hlth (In press). Dennis KE. Dimensions of client control. Nurs Res 1987; 36: 151-155. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science 1981; 211: 453458. McNeil BJ, Pauker SG. Sox HC, Tversky A. On the elicitation of preferences for alternative therapies. N Engl J Med 1982; 306: 1259-1262. O’Connor A, Boyd NF, Tritchler D, Kriukov Y, Sutherland H, Till J. Elicitating preferences for alter- native cancer drug treatments: the influence of fram- ing, medium and rater variables. Med Dee Making 1985; 5: 453463. Eraker SA, Sox HC. Assessment of patients’ prefer- ences for therapeutic outcomes. Med Dee Making 1981; ?,: 29-39. O’Connor A, Boyd NF, Till JE. Influence of elicitation technique, position order, and test-retest error on preferences for alternative cancer drug therapy. Nurs- ing research: science for quality care. Proc 10th Nat. Nursing Research Conf. University of Toronto; 1985: 49-58.

126 A~tozrrx M.

8.

9.

10.

11.

12.

Hershey JC, Kunreuther HC, Schoemaker PJH. Sources of bias in assessment procedures for utility functions. Mgmt Sei 1982; 28: 936953. Sackett DL, Torrance GW. The utility of different health states as perceived by the general public. J Chron Dis 1978; 31: 697-704. Boyd NF, Sutherland HJ, Cummings BJ. The selection of primary therapy for patients with cancer of the rectum. Med Dee Making 1982; 2: 254 (Abstr.). O’Connor A. Eliciting preferences for alterative therapies in oncology: sources of intra-rater variation. Thesis Toronto, University of Toronto; 1986. Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 1979; 47: 263-291.

.

OTONNOR

13.

14.

15.

16.

O’Connor A, Boyd N, Warde P, Stolbach L, Till J. Eliciting preferences for alternative drug therapies in oncology: Influence of treatment outcome descrip- tion, elicitation technique and treatment experience on preference. J Chroa Dis 1987: 401 811-818. Llewellyn-Thomas HA, Sutherland HJ, Tibshirani R, Ciampi A, Till JE, Boyd NF. Describing health states: methodologic issues in obtaining values for health states. Med Care 1984; 22: 543-552. Kleinbaum DG, Kupper LL. Applied regression analysis and other multivariate methods. North Scituate, Mass.: Duxbury, 1978, p. 271. Stotland E. The Psychology of Hope. California: Jossey-Bass; 1969.