Embed Size (px)
Citation preview
This article was downloaded by: [North Dakota State University]On: 10 October 2014, At: 19:19Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Health Communication:International PerspectivesPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/uhcm20
Alternate Methods of FramingInformation About Medication SideEffects: Incremental Risk Versus TotalRisk of OccurrenceBrian J. Zikmund-Fisher a b c , Angela Fagerlin a b c , Todd R. Robertsc , Holly A. Derry c & Peter A. Ubel a b c da VA Center for Clinical Management Research , VA Ann ArborHealthcare System , Ann Arbor, Michigan, USAb Division of General Internal Medicine , University of Michigan , AnnArbor, Michigan, USAc Center for Behavioral and Decision Sciences in Medicine , AnnArbor, Michigan, USAd Department of Psychology , University of Michigan , Ann Arbor,Michigan, USAPublished online: 25 Feb 2008.
To cite this article: Brian J. Zikmund-Fisher , Angela Fagerlin , Todd R. Roberts , Holly A. Derry& Peter A. Ubel (2008) Alternate Methods of Framing Information About Medication Side Effects:Incremental Risk Versus Total Risk of Occurrence, Journal of Health Communication: InternationalPerspectives, 13:2, 107-124, DOI: 10.1080/10810730701854011
To link to this article: http://dx.doi.org/10.1080/10810730701854011
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Alternate Methods of Framing Information AboutMedication Side Effects: Incremental Risk Versus
Total Risk of Occurrence
BRIAN J. ZIKMUND-FISHER ANDANGELA FAGERLIN
VA Center for Clinical Management Research, VA Ann ArborHealthcare System, Ann Arbor, Michigan, USADivision of General Internal Medicine, University of Michigan, Ann Arbor,Michigan, USACenter for Behavioral and Decision Sciences in Medicine, Ann Arbor,Michigan, USA
TODD R. ROBERTS AND HOLLY A. DERRY
Center for Behavioral and Decision Sciences in Medicine, Ann Arbor,Michigan, USA
PETER A. UBEL
VA Center for Clinical Management Research, VA Ann ArborHealthcare System, Ann Arbor, Michigan, USADivision of General Internal Medicine, University of Michigan, Ann Arbor,Michigan, USACenter for Behavioral and Decision Sciences in Medicine, Ann Arbor,Michigan, USADepartment of Psychology, University of Michigan, Ann Arbor,Michigan, USA
Portions of this research were presented at the Annual Meeting of the Society for MedicalDecision Making, Atlanta, Georgia, October 18, 2004. Financial support for this study wasprovided by grants from the National Institutes for Health (R01 CA87595 and P50CA101451). Dr. Zikmund-Fisher is supported by a career development award from theAmerican Cancer Society (MRSG-06-130-01-CPPB), and Dr. Fagerlin was supported by anMREP early career award from the U.S. Department of Veterans Affairs. During part of thisresearch, Dr. Zikmund-Fisher was supported by an HSR&D Post-Doctoral Fellowship fromthe U.S. Department of Veterans Affairs and Dr. Ubel was recipient of a Presidential EarlyCareer Award for Scientists and Engineers (PECASE). The funding agreements ensured theauthors’ independence in designing the study, interpreting the data, and publishing the report.The authors acknowledge the excellent research assistance of Jonathan Kulpa.
Address correspondence to Brian J. Zikmund-Fisher, Center for Behavioral and DecisionSciences in Medicine, 300 North Ingalls Building, Rm. 7C27, Ann Arbor, MI 48109-5429,USA. E-mail: [email protected].
Journal of Health Communication, 13:107–124, 2008Copyright # Taylor & Francis Group, LLCISSN: 1081-0730 print/1087-0415 onlineDOI: 10.1080/10810730701854011
107
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Communications of treatment risk, such as medication package inserts, commonlyreport total rates of adverse reactions (e.g., 4% get heartburn with placebo, 9%with medication). This approach, however, requires mental arithmetic to distinguishthe incremental risk caused by medication (here, 5%) from the total post-treatmentrisk. In two Internet-administered survey experiments (N ¼ 2,012 and 1,393), wetested whether explicitly reporting the incremental risk and framing it as the‘‘additional risk’’ of complications influenced people’s impressions of adverse eventrisks. Study 1 compared side-by-side displays of total risks against sequentialpresentations that highlighted the incremental risk, using both text and graphicalformats. Results showed that incremental risk formats significantly loweredparticipants’ worry about complications and reduced biases caused by varying therisk denominator. Study 2 unpacked this factor and showed that its effect on bothperceived likelihood and worry derives primarily from the incremental risk framingrather than from sequential presentation. Explicitly reporting incremental riskstatistics appears to facilitate recognition of how much risk already exists at base-line. Presenting adverse reaction risks in this manner may improve patient com-prehension of the effects of treatment decisions and support effective riskcommunication.
Consider what it is like to read the risk information described in the patient infor-mation brochures that accompany prescription medications. In the section describ-ing what side effects (‘‘adverse reactions’’) were observed in clinical trials, a tableusually shows the percentage of participants who experienced each side effect (e.g.,diarrhea, headaches, drowsiness) for all reactions in which the rate in the medicationgroup exceeded the placebo group (Food and Drug Administration guidance for indus-try: Adverse reactions section of labeling for human prescription drug and biologicalproducts—content and format, 2006). For example, you might read that 9% of peopletaking Drug X experienced heartburn, while 4% of people taking a placebo did so.But take a moment to ask yourself the following two questions: What is yourintuitive perception of the risks of taking Drug X given the information you read,and how well does that perception correspond to the statistical probabilities?
The problem with the tables described above is that they present people withstatistics for the total risk experienced by treatment and no treatment groups. Whatpatients really need to understand, however, is how much more or less risk they faceby taking the drug instead of not taking it. The goal is similar in many other riskcommunication contexts, such as risk-reducing medical interventions and consumerproducts that improve safety. In each case, patients need to consider the incrementalrisk associated with one intervention versus another.
When people read risk statistics, they typically process them using two parallelinformation processing systems. One, termed ‘‘System 1,’’ is predominantly associa-tive or experiential, while the second, ‘‘System 2,’’ is more rule based and rational(Kahneman & Frederick, 2002; Sloman, 1996; Smith & DeCoster, 2000). The intui-tive reactions we refer to above are System 1 outputs, generated by automatic pro-cessing of new stimuli based on saliency, similarity, and other basic heuristics. Thisprocess is associated more closely with the emotional impact of experiencing (orimagining) illness and with people’s ‘‘gut-feeling’’ perceptions of the chance they willexperience a side effect. By contrast, the System 2 process interprets risk informationsymbolically, rationally calculating and evaluating the probabilities of occurrence.Although System 2 thinking sometimes can override emotional reactions and inducemore logically consistent behavior, people can hold simultaneous contradictorybeliefs, for example, by cognitively knowing that the risk of a treatment complication
108 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
is small while intuitively or emotionally perceiving that risk to be significant andworrisome (Sloman, 1996; Windschitl, Martin, & Flugstad, 2002).
In order to understand how taking the drug changes the risk of adverse events,readers of total risk statistics must perform a System 2 calculation, contrasting theintervention group with the placebo group to identify the statistic of interest, i.e.,the increase in risk. Multiple factors may, however, lead to perceptions of risk thatmay differ from that calculation. First, people’s intuitive, System 1 processing mayfocus only on the total risk after treatment, leading people to react as if the entirerisk incurred by people taking the drug is caused by the drug and ignore the baselinerisks they face even if they do not take the drug. Second, even if people understandthat a calculation is needed, they may have difficulty performing the arithmetic,whether due to a general lack of numeracy skills (Lipkus, Samsa, & Rimer, 2001;Schwartz, Woloshin, Black, & Welch, 1997; Woloshin, Schwartz, Moncur, Gabriel,& Tosteson, 2001) or to context-specific barriers such as time pressure. Eitherway, comprehension will be disrupted and risk perceptions potentially distorted.Third, classic psychological research on numerical anchors (Tversky & Kahneman,1974) suggests that people may have stronger intuitive reactions whenever theabsolute risk statistic is perceived as large, even if the change in risk is actually small.If so, anything that makes the with-treatment risk seem larger (such as variationsin base rates or risk ratio denominators) may lead to increased concern.
Unfortunately, the risk communication literature generally has overlooked thisconcern, focusing primarily on how to improve comprehension of single risk sta-tistics without considering issues of comparison (Edwards, Elwyn, & Mulley,2002; Lipkus & Hollands, 1999). Stone and colleagues (Stone et al., 2003; Stone,Yates, & Parker, 1997) did focus their investigations of risk communication graphson how people valued changes in risk, but all of their work used side-by-side presen-tations (graphic, textual, or both) of the absolute risk amounts. Similarly, in medi-cine, decision boards to aid patient decision making about chemotherapy haveused pairs of pie charts to show the total cancer risks associated with two treatmentoptions (e.g., the risk of cancer relapse with and without adjuvant therapy, or therisk of being cancer-free with two different treatments; Stiggelbout & de Haes,2001; Whelan & Loprinzi, 2005).
We evaluate a less common approach: specifically reporting and highlightingthe size of risk increases (such as treatment side effects). This incremental risk isdescribed as the ‘‘additional risk’’ caused by treatment, and we consider the impactof risk communications using precisely that language. By this method, the risk ofheartburn caused by Drug X (as discussed above) is noted by reporting that 4%of people experience heartburn without medications, but an additional 5% of peopleexperience heartburn if they take Drug X. This approach is similar to both the‘‘absolute risk reduction with baseline risk’’ approach demonstrated to lead to moreaccurate comprehension of risk statistics (Schwartz et al., 1997) and to multicategorypictographs that identify how many people were ‘‘saved’’ and ‘‘harmed’’ by taking adrug (Edwards et al., 2002). It uses language previously suggested for probability-tradeoff preference elicitation tasks (Llewellyn-Thomas, McGreal, Thiel, Fine, &Erlichman, 1991; Llewellyn-Thomas, Williams, Levy, & Naylor, 1996). It also issimilar to the method used by Adjuvant! (www.adjuvantonline.com), a decisionaid for adjuvant therapies for breast cancer, which uses multicolor graphs to identifythe total ‘‘benefit’’ achieved versus no additional therapy (Peele, Siminoff, Xu, &Ravdin, 2005; Whelan & Loprinzi, 2005). Experimental evidence empirically
Framing Side Effects as Incremental Risks 109
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
comparing such an approach versus the current standard of presenting side-by-sideabsolute risk statistics, however, so far has been lacking.
In this article, we present results from two studies. Study 1 tested incrementalrisk presentations in the context of the real anticancer drug, tamoxifen. The inter-vention in Study 1 combined two manipulations into a single factor, however, sowe conducted Study 2 to identify the essential elements of the incremental riskapproach, using a scenario about a hypothetical antiseizure medication. In bothstudies, our goal was to assess how emotional and intuitive reactions to risk infor-mation are influenced by different presentation formats and whether descriptionsthat highlight incremental risk yield greater consistency of interpretation in the faceof known biasing factors.
Study 1
Methods
Study DesignIn Study 1, an Internet-administered survey, women read a brief description oftamoxifen, including an overview of its use to prevent breast cancer and the fact thatit increases the risk of four conditions: endometrial cancer, cardiovascular events(stroke, heart attacks, etc.), cataracts, and menopausal symptoms. Women then reada detailed description of each side effect that was paired with risk information abouteach one’s likelihood of occurrence. The exact probabilities (Table 1) were takenfrom published results of the P-1 clinical trial of tamoxifen to prevent breast cancerin high-risk women (Fisher et al., 1998). As shown in Figure 1, respondents answeredquestions about each side effect as each was presented, and then answered questionsabout the overall risk of side effects. We used a fully randomized, four-factor(2� 2� 2� 2), between-subjects design to test the effect of presentation type (totalrisk versus incremental risk), presentation mode (text only versus text plus picto-graph), risk denominator (out of 100 versus out of 1,000), and side effect order(increasing versus decreasing probability) on participants’ subjective reactions tothe risk statistics.
Presentation Type (Total Risk Versus Incremental Risk). Our primary manipu-lation varied whether the displayed risk statistics compared the total risk of each
Table 1. Side effect risks of tamoxifen presented to participants in Study 1
Number of women with each side effect in 5 years(Out of 1,000)
Baseline risk Risk Incremental riskSide effects without tamoxifen with tamoxifen of tamoxifen
Endometrial cancer 3 6 3Cardiovascular 21 28 7Cataracts 95 113 18Menopausal symptoms 686 816 130
Note: All risk information derived from the P-1 trial.
110 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
condition with and without tamoxifen, or whether they specifically highlighted theincremental risk incurred by taking tamoxifen. In the total risk displays, respondentssaw both the total baseline risk without tamoxifen and the total risk with tamoxifenin a single, side-by-side presentation. (See Figure 2 for an example.) The incrementalrisk diplays, however, used a two-stage process. Respondents first viewed the totalbaseline risk by itself. For example, the baseline risk of cataracts was described as‘‘9.5 women out of 100 would get cataracts without taking tamoxifen.’’ On the next
Figure 1. Instrument design, Study 1.
Figure 2. Total risk presentation of side effect risk.
Framing Side Effects as Incremental Risks 111
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
survey page, a second presentation (either graphical or textual; see below) indicatedthe incremental risk caused by taking tamoxifen in ‘‘additional risk’’ terms; forexample, ‘‘1.8 additional women out of 100 would get cataracts if they took tamox-ifen.’’ (See Figure 3.)
Presentation Mode (Text Versus TextþPictograph). Risk information waspresented in one of two modes: textually or in a pictograph. In the text displays, riskinformation was described in sentence form, with each statistic (e.g., 2.1 out of 100)repeated in a large font in a color-coded box. (Blue represented the risk of getting thecondition without tamoxifen, and orange represented either the total or incrementalrisk of getting the condition with tamoxifen.) In the textþ pictograph surveys, thegraphic showed a matrix of colored blocks representing the full population. Individ-ual shapes were color coded as above, with the addition of grey for the risk of notgetting the condition. Each graph’s legend included the numeric risk informationas well, to facilitate comprehension. Figure 2 shows a sample side-by-side graphicfrom the total risk conditions, while Figure 3 shows the same information as dis-played in the incremental risk conditions.
The remaining two factors, risk denominator and side effect order, both areknown to bias risk perceptions. We included these secondary factors in Study 1 toexplore whether highlighting the incremental risk might reduce these biases.
Risk Denominator (100 Versus 1,000). Several researchers have suggested thatinterpretation of risk information is facilitated by using frequencies instead of prob-abilities (Gigerenzer & Edwards, 2003; Hoffrage, Lindsey, Hertwig, & Gigerenzer,2000; Paling, 2003; Schapira, Nattinger, & McHorney, 2001). As a result, all riskswere described as frequencies (e.g., 9.5 women out of 100) rather than percentages.
Figure 3. Incremental risk presentation of side effect risk.
112 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
People’s reactions to identical risk information, however, can vary by the size of thedenominator used to describe the risk statistic (Denes-Raj, Epstein, & Cole, 1995;Yamagishi, 1997). In general, these ‘‘ratio-bias’’ studies have found that the largerthe denominator, the larger the perceived risk. We manipulated this dimension byhaving all risks either presented as ‘‘out of 100’’ or ‘‘out of 1,000.’’ In the pictographsurveys, the graphic either contained 100 colored rectangles or 1,000 smaller coloredsquares arranged similarly.
Side Effect Order (Increasing Versus Decreasing Likelihood). Because ofanchoring biases (Chapman & Johnson, 2002), women who viewed the large totalrisk of menopausal symptoms first may perceive the risks of less common side effectsto be smaller by comparison and thus easier to dismiss. By contrast, women whoviewed the smallest risks first may have been increasingly alarmed by subsequent,more frequent risks. Furthermore, women who first read about life-threatening sideeffects, such as endometrial cancer, might experience an emotional response thatcould make quality of life issues, such as menopausal symptoms, seem less worri-some. We tested for such order effects by having participants either view the sideeffects in order of increasing likelihood=decreasing severity (endometrial cancer, car-diovascular events, cataracts, menopausal symptoms) or the reverse order.
ParticipantsStudy participants were women drawn from a panel of Internet users administeredby Survey Sampling International (SSI) who voluntarily agreed to participate inresearch surveys. Electronic mail invitations were sent to a stratified random sampleof panel members, with the selection process weighted to mirror the U.S. censuspopulation based on age, education level, and income. Individuals completing ourweb-based survey were entered into a drawing to win cash prizes.
Outcome MeasuresBecause our primary focus was on intuitive processes more than on rational, calcu-lating processes, we asked our respondents to rate ‘‘how worried would you be about[side effect] if you took tamoxifen?’’ as each side effect was presented, plus a finalrating of all side effects together at the end of the survey. All ratings were madeon 11-point (0–10) scales, with endpoints labeled as ‘‘Not at all Worried’’ and‘‘Extremely Worried.’’ We chose this measure because worry about a risk has beenshown to influence people’s desire to act (Baron, Hershey, & Kunreuther, 2000) andpredicts perceived absolute risk (Lipkus, Klein, Skinner, & Rimer, 2005). We did notask for assessments of likelihood or numerical comprehension questions, in part outof concern that doing so would introduce excessively analytical reactions intoparticipants’ worry ratings.
Individual DifferencesWe also collected data on two individual difference measures. First, recent researchhas shown that even highly educated people have difficulty comprehending abstractquantitative information, such as statistics about the chance of experiencing compli-cations from treatment (Lipkus et al., 2001). This specific ability is termed numeracy(Schwartz, Woloshin, & Welch, 1999), and low numeracy has been shown to impedecomprehension and elicitation of health risk statistics (Black, Nease, & Tosteson,1995; Schapira, Davids, McAuliffe, & Nattinger, 2004; Schwartz et al., 1997;
Framing Side Effects as Incremental Risks 113
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Woloshin, Schwartz, Byram, Fischhoff, & Welch, 2000; Zikmund-Fisher, Smith,Ubel, & Fagerlin, 2007). To control for individual differences in numeracy, allparticipants completed the Subjective Numeracy Scale (SNS; Fagerlin et al., 2007),an 8-item self-assessment of numerical aptitude and preferences for numbers thatcorrelates strongly with objective numeracy measures and has been demonstratedto have validity in predicting people’s ability to recall and interpret risk information(Zikmund-Fisher et al., 2007). Second, to control for differences in prior knowledgeabout tamoxifen, we also asked participants to indicate on a 5-point (1–5) scale howmuch they knew about tamoxifen before taking our survey.
HypothesesWe had two main hypotheses:
H1: Because highlighting the incremental risk also clarifies how muchrisk already exists at baseline, we hypothesized that women’s worryabout the side effects of tamoxifen would be lower for incrementalrisk presentations than for total risk presentations.
H2: Because incremental risk presentations focus people’s attention onthe increase in risk without requiring them to do mental arithmeticon total risk statistics, we hypothesized that this approach wouldyield greater resistance to both order effects and changes in thedenominator of the risk statistics.
We had no a priori hypotheses regarding the relationship between textual pre-sentations and pictographs. We included both to see whether the effect of highlight-ing the incremental risk would differ across presentation formats.
Statistical AnalysisOur primary analysis consisted of ANOVAs for each side effect and all side effectstaken together, using reported worry as the dependent variable in each case. Initialanalyses included all interaction terms for this fully factorial design, but sub-sequently we dropped insignificant interaction terms not related to the incrementalrisk factor and report only the reduced models here. Effect size and directions thenwere confirmed by t tests. All analyses were performed using STATA, Version 8.
Results
A total of 2,512 women clicked on the study website, and 80% of these (N ¼ 2,012)provided useable data. We successfully recruited a sample with wide variation in ageand education, and our randomization process successfully resulted in no significantdifferences in reported demographics between experimental groups. In the totalsample, mean participant age was 42 years (range 18–78), and while 27% of studyparticipants had completed a bachelor’s degree or higher level of education, 23%had only a high school diploma or less education. Participation of racial and ethnicminorities, however, was less representative: Only 5% of the study populationidentified themselves as African American, and another 5% identified themselvesas Hispanic or Latino.
114 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Primary FactorsDrawing respondents’ attention to the incremental risk significantly lowered howmuch participants thought they would worry about all four side effects. Table 2displays the mean worry ratings, broken down by our two primary factors, presen-tation type (incremental vs. total risk) and presentation mode. Considering firstthose participants who saw risk information presented only as numerical text,reported worry about each individual side effect and about all side effects takentogether was significantly higher (all p’s < 0.001) in the total risk groups versusthe incremental risk groups, thus providing strong support for Hypothesis 1. Thispattern was repeated, albeit at somewhat lower levels of significance, when riskinformation was visually displayed in pictograph form.
As noted above, our initial ANOVA analyses included terms for all four experi-mental main effects, all 2-, 3-, and 4-way interactions, participants’ ratings of priortamoxifen knowledge, and respondents’ average SNS rating. All 3- and 4-way inter-actions were nonsignificant, as were all 2-way interactions not involving presentationtype (total risk vs. incremental risk). We therefore report only reduced models here.
The reduced ANOVA model results are reported in Table 3. We found highlysignificant main effects of presentation type (incremental risk vs. total risk formats)in each of our four ANOVAs (all p’s < 0.001), confirming the univariate analyses.The pictograph manipulation had a significant main effect only for cataracts( p < 0.01). The significance of the incremental risk� pictograph interaction termin four of the five ANOVAs supports the belief that the effect of incremental riskformats is generally larger for text-only presentations and appears to be muted bythe addition of risk graphics.
Secondary Factors: Main EffectsVarying the order of presentation had a statistically significant effect on respondents’perceived worry in four out of the five ANOVAs ( p’s < 0.01), with higher levels ofworry reported when participants saw the risks in order of increasing probability=decreasing severity versus the reverse order. Varying the risk denominator increasedworry about the two side effects with smallest likelihood of occurrence ( p < 0.001for endometrial cancer, p < 0.05 for cardiovascular problems), suggesting that theimpact of this factor may be limited to low probability events.
Secondary Factors: Interactions With Incremental Risk FormatsWe observed mixed support for Hypothesis 2. Our evidence regarding ratio bias ordenominator effects is consistent with the idea that incremental risk formats increaseresponse consistency. Table 3 shows that the incremental risk format� denominatorinteraction term tracks the main effect: For endometrial cancer, the main effect ofthe denominator is highly significant ( p < 0.001), as is the interaction ( p ¼ 0.01).For cardiovascular problems, the main effect is significant ( p < 0.05), and the inter-action with incremental risk formats is marginally significant ( p < 0.10). Bothresults suggest that when denominator effects exist (e.g., in cases where the probabil-ities are quite small), the bias is minimized by using incremental risk formats. Neitherthe main effect of the denominator nor its interaction with presentation type,however, is significant in any of the other three ANOVA analyses.
Our evidence regarding order effects is much weaker. Although there were maineffects of presentation order for four out of the five ANOVAs reported in Table 3,
Framing Side Effects as Incremental Risks 115
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Ta
ble
2.
Per
ceiv
edw
orr
ya
bo
ut
tam
ox
ifen
sid
eef
fect
s,S
tud
y1
Va
ria
ble
En
do
met
ria
lca
nce
rC
ard
iov
asc
ula
rp
rob
lem
sC
ata
ract
sM
eno
pa
usa
lsy
mp
tom
sA
llsi
de
effe
cts
Tex
tua
lp
rese
nta
tio
ns
To
tal
risk
form
at
5.0
65
.10
5.3
55
.22
5.7
4In
crem
enta
lri
skfo
rma
t3
.76
4.2
53
.94
3.8
04
.25
tst
ati
stic
6.5
8���
4.3
5���
7.2
3���
7.3
4���
8.1
4���
Co
hen
’sd
0.4
20
.28
0.4
60
.47
0.5
3T
extþ
pic
tog
rap
hp
rese
nta
tio
ns
To
tal
risk
form
at
4.5
44
.65
4.5
54
.92
5.2
4In
crem
enta
lri
skfo
rma
t3
.96
4.2
83
.92
4.3
84
.42
tst
ati
stic
2.8
5��
1.8
7y
3.2
7��
2.8
2��
4.3
3���
Co
hen
’sd
0.1
80
.12
0.2
10
.18
0.2
9
No
tes:
Wo
rry
mea
sure
do
na
n1
1-p
oin
tsc
ale
wh
ere
0¼
No
ta
ta
llW
orr
ied
an
d1
0¼
Ex
trem
ely
Wo
rrie
d.
��� p<
0.0
01
;��
p<
0.0
1;� p<
0.0
5;y p<
0.1
0.
116
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Ta
ble
3.
Pre
dic
tors
of
sid
eef
fect
wo
rry
inA
NO
VA
an
aly
ses
of
Stu
dy
1
Va
ria
ble
En
do
met
ria
lca
nce
rC
ard
iov
asc
ula
rp
rob
lem
sC
ata
ract
sM
eno
pa
usa
lsy
mp
tom
sA
llsi
de
effe
cts
Incr
emen
tal
risk
form
at
42
.56���
20
.32���
58
.00���
52
.47���
77
.95���
Pic
tog
rap
h0
.97
1.3
28
.80��
0.6
11
.51
1,0
00
den
om
ina
tor
19
.71���
4.8
7�
1.3
50
.79
11
.53���
Ord
ero
fp
rese
nta
tio
n8
.16��
3.1
4y
12
.45���
14
.43���
27
.58���
Incr
emen
tal
risk�
pic
tog
rap
h6
.56�
2.9
7y
9.0
4��
8.9
3��
6.3
5�
Incr
emen
tal
risk�
den
om
ina
tor
6.4
3�
2.8
9y
0.5
00
.63
0.7
6In
crem
enta
lri
sk�
ord
er2
.47
2.0
32
.21
1.5
01
.57
Nu
mer
acy
sco
re(S
NS
)0
.43
0.0
85
.43�
0.8
02
.56
Pri
or
tam
ox
ifen
kn
ow
led
ge
1.5
30
.03
5.2
2�
0.3
90
.00
No
tes:
Ta
ble
rep
ort
ses
tim
ate
dF
sta
tist
ics.
Dep
end
ent
va
ria
ble
isre
po
rted
wo
rry
ab
ou
tth
esp
ecif
ied
sid
eef
fect
,o
ra
llsi
de
effe
cts
tog
eth
er,
ifth
esu
bje
ctto
ok
tam
ox
ifen
(as
mea
sure
do
na
n1
1-p
oin
tsc
ale
wh
ere
0¼
No
ta
ta
llW
orr
ied
an
d0¼
Ex
trem
ely
Wo
rrie
d).
��� p<
0.0
01
;��
p<
0.0
1;� p<
0.0
5;y p<
0.1
0.
117
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
the interaction term of incremental risk x order was never significant. Examinationof univariate analyses showed trends in the direction of our hypothesis (i.e., smallerorder effects with presentations that highlighted the risk increase), but the effect sizeswere nonsignificant.
Study 2
Study 1 demonstrated that risk communications that highlight the incremental riskresult in less worry. Was the effect caused by the incremental risk framing, thesequential presentation, or both? Study 2 was designed to answer this question, toconfirm that difference-highlighting approaches affect the perceived likelihood ofside effects and not just worry, and to extend our findings to a new decision scenario.
Methods
Study Design and ParticipantsRespondents to an Internet-administered survey read a brief scenario asking them toimagine that they had mild epilepsy. The scenario then described a hypothetical medi-cation, ‘‘SeizeStop,’’ which would eliminate their seizures but carried the risk of threecomplications: strokes, headaches, and increased frequency of colds. As in Study 1,respondents read detailed descriptions of each side effect, followed by risk informationabout likelihoods of occurrence. The exact probabilities used are shown in Table 4. Weassessed participants’ subjective reactions to different communication formats, using a2� 2 factorial design to independently vary risk framing (total risk versus incrementalrisk, as described for Study 1) and number of graphs (single graph versus a sequence oftwo graphs). We did not repeat the secondary factor manipulations from Study 1; allparticipants saw risk information in 1,000 square pictograph format in the order ofincreasing likelihood (i.e., strokes, headaches, and then colds).
Study participants were men and women again drawn from the SSI panel ofInternet users. Recruitment and sampling procedures were as in Study 1 exceptfor the following changes: (a) We relaxed the gender restriction to include men aswell as women. (b) We age stratified the sample (1=3 each ages 18–39, 40–59, and60 and older) to offset differential response rates. (c) We intentionally selected onlyAfrican American and White participants, in equal numbers, to enable sufficientpower for focused racial analyses.
Outcome Measures and Independent VariablesStudy 2 included two primary outcome measures. For each individual side effect,respondents first indicated their perceived likelihood of experiencing each side effect
Table 4. Hypothetical risks presented to participants in Study 2
Number of people with each side effect (Out of 1000)
Baseline risk Risk Incremental riskSide effects without ‘‘SeizeStop’’ with ‘‘SeizeStop’’ of ‘‘SeizeStop’’
Strokes 8 13 5Headaches 109 145 36Increased colds 689 770 81
118 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
if they took ‘‘SeizeStop,’’ using an 11-point (0–10) scale with endpoints labeled ‘‘Notat all Likely’’ and ‘‘Extremely Likely.’’ They then reported how much they thoughtthey would worry about each side effect if they took ‘‘SeizeStop,’’ using the samescale as in Study 1. After providing separate ratings for each of the three conditions,participants then provided an overall rating of the likelihood of experiencing sideeffects from taking ‘‘SeizeStop,’’ using the likelihood scale described above. As inStudy 1, we also collected ratings of individual differences in participant numeracyusing the SNS (Fagerlin et al., 2007).
Statistical AnalysisOur primary analyses followed Study 1, focusing on respondents’ assessments ofworry about each condition as well as a new measure of perceived side effect likeli-hood as dependent variables. As before, effect size and directions were confirmedby t tests, and all analyses were performed using STATA, Version 8.
Results
A total of 1,666 people reached the study website, and 84% of these (N ¼ 1,393)provided useable data. As in Study 1, we successfully randomized participants acrossthe different survey versions, with each version showing similar reported demo-graphics. In the total sample, mean age was 49 years (range 18–86), with 34% havingcompleted a bachelor’s degree or higher level of education but 18% with only a highschool diploma or less education. As noted above, we racially stratified the sampleso that exactly 50% of participants were African American.
Table 5 reports the ANOVA analysis results from Study 2. A clear and consis-tent pattern is observed for all dependent variables: We observe a significant maineffect of incremenal risk framing but no main effect or interaction effect related toa using a two-graph sequence instead of just a single pictograph. T tests confirm thatincremental risk framing reduced both perceived likelihood of experiencing compli-cations, as well as the corresponding worry. Effect sizes ranged from d ¼ 0.15 forjudgments of the likelihood of strokes to d ¼ 0.39 for perceived likelihood of experi-encing more colds.
Unlike in Study 1, individual differences in numeracy did consistently influencesurvey responses in Study 2: Participants with higher numeracy scores perceivedsignificantly less risk and were less worried about side effects than participants withlower numeracy scores. We also observed a strong effect of race: In every analysis,African American participants perceived the side effects of ‘‘SeizeStop’’ to be signifi-cantly more likely and more worrisome than did White participants. Neither numer-acy nor race, however, showed any consistent interaction with the incremental riskframing or sequential presentation manipulations.
General Discussion
In Study 1, women received information about the side effects of tamoxifen in one oftwo formats: a side-by-side presentation of the total risk both with and withouttamoxifen or a sequential presentation that emphasized the incremental risk causedby taking tamoxifen. Women who viewed an incremental risk presentation reportedsignificantly less worry about complication risks. In Study 2, participants receivinginformation about complications of the hypothetical drug ‘‘SeizeStop’’ showed
Framing Side Effects as Incremental Risks 119
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Ta
ble
5.
Pre
dic
tors
of
sid
eef
fect
wo
rry
an
dp
erce
ived
lik
elih
oo
din
AN
OV
Aa
na
lyse
so
fS
tud
y2
Str
ok
esH
ead
ach
esC
old
sA
llsi
de
effe
cts
Va
ria
ble
Lik
elih
oo
dW
orr
yL
ikel
iho
od
Wo
rry
Lik
elih
oo
dW
orr
yL
ikel
iho
od
Incr
emen
tal
risk
form
at
7.1
4��
13
.93��
33
.98���
27
.27���
50
.41���
49
.01���
28
.95���
Ser
ial
pre
sen
tati
on
s0
.06
0.1
70
.08
0.1
91
.93
2.1
01
.35
Incr
emen
tal
risk�
seri
al
0.0
00
.05
0.4
20
.23
0.4
70
.31
0.0
4N
um
era
cysc
ore
(SN
S)
15
.58���
14
.31���
18
.40���
17
.21���
15
.03���
17
.93���
13
.30���
Afr
ica
n-A
mer
ica
net
hn
icit
y2
3.6
9���
39
.01���
10
.95���
33
.67���
23
.03���
46
.62���
54
.14���
No
tes:
Ta
ble
rep
ort
ses
tim
ate
dF
-sta
tist
ics.
Dep
end
ent
va
ria
ble
isth
e(a
)p
erce
ived
lik
elih
oo
do
fth
esp
ecif
ied
sid
eef
fect
or
(b)
per
ceiv
edw
orr
ya
bo
ut
the
sid
eef
fect
wit
htr
eatm
ent
(bo
thm
easu
red
on
0–
10
sca
les)
.��� p<
0.0
01
;��
p<
0.0
1.
120
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
analogous reductions in both perceived likelihood of side effects and perceived worrywhen participants viewed graphics that highlighted the incremental risk, regardlessof whether a single figure or a series of graphs were shown. These results stronglysuggest that framing risk information in terms of incremental risk can focus people’sattention on the statistic most relevant to decision making (i.e., the change in risk)and enable them to understand and remember how much side effect risk a medicationcauses, as opposed to how likely it is that they might experience a side effect (whethermedication-caused or not). Since most medication complications can occur naturally(and, for certain conditions, quite frequently), this focusing effect limits the amountof concern experienced. While we cannot know what the right degree of worry shouldbe for each side effect or for any particular individual, highlighting the change in riskshould facilitate the important goal of helping patients to distinguish more accuratelybetween risks caused by treatment and those caused by other factors.
Explicitly reporting the incremental risk also may moderate ratio-bias effects.In Study 1, when we varied the denominator of our risk statistics, respondentsreported significantly greater worry about two side effects, endometrial cancer andcardiovascular problems, when they viewed total risk displays ‘‘out of 1,000’’ versuswhen they saw the same risk information ‘‘out of 100.’’ Women’s reactions weremore consistent, however, when viewing incremental risk presentations that iden-tified the ‘‘additional risk’’ caused by taking tamoxifen.
It is worth clarifying the distinction between incremental risk and relative riskpresentations. In relative risk communications, changes in risk are described in per-centages or ratios without defining the baseline risk level. Thus, a medication may bedescribed as conferring ‘‘50% more risk’’ than another, even though the increase isonly from 2% to 3%. Such approaches tend to magnify perceptions of risk changes,and the perils of using them have been well documented in the medical profession(Bobbio, Demichelis, & Giustetto, 1994; Fahey, Griffiths, & Peters, 1995; Forrow,Taylor, & Arnold, 1992; Malenka, Baron, Johansen, Wahrenberger, & Ross, 1993;Naylor, Chen, & Strauss, 1992), as well as the literature on valuing life-saving inter-ventions (Baron, 1997; Fetherstonhaugh, Slovic, Jonhnson, & Friedrich, 1997; Jenni& Loewenstein, 1997). Incremental risk presentations, whether tabular or graphical,focus attention on the marginal increment in risk while simultaneously providinginformation about baseline risk levels for context. They thus represent a middleground, focusing attention on the change in risk, yet quantifying that change in onlyabsolute, not relative, terms.
Our research has several limitations. First, our surveys involved general publicsamples rather than specific patient populations. For example, women who areconsidering taking tamoxifen themselves might respond differently to the riskinformation shown than Study 1 participants did. Second, our Internet survey meth-odology led to limited participation of ethnic and racial minorities in Study 1 andcould have made our samples less representative in other ways as well. Third, we choseto present all risk statistics using a frequency format rather than the more commonlyused probability format. Given that people generally have greater difficulty under-standing probabilities than they do frequencies, however, the attention-focusingnature of incremental risk presentations may be even more beneficial when risksare shown as probabilities than how they were shown in our research.
We also note that our decision to intentionally limit our investigation to consideronly situations where treatment increases risk, i.e., side effects, limits the generaliz-ability of our findings. While change-highlighting approaches may be valuable for
Framing Side Effects as Incremental Risks 121
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
communicating risk reductions, design of such approaches is complicated by the factthat the baseline risk overlaps with the risk difference. Furthermore, any researchthat examines communications about both risk increases and risk decreases simul-taneously (e.g., by asking respondents to make risk–benefit tradeoffs) runs the riskthat relevant effects might cancel out each other.
Nevertheless, we believe that our findings have important implications for thelabeling of prescription drugs; the design of health communications, patient infor-mation brochures, and decision aids; and even the way health care professionals dis-cuss risks with patients. People’s intuitive, System 1 reactions to risk statistics areinfluenced by how much risk exists at baseline. While rational, System 2 processingmay correct for this bias in some decision-making contexts, the use of presentationsthat emphasize the incremental risk caused by treatment may be an appropriatepreventive measure. Focusing patient attention on the incremental risk also mayfacilitate the type of systematic, tradeoff-based thinking that is often the goal ofhealth communications and patient decision aids and that is essential to effective dis-cussions of treatment options in clinical consultations. We urge further research toclarify the potential benefits of incremental risk communications and to test theireffectiveness in applied settings.
References
Baron, J. (1997). Confusion of relative and absolute risk in valuation. Journal of Risk andUncertainty, 14, 301–309.
Baron, J., Hershey, J. C., & Kunreuther, H. (2000). Determinants of priority for riskreduction: The role of worry. Risk Analysis, 20(4), 413–427.
Black, W. C., Nease, R. F., & Tosteson, A. (1995). Perceptions of risk and screening effective-ness in women younger than 50 years of age. Journal of the National Cancer Institute, 87,720–731.
Bobbio, M., Demichelis, B., & Giustetto, G. (1994). Completeness of reporting trial results:Effect on physicians’ willingness to prescribe. Lancet, 343, 1209–1211.
Chapman, G. B., & Johnson, E. J. (2002). Incorporating the irrelevant: Anchors in judgmentsof belief and value. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristicsand Biases: The psychology of intuitive judgment (pp. 120–138). New York: CambridgeUniversity Press.
Denes-Raj, V., Epstein, S., & Cole, J. (1995). The generality of the ratio-bias phenomenon.Personality and Social Psychology Bulletin, 21(10), 1083–1092.
Edwards, A., Elwyn, G., & Mulley, A. (2002). Explaining risks: Turning numerical data intomeaningful pictures. British Medical Journal, 324, 827–830.
Fagerlin, A., Zikmund-Fisher, B. J., Ubel, P. A., Jankovic, A., Derry, H., & Smith, D. M.(2007). Measuring numeracy without a math test: Development of the subjective numer-acy scale (SNS). Medical Decision Making, 27, 672–680.
Fahey, T., Griffiths, S., & Peters, T. J. (1995). Evidence based purchasing: Understandingresults of clinical trials and systematic reviews. BMJ, 311, 1056–1059.
Fetherstonhaugh, D., Slovic, P., Johnson, S. M., & Friedrich, J. (1997). Insensitivity to thevalue of human life: A study of psychological numbing. Journal of Risk and Uncertainty,14, 283–300.
Fisher, B., Costantino, J. P., Wickerham, D. L., Redmond, C. K., Kavanah, M., Cronin, W.M., et al. (1998). Tamoxifen for prevention of breast cancer: Report of the NationalSurgical Adjuvant Breast and Bowel Project P-1 Study. Journal of the National CancerInstitute, 90(18), 1371–1388.
122 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Food and Drug Administration guidance for industry: Adverse reactions section of labeling forhuman prescription drug and biological products—content and format.(2006). U.S. Depart-ment of Health and Human Services. Retrieved from http://www.fda.gov/cder/guidance/5537fnl.htm.
Forrow, L., Taylor, W. C., & Arnold, R. M. (1992). Absolutely relative: How research resultsare summarized can affect treatment decisions. American Journal of Medicine, 92,121–124.
Gigerenzer, G., & Edwards, A. (2003). Simple tools for understanding risks: From innumeracyto insight. British Medical Journal, 327, 741–744.
Hoffrage, U., Lindsey, S., Hertwig, R., & Gigerenzer, G. (2000). Communicating statisticalinformation. Science, 290(5500), 2261–2262.
Jenni, K. E., & Loewenstein, G. (1997). Explaining the ‘‘Identifiable victim effect.’’ Journal ofRisk and Uncertainty, 14, 235–237.
Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitutionin intuitive judgment. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristicsand biases: The psychology of intuitive judgment (pp. 49–81). New York: CambridgeUniversity Press.
Lipkus, I. M., & Hollands, J. G. (1999). The visual communication of risk. Journal of theNational Cancer Institute. Monographs, 25, 149–163.
Lipkus, I. M., Klein, W. M. P., Skinner, C. S., & Rimer, B. K. (2005). Breast cancer riskperceptions and breast cancer worry: what predicts what? Journal of Risk Research,8(5), 439–452.
Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001). General performance on a numeracy scaleamong highly educated samples. Medical Decision Making, 21(1), 37–44.
Llewellyn-Thomas, H. A., McGreal, M. J., Thiel, E. C., Fine, S., & Erlichman, C. (1991).Patients’ willingness to enter clinical trials: Measuring the association with perceivedbenefit and preference for decision participation. Social Science & Medicine, 32(1), 35–42.
Llewellyn-Thomas, H. A., Williams, J. I., Levy, L., & Naylor, C. D. (1996). Using a trade-offtechnique to assess patients’ treatment preferences for benign prostatic hyperplasia.Medical Decision Making, 16(3), 262–282.
Malenka, D. J., Baron, J. A., Johansen, S., Wahrenberger, J. W., & Ross, J. M. (1993). Theframing effect of relative and absolute risk. Journal of General Internal Medicine, 8(10),543–548.
Naylor, C. D., Chen, E., & Strauss, B. (1992). Measured enthusiasm: Does the method ofreporting trial results alter perceptions of therapeutic effectiveness? Annals of InternalMedicine, 117, 916–921.
Paling, J. (2003). Strategies to help patients understand risks. British Medical Journal, 327,745–748.
Peele, P., Siminoff, L., Xu, Y., & Ravdin, P. (2005). Decreased use of adjuvant breast cancertherapy in a randomized controlled trial of a decision aid with individualized risk infor-mation. Medical Decision Making, 25, 301–307.
Schapira, M. M., Davids, S. L., McAuliffe, T. L., & Nattinger, A. B. (2004). Agreementbetween scales in the measurement of breast cancer risk perceptions. Risk Analysis,24(3), 665–673.
Schapira, M. M., Nattinger, A. B., & McHorney, C. A. (2001). Frequency or probability? Aqualitative study of risk communication formats used in health care. Medical DecisionMaking, 21(6), 459–467.
Schwartz, L. M., Woloshin, S., Black, W. C., & Welch, H. G. (1997). The role of numeracyin understanding the benefit of screening mammography. Annals of Internal Medicine,127(11), 966–972.
Schwartz, L. M., Woloshin, S., & Welch, H. G. (1999). Risk communication in clinical prac-tice: Putting cancer in context. Journal of the National Cancer Institute. Monographs, 25,124–133.
Framing Side Effects as Incremental Risks 123
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological Bulletin,119(1), 3–22.
Smith, E. R., & DeCoster, J. (2000). Dual-process models in social and cognitive psychology:Conceptual integration and links to underlying memory systems. Personality and SocialPsychology Review, 4(2), 108–131.
Stiggelbout, A. M., & de Haes, J. C. J. M. (2001). Patient preference for cancer therapy: Anoverview of measurement approaches. Journal of Clinical Oncology, 19(1), 220–230.
Stone, E. R., Sieck, W. R., Bull, B. E., Yates, J. F., Parks, S. C., & Rush, C. J. (2003). Fore-ground: Background salience: Explaining the effects of graphical displays on riskavoidance. Organizational Behavior and Human Decision Processes, 90, 19–36.
Stone, E. R., Yates, J. F., & Parker, A. M. (1997). Effects of numerical and graphical displayson professed risk-taking behavior. Journal of Experimental Psychology, 3, 243–256.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases.Science, 185, 1124–1131.
Whelan, T., & Loprinzi, C. (2005). Physician=patient decision aids for adjuvant therapy.Journal of Clinical Oncology, 23, 1627–1630.
Windschitl, P. D., Martin, R., & Flugstad, A. R. (2002). Context and the interpretation oflikelihood information: The role of intergroup comparisons on perceived vulnerability.Journal of Personality and Social Psychology, 82(5), 742–755.
Woloshin, S., Schwartz, L. M., Byram, S., Fischhoff, B., & Welch, H. G. (2000). A new scalefor assessing perceptions of chance: A validation study. Medical Decision Making, 20(3),298–307.
Woloshin, S., Schwartz, L. M., Moncur, M., Gabriel, S., & Tosteson, A. N. (2001). Assessingvalues for health: Numeracy matters. Medical Decision Making, 21(5), 382–390.
Yamagishi, K. (1997). When a 12.86% mortality is more dangerous than 24.14%: Implica-tions for risk communication. Applied Cognitive Psychology, 11, 495–506.
Zikmund-Fisher, B. J., Smith, D. M., Ubel, P. A., & Fagerlin, A. (2007). Validation of thesubjective numeracy scale (SNS): Effects of low numeracy on comprehension of riskcommunications and utility elicitations. Medical Decision Making.
124 B. J. Zikmund-Fisher et al.
Dow
nloa
ded
by [
Nor
th D
akot
a St
ate
Uni
vers
ity]
at 1
9:19
10
Oct
ober
201
4
