Presenting statistical results to nonstatistical audiences

The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.

Presenting statistical results to nonstatistical audiences

Jane E. Miller, PhD


Overview

• Academic and nonstatistical audiences– Defined– Interests and background

• Adapting description of methods • Adapting presentation of results


Why adapt material?

• A survey by Sorian and Baugh of government policy makers showed that they– Want to know how the findings relate to issues– Don’t want to wade through a formal research

paper

• Complaints about many research reports – “too long, dense, or detailed”– “too theoretical, technical, or jargony”


Example: Policy analysts and consultants

• Policy analysts must explain results of their models to experts in government or nonprofit agencies.

• Economic consultants have to communicate results of their models to corporations or community development agencies.

• Those experts are principally interested in– How to interpret and apply the findings.– Reassurance that you know the correct statistical

methods.


How to adapt material

• Familiarize yourself with your audience’s interests and likely applications of your study findings. – Present your analyses to match issues of concern

to them.– DON’T make them translate statistical results to fit

their interests.


What statistics courses teach• Statistics courses emphasize

– understanding statistical assumptions– estimating models– interpreting statistical tests– assessing coefficients and model fit.

• Students expected to demonstrate mastery by – working with equations written in statistical

notation– identifying the numbers for formal hypothesis

testing


What academic papers look like

• Detailed review of the literature• Comprehensive data and methods section• Statistical tables• Jargon and equations used as shorthand

• A real mismatch with many applied audiences


Example: Study of family and county level factors associated with SCHIP disenrollment

• SCHIP = State Children’s Health Insurance Program– Health insurance for children in low- to moderate-income

families who lack other health insurance

• Collaborative effort of– Rutgers University’s Center for State Health Policy– New Jersey Department of Human Services

• Project applied discrete time hazards models in a multilevel (hierarchical linear model [HLM]) framework


Academic audiences for SCHIP study

• Northwestern/University of Chicago Joint Center for Poverty Research (JCPR)– Funding agency– Policy oriented

• Rutgers Institute for Health, Health Care Policy and Aging Research and Bloustein School of Planning and Public Policy– Both policy-oriented research units

• University of Pennsylvania– Academic but not policy oriented

• Health Services Research– Journal with research emphasis


Applied audiences for SCHIP study

• New Jersey Department of Human Services– Raised policy question– Provided data– Client for deliverable

• US Department of Health and Human Services– Funding source


Which would you rather have?


Which would your client rather have?Table 1. Multilevel discrete-time hazards models of disenrollment from SCHIP, New Jersey, 1998–2000

Variable

County FixedEffects Model

Random Effects ModelFamily Factors Only

Random Effects ModelFamily + County Factors

Log Rel. Haz s.e. Log Rel. Haz s.e. Log Rel. Haz s.e.

Intercept –5.581 (.159) –5.421 (.142) –5.455 (.159)

Family-Level Characteristics

Black Race 0.047 (.150) 0.038 (.149) 0.198 (.165)

Hispanic Race 0.121 (.064) 0.109 (.063) 0.124 (.064)

Plans C and D (ref = Plan B) 0.826 (.142) 0.823 (.142) 0.825 (.142)

Interactions

Black * Plans C/D 0.449 (.154) 0.456 (.154) 0.451 (.154)

Plans C/D * Months 0.078 (.036) 0.078 (.036) 0.077 (.036)

Plans C/D * Months2 –0.0069 (.0019) –0.0069 (.0019) –0.0068 (.0019)

County-Level Characteristics

% Black Physicians .007 (.012)

Cross-Level Interaction

Black * % Black Physicians –0.039 (.019)

Random Effects

Between-County Variance 0.012 (.007) 0.005 (0.006)

Scaled Deviance Statistic 30,824.5 30,948.4 30,895.4

The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.The Chicago Guide to Writing about Multivariate Analysis, 2nd edition.

Chances of disenrollment by race, SCHIP plan, and county physician racial composition


Adapting results for nonstatisticians• Increase prominence of the substantive

question.• Reduce emphasis on technical details of data

and methods.– Rephrase jargon and statistical concepts into

colloquial language.– Avoid equations or Greek symbols.– Minimize use of formal citations.

• Translate results to show how they apply to real-world issues of interest to that audience.


Writing style and organization• Write a clear, well-organized narrative

– What questions did you address?– What answers did you find? – How can the findings be applied?

• Use standard expository writing guidelines– Good introduction– Present numbers as evidence

• Explain what question each is intended to answer

– Good summary of findings and what they mean• See article in Chance and podcast on presenting numbers as

evidence.


How to write about technical stuff

• Explaining why your methods are needed– Especially if using multivariate models

• Showing how key variables are measured• Interpreting numeric values (coefficients) • Reporting statistical significance• Adapting tables and charts


Acronyms and statistical vocabulary• Even with a quantitatively sophisticated audience,

don’t assume that people will know the statistical vocabulary used in other fields. – Define the term you use, then mention synonyms.

• If you use acronyms, spell them out at first usage.– “HEDIS”(Health Plan Employer Data and Information Set) – “HLM”(hierarchical linear model)

• Avoid acronyms if they are not familiar to the field or are used only once or twice.


Why your methods are needed• Explain what your model did that couldn’t have

been answered with simpler techniques. • Incorporate the specific concepts you study.Poor: “We use logistic regression and a discrete-time hazards

specification to assess relative hazards of SCHIP disenrollment, with plan level as our key independent variable.”

Better: “Because chances of disenrollment from the State Children’s Health Insurance Program (SCHIP) vary by the amount of time enrolled, our analyses correct for differences in duration of enrollment across families when estimating the patterns for different income levels.”


Application of a method to your topic

• Replace technical terms with familiar names.• Show how that method applies to your

research question and data.

• Poor: “The data structure can be formulated as a two-level hierarchical linear model, with families (the level-1 unit of analysis) nested within counties (the level-2 unit of analysis).”


Better presentation of methods: Tailored to the audience

Better [for a nonstatistical but academic audience]: “The data have a hierarchical (or multilevel) structure, with families clustered within counties.”

Better [for a lay audience]: “To disentangle the contributions of families’ and counties’ characteristics to the problem of program disenrollment, we used models that incorporated information at both levels.”


Measurement of key variables• To report an unfamiliar type of statistic,

embed the definition in your explanation.Poor: “The sensitivity of the new screening test

for diabetes is 0.90.”

Better: “The new screening test had a sensitivity of 0.90, correctly identifying 90% of diabetics.”


Adapting tables and charts

• Create small tables or charts– Divide up large complex tables into smaller parts– Focus each on one fact or pattern– Use simple, familiar formats

• Replace standard errors and test statistics with – p-values– Symbols such as asterisks or daggers– Formatting such as color, italics, or bold


Birth weight and socioeconomic characteristics by race/ethnicity, US, 1988–1994 NHANES III

Non-Hispanic white

Non-Hispanic black

Mexican American All

Birth weight

Mean (grams) 3,426.8 3,181.3 3,357.3 3,379.2

% Low birth weight 5.8 11.3 7.0 6.8

Socioeconomic characteristics

% Teen mother 9.4 22.9 18.4 12.5

% Mother <high school 14.7 30.1 58.4 21.6

% Poor 14.7 48.5 50.7 23.9

Unweighted N 3,733 2,968 3,112 9,813

Statistics are weighted to population level using weights provided with the NHANES III (US DHHS 1997).Differences across racial/ethnic origin groups were statistically significant for all variables shown (p < 0.01).

OK, but…


Low birthweight by race/ethnicity

From second row of preceding table.

p < 0.05


Minority racial groups have lower SES

From bottom three rows of table.

All p < 0.05


Interpreting OLS coefficients

• Emphasize direction and size of the association

• Name the specific variables involved• Incorporate units of measurement • Use colloquial language

“OLS” = ordinary least squares regression


Examples of interpreting βsPoor: “Age and weight were correlated.”Poor version number2: “Beta was 10.7.”Better: “For each additional year of mother’s age

at the time of her child’s birth, birth weight increased by 10.7 grams.”


Coefficients from logit models

• Replace log-odds (logit coeffs) with odds ratios.– Can be described in terms of simple multiples.– Don’t need to use the term “odds ratio” at all!

Poor: “The log-hazard of disenrollment for one-child families was 0.316.”

Better: “Families with only one child enrolled in the program were about 1.4 times as likely as larger families to disenroll.”


Wording for statistical significance

• State the conclusion of the statistical test, not the raw numbers or calculations.

Poor: “The log-relative hazard for SCHIP plans C and D was 0.826 with a standard error of 0.142. Because the beta was more than 2.56 times the standard error, we conclude that the effect is statistically significant at p < 0.01.”

Better: “Families in SCHIP plans C and D were roughly 2.3 times as likely to disenroll as those in plan B. A difference that large is unlikely to occur by random chance alone.”


Wording for LACK of statistical significance

• “The difference between the disenrollment rates for Plans C and D could easily have occurred by chance alone.”


Summary• Get to know your audience before you write.

– What questions do they want answered?– How familiar are they with statistics?

• Avoid statistical language.– Report direction and size of associations in plain

English.– Mention conclusions of inferential statistics, not the

raw numbers or calculations.

• Use charts or simple tables to convey shape and size of numeric patterns visually.


Suggested resources• Chapter 20 in Miller, J. E. 2013. The Chicago Guide to

Writing about Multivariate Analysis, 2nd Edition.• Miller, J.E. 2006. “How to Communicate Statistical

Findings: An Expository Writing Approach.” Chance. 19(4):43-49.

• Nelson, D. E., R. C. Brownson, P. L. Remington, and C. Parvanta, editors. 2002. Communicating Public Health Information Effectively: A Guide for Practitioners. Washington DC: American Public Health Association.

• Sorian, R., and T. Baugh. 2002. “Power of Information: Closing the Gap between Research and Policy.” Health Affairs 21 (2): 264–73.


Suggested online resources

• Podcasts on– Reporting one number– Comparing two numbers or series of numbers– Creating effective tables and charts– Interpreting multivariate coefficients– Designing slides for a speech


Suggested practice exercises

• Study guide to The Chicago Guide to Writing about Multivariate Analysis, 2nd Edition.– Questions #1 through 3 in the problem set for

chapter 20– Suggested course extensions for chapter 20

• “Reviewing” exercises #1 through 5• “Writing” exercises #1, 2, 3, 6, 7 and 9• “Revising” exercises #2 and 4


Contact information

Jane E. Miller, [email protected]

Online materials available athttp://press.uchicago.edu/books/miller/multivariate/index.html

Documents

Presenting statistical results to nonstatistical audiences