On the Trail of Improved Cancer Outcomes: An

Outcomes Research Example

Arthur R. Williams, PhD, MA, MPA

Director, Center for Health Outcomes and Health Services

Research

Outcomes Researchers

Health Outcomes Researcher

NurseResearcher

Health Care System

HELP!!

What is Outcomes Research?

“Outcomes research seeks to understand the end results of particular health care practices and interventions. End results include effects that people experience and care about, such as change in the ability to function. … By linking the care people get to the outcomes they experience, outcomes research has become the key to developing better ways to monitor and improve the quality of care. “

-- Agency for Healthcare Research and Quality, March 2000 www.ahrq.gov

Outcomes Research can be viewed as a subarea of Health Services Research. Where Health Services Research is focused “on how people get access to health care, how much care costs, and what happens to patients as a result of this care. The main goals of health services research are to identify the most effective ways to organize, manage, finance, and deliver high quality care; reduce medical errors; and improve patient safety. “

-- Agency for Healthcare Research and Quality, February 2002 www.ahrq.gov also AcademyHealth www.academyhealth.org

A Health Services Research Paradigm circa 2003

Figure 1: An Ideal Type Old-Fashion Clinical Practice Paradigm

The clinician is all heart. Intervention moves the patient from a dark and stormy night to health and happiness if successful. Note the difference, however, between output and outcomes. One can argue that it is the outcome not the output that is truly of Value for the Patient (VFP).

PHYSICIAN

PATIENT OUTPUT

e.g. TEST RESULT

OUTCOME

e.g. QUALITY OF LIFE

Figure 2: An Ideal Type Organized Clinical Practice Paradigm

The model in Figure 2 resembles Figure 1 with an injection of “management.” As a result, “dollars” or the economic costs of health services can loom large. Indeed, a strong emphasis on management can be seen by patients as attempts by providers to limit care and expenditures despite patient perceived needs. Feedback (white arrow) is primarily used as a management tool.

PHYSICIAN

PATIENT

OUTPUT

e.g. TEST

RESULT

OUTCOME

e.g.

QUALITY OF LIFE

MANAGEMENT

Figure 3: An Ideal Type Paradigm (Health Services Research) that is Value for the Patient (VFP) Focused

Comprehensive feedback occurs among patient and clinician about treatment outputs and outcomes (white arrows) and to a management which is output/outcome focused (yellow arrows). Financial considerations are important, but the patient is central. Evaluation of outcomes recognizes the importance of the family and community and that outcomes are different from outputs.

PHYSICIAN/ CLINICIANS

PATIENT

OUTPUT

e.g.TEST RESULT

OUTCOME

e.g. QUALITY OF LIFE

MANAGERS

Health Services Research contains elements of evidence based medicine, clinical epidemiology, psychology, medical sociology, organizational behavior (understudied), quality and safety analyses (industrial engineering), advanced multivariate statistics, and, especially, health economics.

Patient Shared Decision-Making

TENDENCIES IN OUTCOMES RESEARCH

1. Research generally arises from a specific clinical issue or problem .

2. Use (often) of quasi-experimental designs.

3. Use (often) of administrative or existing data sets.

4. Use of complex statistical procedures, which may be consequence of 2 and 3 .

5. Use of measures of both outputs (and process) and outcomes.

6. Use of highly multi-disciplinary teams.

Improving Cancer Outcomes Using the Therapy-Related Symptom Checklist (TRSC) for Adult Oncology Patients

Arthur R. Williams, PhD, MA, MPA

Phoebe D. Williams, PhD, RN, FAAN

TRSC Research Team*

*Team consisted of economist/statistician, nurse researcher, 6 oncology nurses, 3 physicians, computer analyst, biostatistician, nutritionist, and other clinical staff at a Cancer Center within a community hospital system in small town/rural Wisconsin.

Tendency 6: MULITIDISCIPLINARY

TEAM

Initial Reason for Study of Symptoms

(1984-1986) 3 Advanced Practice Nurses at the University of Florida believed under-documentation of patient symptoms was impeding symptom management and patient recovery.

Researched and published : 1.8 symptoms per patient documented in medical records yet patients reported 11.0 symptoms on average of concern to them. Youngblood M, Williams PD, Eyles H, Waring J, Runyon S. (1994). A comparison of two methods of assessing cancer therapy-related symptoms. Cancer Nursing; 17(1): 37-44.

Tendency 1: CLINCAL ISSUE

FOLLOW-UP: Calibration, Testing, and Development of a 25-Item Therapy-Related Symptom Checklist (TRSC) for

Adult Oncology PatientsTHERAPY-RELATED SYMPTOMS CHECKLIST (TRSC)

Name:__________________________________ Hospital #________ Date:_______ PLEASE CHECK THE PROBLEMS YOU HAVE HAD IMMEDIATELY AFTER AND SINCE YOUR LAST TREATMENT. PLEASE CIRCLE HOW SEVERE THE PROBLEM WAS ACCORDING TO THE FOLLOWING SCALE: 0 = NONE 1 = MILD 2 = MODERATE 3 = SEVERE 4 = VERY SEVERE EXAMPLE Degree of Severity Pain ................................................................................................. 0 1 2 3 4 Taste Change ................................................................................... 0 1 2 3 4 Loss of appetite ................................................................................ 0 1 2 3 4 Nausea ............................................................................................. 0 1 2 3 4 Vomiting.......................................................................................... 0 1 2 3 4 Weight loss ...................................................................................... 0 1 2 3 4 Sore mouth....................................................................................... 0 1 2 3 4 Cough .............................................................................................. 0 1 2 3 4 Sore throat ....................................................................................... 0 1 2 3 4 Difficulty swallowing....................................................................... 0 1 2 3 4 Jaw pain........................................................................................... 0 1 2 3 4 Shortness of breath ........................................................................... 0 1 2 3 4 Numbness in fingers and/or toes ....................................................... 0 1 2 3 4 Feeling sluggish ............................................................................... 0 1 2 3 4 Depression ....................................................................................... 0 1 2 3 4 Difficulty concentrating.................................................................... 0 1 2 3 4 Fever................................................................................................ 0 1 2 3 4 Bruising ........................................................................................... 0 1 2 3 4 Bleeding........................................................................................... 0 1 2 3 4 Hair loss........................................................................................... 0 1 2 3 4 Skin changes .................................................................................... 0 1 2 3 4 Soreness in vein where chemotherapy was given............................... 0 1 2 3 4 Difficulty sleeping............................................................................ 0 1 2 3 4 Pain ................................................................................................. 0 1 2 3 4 Decreased interest in sexual activity ................................................. 0 1 2 3 4 Constipation..................................................................................... 0 1 2 3 4 Other problems (please list) 0 1 2 3 4 ______________________............................................................... 0 1 2 3 4 ______________________............................................................... 0 1 2 3 4 ______________________............................................................... 0 1 2 3 4 ______________________............................................................... 0 1 2 3 4 Phoebe D. Williams, PhD ©Copyright 1995 University of Kansas Medical Center

Many publications; Some summarized in Barry MJ, Dancey JE (2005). Instruments to measure the specific health impact of surgery, radiation, and chemotherapy on cancer patients (pp.201-215). In: Lipscomb, J., Gotay CC, Snyder C (Eds.), Outcomes Assessment I n Cancer: Measures, Methods, and Applications. Cambridge University Press, London.

Objective of the Study by the Wisconsin Team: To Determine in an Outpatient Setting If:

Documentation of patient reported symptoms, management of these symptoms, and HRQOL can be improved through use of the TRSC.

H1: A treatment cohort using the TRSC at clinic visits will show a statistically significant positive increase in HRQOL-LASA compared to a treatment cohort receiving standard of care.

H2: A treatment cohort using the TRSC during clinic visits will show a statistically significant larger number of symptoms documented and managed compared to a treatment cohort receiving standard of care.

Translational Research

Quasi-Experimental Design of Study

Sequential non-equivalent cohort design: early cohort serves as study “control” for later “treatment” cohort. Ref: Happ MB, Sereika S, Garrett K, Tate J. Use of the quasi-experimental sequential cohort design in the study of patient–nurse effectiveness with assisted communication strategies (SPEACS). Contemporary Clinical Trials 2008 29:801-808.

Statistical analysis using generalized estimating equations (GEE): additional strength through use of repeated measure panel. Ref: Twisk JWR. Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide. Cambridge: Cambridge University, 2003.

Tendency 2: QUASI-EXPERIMENTAL DESIGN

Tendency 4: COMPLEX STATISTICS

Demographics of Study SubjectsOf the 128 subjects on which data were collected 2 were excluded for having only one visit and 13 were not able to be staged for their cancer. Stage was considered a critical covariate in the two GEE. Exclusion of these subjects left 113 for the analysis in this paper with 696 observations. The mean number of observations per patient was 5.2. Final “Control” Group n=55 subjects; Final “Treatment” Group n=58 subjects.

No statistically significant differences between groups on: Age: Gender; Marital Status; Education; Presence of Significant Other; Stage; Diagnoses; Radiation; Chemo; Both Radio-Chemo.

Median Age = 61; Married: 79%; Education: 36% HS or Less; 47% HS+; 16% BS/BA+; Race: 96% Caucasian. Profile reflects populations in small town/rural Upper Midwest.

Results of GEE Analysis of HRQOL-LASA on Covariates

Response: HRQOL Corr: Exchangeable

Variable Name Coefficient StErr p-value

constant 16.7528 7.7564 0.031

Baseline QOL .7083 .0829 < 0.001

Treatment Group 3.3144 1.3152 0.012 .

Education -.8092 .7233 0.263

Age .0090 .0673 0.893

Male -3.2381 1.5391 0.035

Significant Other .1893 1.5507 0.903

Stage .6205 .7723 0.422

Radio -2.6490 1.5413 0.086

Chemo -3.6356 2.3372 0.120

Days from Baseline -.0166 .0109 0.129

----------------------------------------------------------------------------------------------------

#observations: 583 scale parameter: 62.3707 #iterations: 4

Estimate of common correlation 0.6095

MAIN FINDING

Results of GEE Analysis of Total Symptoms on Covariates

Response: Number of Symptoms Corr: Exchangeable

Variable Name Coefficient StErr p-value

constant 2.5747 2.2991 0.263

Treatment Group 3.7597 .5939 < 0.001 .

Education -.1588 .2531 0.344

Age -.0310 .0328 0.893

Male -.7299 .6698 0.276

Significant Other .0226 .6699 0.973

Stage .7590 .3482 0.029

Radio .0767 .4155 0.853

Chemo 1.2776 .6814 0.061

Days from Baseline .0005 .0031 0.870

Days × Treatment Group -.0151 .0072 0.037

----------------------------------------------------------------------------------------------------

#observations: 696 scale parameter: 13.5982 #iterations: 5

Estimate of common correlation 0.6977

Main Finding

Important “Process” Finding

Tendency 6: MEASURES OF OUTPUTS AND OUTCOMES

RESULTS & CONCLUSIONS

Treatment group patients had a 7.2% higher population averaged covariate adjusted HRQOL than “control” patients, (3.3 more points on the HRQOL, P = .012.) 116% more covariate and non-covariate adjusted symptoms were documented and managed in the “treatment “group than in the “control” group, (6.14 symptoms versus 2.84, P < .0001). Despite low sensitivity to change, a 2.0 point mean difference improvement was found in Karnofsky scores in the “treatment “group versus the “control” group, (P < .01). The HRQOL, TRSC, and Karnofsky scores correlated r > .40.

Use of the patient-friendly TRSC by patients and clinicians improves HRQOL. Documentation and management of symptoms also are improved.

Tendency 3:

ADMINISTRATIVE DATA SETS

DISCUSSION OR POP QUIZ

Which of the tendencies, if any, of OUTCOMES RESEARCH are not illustrated in this presentation?

What in the table is not a measure of OUTCOME?

Measure Example

Mortality Infant death rate

Physiologic measures Blood pressure

Clinical events Stroke

Symptoms Difficulty breathing

Functional measures SF-36, a 36-item health survey

Patients' experiences with care Consumer Assessment of Health Plans survey