1 04/21/23

Evaluating the Significance of Health-Related Quality of Life

Change in Individual Patients

Ron Haysdrhays@ucla.edu

October 8, 2004

UCLA GIM/HSR Research Seminar Series

2 04/21/23

Motivation

Interest in knowing how many patients benefit from group intervention

Tracking progress on individual patients

3 04/21/23

UCLA Center for East-West Medicine

Staff training in biomedicine and traditional Chinese medicine.

Treat chronic pain conditions (e.g., neck and back pain, headaches, fibromyalgia, sports and occupational-related injuries), and cancer-related symptoms.

Provide comprehensive care, emphasizing health promotion, disease prevention, treatment and rehabilitation.

Patient education, medication adjustments, trigger point injections, acupuncture, acupressure, therapeutic massage, dietary and herbal counseling, and mind-body exercises.

4 04/21/23

Based on Submitted Paper

Ron D. Hays, Marc Brodsky, M. Francis Johnston, Karen L. Spritzer, Ka-Kit Hui

Evaluating Health-Related Quality of Life Change in Individual Patients

Submitted to Evaluation and the Health Professions

5 04/21/23

Methods

54 patients

Average age = 56; 84% white; 58% female

Self-administered SF-36 version 2 at baseline and about at end of therapy (about 6 weeks later).

6 04/21/23

SF-36 Version 2

• Physical functioning (10 items)

• Role limitations/physical (4 items)

• Pain (2 items)

• General health perceptions (5 items)

• Social functioning (2 items)

• Energy/fatigue (4 items)

• Role limitations/emotional (3 items)

• Emotional well-being (5 items)

7 04/21/23

Scoring the SF-36

Average or sum all items in the same scale.

Transform average or sum to

• 0 (worse) to 100 (best) possible range

• z-score (mean = 0, SD = 1)

• T-score (mean = 50, SD = 10)

T-score = 50 + (z-score * 10)

8 04/21/23

Internal Consistency Reliability Formula

BMS

EMSBMS

MS

MSMS

MSMSBMSBMS = mean square between, MS = mean square between, MS

EMSEMS = mean square = mean square

for interaction between respondents and for interaction between respondents and items. items.

9 04/21/23

SF-36 Reliability Estimates

ScaleScale AlphaAlpha

Physical functioningPhysical functioning 0.940.94

Role limitations--physical Role limitations--physical health problems health problems

0.930.93

Pain Pain 0.870.87

General health perceptionsGeneral health perceptions 0.830.83

Social functioning Social functioning 0.850.85

Energy/fatigue Energy/fatigue 0.770.77

Role limitations--emotional Role limitations--emotional health problems health problems

0.940.94

Emotional well-being Emotional well-being 0.790.79

10 04/21/23

Physical Health

Physical functionPhysical function

Role function-physical

Role function-physical

PainPain General Health

General Health

Physical Health

11 04/21/23

Mental Health

Emotional Well-Being

Emotional Well-Being

Role function-emotional

Role function-emotional

EnergyEnergy Social functionSocial

function

Mental Health

12 04/21/23

SF-36 Version 2 PCS and MCS

PCS_z = (PF_z * .42402) + (RP_z * .35119) + (BP_z * .31754) + (GH_z * .24954) + (EF_z * .02877) + (SF_z * -.00753) + (RE_z * -.19206) + (EW_z * -.22069)

MCS_z = (PF_z * -.22999) + (RP_z * -.12329) + (BP_z * -.09731) + (GH_z * -.01571) + (EF_z * .23534) + (SF_z * .26876) + (RE_z * .43407) + (EW_z * .48581)

13 04/21/23

Formula for Reliability of Composite

Mosier 1( j

2w )( j

2S ) ( j2w )( j

2S )( j )( j

2w )( j

2S ) 2( jw )( Kw )( jS )( KS )( jKr )

jw weight given to component J

Kw weight given to component K

jS standard deviation of J

j reliability of J

jKr correlation between J and K

14 04/21/23

Reliability of SF-36 Summary Scores

SF-36 PCS = 0.94

SF-36 MCS = 0.93

15 04/21/23

Analysis Plan

* Comparison on SF-36 physical functioning and emotional well-being scale scores with other samples

* Significance of:

- Within group change

- Within individual change

16 04/21/23

t-test for within group change

XXDD/(SD/(SDdd/n /n ½½))

XXDD = is mean difference, SD = is mean difference, SDd d = standard deviation of difference= standard deviation of difference

17 04/21/23

Formulas for Significance of Individual Change

Standard error of measurement Standard error of measurement (SEM)(SEM)

SDSDb b * (1- reliability)* (1- reliability)1/21/2

Standard error of prediction (SEp) Standard error of prediction (SEp) SDSDbb * (1- reliability * (1- reliability22))1/21/2

SEM CI around Time 1 scoreSEM CI around Time 1 score Time 1 +- 2 SEMTime 1 +- 2 SEM

SEp CI around Time 1 scoreSEp CI around Time 1 score Time 1 +- 2 SEp Time 1 +- 2 SEp

SDSDbb = standard deviation at baseline = standard deviation at baseline

18 04/21/23

Estimated True Score for Score of 60

Mean + reliability (score – mean)

50 + 0.90 (60 – 50) = 59

19 04/21/23

Reliable Change Index

X2 – X1/ SEM * SQRT (2)

20 04/21/23

Formulas for Significance of Individual Change

SEM 95% CISEM 95% CI 1.96 * SD1.96 * SDb b * (1- reliability)* (1- reliability)1/21/2

SEp 90% CISEp 90% CI 1.64* SD1.64* SDbb * (1- reliability * (1- reliability22))1/21/2

SEp 95% CISEp 95% CI 1.96* SD1.96* SDbb * (1- reliability * (1- reliability22))1/21/2

Estimated true scoreEstimated true score Mean + reliability (score – mean)Mean + reliability (score – mean)

Reliable change indexReliable change index XX22-X-X11// .2SEM

SDSDbb = standard deviation at baseline = standard deviation at baseline

21 04/21/23

Minimum Delta for Individual Significance

SEM: > 1.96 SEM

RCI: > 1.96 * SQRT (2) * SEM

SEp-90: > 1.64 SEp; SEp-95: > 1.96 SEp

22 04/21/23

Physical Functioning and Emotional Well-Being at Baseline Physical Functioning and Emotional Well-Being at Baseline

for 54 Patients at UCLA-Center for East West Medicinefor 54 Patients at UCLA-Center for East West Medicine

0 10 20 30 40 50 60

Asymptomatic

Symptomatic

AIDS

General Pop

Epilepsy

GERD

Prostate disease

Depression

Diabetes

ESRD

MS

East-WestEWBPhysical

Hays et al. (2000), American Journal of Medicine

23 04/21/23

Change in SF-36 Scores Over Time

0

5

10

15

20

25

30

35

40

45

50

PF10 Role-P Pain Gen H Energy Social Role-E EWB PCS MCS

Baseline

Followup

0.13 0.35 0.35 0.21 0.53 0.36 0.11 0.41 0.24 0.30

Effect Size

24 04/21/23

Significance of Group ChangeDelta t-test prob.

PF-10 1.7 2.38 .0208

RP-4 4.1 3.81 .0004

BP-2 3.6 2.59 .0125

GH-5 2.4 2.86 .0061

EN-4 5.1 4.33 .0001

SF-2 4.7 3.51 .0009

RE-3 1.5 0.96 .3400 <-

EWB-5 4.3 3.20 .0023

PCS 2.8 3.23 .0021

MCS 3.9 2.82 .0067

25 04/21/23

Amount of Change in Observed Score Needed for Significant Change

SEM 90% SEp

95% SEp

RCI Effect size

PF-10 5.9 6.9 8.2 8.4 0.47-0.67

RP-4 6.0 6.9 8.3 8.4 0.52-0.72

BP-2 7.4 8.4 10.1 10.4 0.72-1.01

GH-5 9.2 10.4 12.5 13.0 0.80-1.13

EN-4 9.0 10.1 12.0 12.8 0.94-1.33

SF-2 9.8 11.1 13.3 13.8 0.76-1.07

RE-3 6.8 8.0 9.5 9.7 0.50-0.71

EWB-5 9.5 10.6 12.7 13.4 0.90-1.26

PCS 5.0 5.9 7.0 7.1 0.43-0.62

MCS 6.9 8.0 9.5 9.7 0.52-0.73

26 04/21/23

Proportion of 54 Cases Declining SignificantlySEM SEp 90 SEp 95 RCI

PF-10 9% 7% 2% 2%

RP-4 7% 6% 2% 2%

BP-2 17% 11% 9% 7%

GH-5 4% 0% 0% 0%

EN-4 4% 4% 2% 2%

SF-2 13% 11% 6% 4%

RE-3 19% 19% 15% 15%

EWB-5 9% 6% 6% 4%

PCS 7% 7% 7% 7%

MCS 13% 11% 11% 11%

27 04/21/23

Proportion of 54 Cases Improving SignificantlySEM SEp 90 SEp 95 RCI

PF-10 19% 15% 13% 13%

RP-4 35% 31% 30% 31%

BP-2 31% 28% 24% 22%

GH-5 9% 7% 7% 7%

EN-4 24% 17% 11% 9%

SF-2 30% 20% 17% 17%

RE-3 24% 19% 15% 15%

EWB-5 26% 20% 19% 19%

PCS 33% 30% 24% 24%

MCS 37% 30% 22% 22%

28 04/21/23

% Improved – % Declined SEM SEp 90 SEp 95 RCI

PF-10 10% 8% 11% 11%

RP-4 28% 25% 28% 29%

BP-2 14% 17% 15% 15%

GH-5 5% 7% 7% 7%

EN-4 11% 13% 9% 7%

SF-2 11% 9% 11% 13%

RE-3 5% 0% 0% 0%

EWB-5 19% 14% 13% 15%

PCS 26% 23% 17% 17%

MCS 24% 19% 11% 11%

29 04/21/23

Questions

30 04/21/23

Bibliography

Bauer, S., Lambert, M. J., & Nielsen, S. L. (2004). Clinical significance methods: A comparison of statistical techniques. Journal of Personality Assessment, 82, 60-70.

Dudek, F. J. (1979). The continuing misinterpretation of the standard error of measurement. Psychological Bulletin, 86, 335-337.

Ferguson, R. J., Robinson, A. B., & Splaine, M. (2002). Use of the reliable change index to evaluate clinical significance in SF-36 outcomes. Quality of Life Research, 11, 509-516.

Hsu, L. M. (1989). Reliable changes in psychotherapy: Taking into account regression toward the mean. Behavioral Assessment, 11, 459-467.

Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Toward a standard definition of clinically significant change. Behavior Therapy, 17, 308-311.

Speer, D. C., & Greenbaum, P. E. (1995). Five methods for computing significant individual client change and improvement rates: Support for an individual growth curve approach. Journal of Consulting and Clinical Psychology, 63, 1044-1048.