But What Do These Numbers Represent?!?

Quality Forum 2016

Outline

• Purposes of measurement in health care

• Sources of data and their characteristics

• Key questions to ask of any indicators

• Some examples of indicators

Source: Solberg LI, Moser G, McDonald S. (1997) The three faces of performance measurement: Improvement, Accountability and Research. Journal of Quality Improvement, 23(3).

Source: Solberg LI, Moser G, McDonald S. (1997) The three faces of performance measurement: Improvement, Accountability and Research. Journal of Quality Improvement, 23(3).

Source: Solberg LI, Moser G, McDonald S. (1997) The three faces of performance measurement: Improvement, Accountability and Research. Journal of Quality Improvement, 23(3).

Source: Solberg LI, Moser G, McDonald S. (1997) The three faces of performance measurement: Improvement, Accountability and Research. Journal of Quality Improvement, 23(3).

Measurement systems

• It is not ideal to use a measurement system for a purpose other than for which it was designed.

Measurement source examples

• Manual data collection on a local unit

• EMR extract about the patient panel of a clinic

• Discharge abstract database

• Patient questionnaires

Data Quality

1. Accuracy

2. Timeliness

3. Comparability

4. Usability

5. Relevance

Canadian Institute for Health Information, The CIHI Data Quality Framework, 2009 (Ottawa, Ont.: CIHI, 2009)

Considerations for decision makers

1. Accuracy

2. Timeliness

3. Comparability

4. Usability

5. Relevance

Canadian Institute for Health Information, The CIHI Data Quality Framework, 2009 (Ottawa, Ont.: CIHI, 2009)

Key questions

–How was this data collected?

–How is the indicator defined and calculated?

–How are we performing?

How was this data collected?

• Why was it collected?

• What is the source?

• What is missing?

• What are the sources of error?

How is the indicator defined and calculated?

• Inclusions and exclusions?

• Risk adjustment?

• Aggregation?

How are we performing?

• Are we getting better or worse?

• Is our performance acceptable?

• Is the performance stable?

• Do we have more or less variability?

Hip Fracture within 48 hours

• Data source: DAD

• Level of aggregation – HA

• Inclusion – 65+

• Calculation – within 48/total number of fractures

• Performance – 80ish percent

• Data source: DAD

• Level of aggregation – HA

• Exclusion – Newborn, obstetric, and mental health

• Calculation – Number of any of the events per 1000 discharge

• Performance – 25ish percent

Nurse sensitive adverse event rate