Funded by HRSA HIV/AIDS Bureau Selecting an Indicator & Collecting Performance Data Barbara M Rosa, RN-C, MS

Funded by HRSAHIV/AIDS Bureau

Selecting an Indicator & Collecting Performance Data

Barbara M Rosa, RN-C, MS

National Quality Center (NQC)2

Objectives

Understand the criteria in selecting an indicator for improvement

Develop an indicator with numerator & denominator for a network Part D program or a single site Part D program

Identify data sources and develop a data collection plan &/or manual data collection tool

Understand sampling methodology


Agenda

15 min—Overview of indicator development and data display/analysis

15 min—Individual programs develop an appropriate indicator with numerator & denominator defined.

10 min—Share indicators with group 10 min—Breakout in 2 or 3 groups: networks,

single sites, and/or programs with electronic data collection to develop a data collection plan.

10 min—Share plans with group


What Is A Quality Indicator?

A quality indicator is tool to measure specific aspects of care and services that are optimally linked to better health outcomes while being consistent with current professional knowledge and meeting client needs.


What Makes a Good Indicator?

Relevance Does the indicator affect a lot of people or programs? Does the indicator have a great impact on the

programs or patients/clients in your network or clinic?

Measurability Can the indicator realistically and efficiently be

measured given finite resources?


What Makes a Good Indicator?

Accuracy/Validity Is the indicator based on accepted guidelines or

developed through formal group-decision making methods?

Improvability Can the performance rate associated with the

indicator realistically be improved given the limitations of your services and population?


Define Indicator

Who is eligible to be evaluated? What part of this population should have

received the care being measured?

(Who should be counted in the denominator?) What part of those who should have received

the care did receive the recommended care? (Who should be counted in the numerator?)


Example

Eligibility definition by the National HIVQUAL Project: HIV+ ambulatory patients with at least 2 medical visits

during the study period (i.e., 1/1/08 through 12/31/08)

with at least one visit in each six-month period (January-

June/July-December)


Examples

Location: all sites, or only some? Gender: men, women, or both? Age: any limits? Client conditions: all HIV-infected clients, or only

those with a specific diagnosis? Treatment status?


Base the indicator on guidelines and standards of care when possible

Include staff and consumers when

developing an indicator to create ownership Be clear in terms of patient / program characteristics

(gender, age, patient condition, provider type, etc.) Set specific time-frames in indicator definitions

Tips for Defining Indicators


Types of Measures Outcome measures are the voice of the

customer

Outcome measures reflect the impact of our work on the care provided to our clients

Examples: CD4/VL, number of medical provider visits, incidence of disease rates, functional status indicators, mortality & morbidity rates, client/care team satisfaction rates, or cost of care delivered


Types of Measures Process measures are the voice of

the system

Process measures tell us whether parts or steps in the system are working as planned

Examples: Waiting time for appointments, no show rates, percent of clients with self-management goals in place, the number of clients organized by acuity level in case management


Types of Measures

Balance measures are used to monitor the effect one change might have on multiple parts of the system

Example: Mental Health screening & referral is improved. To prevent a potential negative outcome of the increase in referrals leading to lengthy waits for appointments, a balancing measure might be to monitor time between the referral and the mental health appointment


Collecting & Managing Data

“Data In” characteristics: Data that requires manual chart

review should be completed as efficiently as possible (pull the chart only once)

Include the data collectors (e.g., HIV case managers) in the feedback process: this validates their work and improves data quality

Streamline potential duplication of data entry by systematic alignment


Collecting & Managing Data

“Data out” characteristics: Data should be displayed clearly; graphs and

charts are effective means Data should be shared throughout the system

along with program goals and improvement activities

Data should be evaluated and acted upon as outlined in the QM Plan

Share data, as appropriate, with consumers, partners, funders, and other stakeholders

Use quality improvement data to inform operational and strategic planning efforts; alignment within the organization and system is smart business


Constructing A Sample--Definitions

Sampling: allows teams to make inferences about a total population based on observations of a smaller subset of that group

Over sampling: may be required if measurement pertains exclusively to one group within a population


Constructing A Sample--Methodology

Define the selection criteria Gender Age Patient condition Treatment status

Identify eligible cases Randomly select cases


Constructing A Sample--Sampling Chart

Total Eligible Population

Minimum Total Records

Charts to Pull*

140-159 64 83

160-179 67 87

180-199 70 91

200-249 75 98

250-299 79 103

300-349 82 107

350-399 85 111


Sampling Methodology

Create a numbered list of all eligible patients Go to web site: www.randomizer.org Click on link to Randomize Put in total number of eligible patients, e.g. 200 Put in number of sample size, e.g., 41 Click randomize now, and you will get a list of

41 random numbers between 1 and 200 You may have to do women and men separately

because of over sampling for women


www.randomizer.org


• How many sets of number would you like?• How many numbers per set?• Number range? From

To• Do you wish each number in a set to remain unique?• Do you wish to sort your outputted numbers?• How do you wish to view the output? • Randomize Now

www.randomizer.org

41

1

200

1

YES

NO

Place Markers Off


Contact Information

National Quality Center (NQC)

212-417-4730212-417-4730NationalQualityCenter.orgNationalQualityCenter.org

[email protected]

HIVQUAL-US

212-417-4620212-417-4620HIVQUAL.orgHIVQUAL.org

[email protected]

Documents

Funded by HRSA HIV/AIDS Bureau Selecting an Indicator & Collecting Performance Data Barbara M Rosa, RN-C, MS