Performance Improvement Projects (PIPs)

Technical Assistancefor Florida Medicaid PMHPs

August 21st, 2007

Christy Hormann, MSWPIP Review Team Project Leader

Purpose of the Webinar

• Provide Technical Assistance for Activities either Partially Met or Not Met overall by PMHPs for the 2006-2007 validation cycle.

• How to submit PIPs.

• Provide resources.

• Address PIP questions and issues.

Presentation Outline

• Review of PIP Activities II, III,VIII, and IX and Examples

• PIP submissions for the 2007-2008 validation cycle

• Questions and Answers

Activity II: The Study Question

HSAG Evaluation Criteria

• States the problem to be studied in simple terms.

• Is answerable.

In general, the question should illustrate the point of: Does doing X result in Y?

Study Question Example

Will the implemented interventions, to improve the percent of new children with serious emotional disturbance starting any needed ongoing service within 14 days of a non-emergent assessment with a professional, result in a higher percent of new children with serious emotional disturbance starting any needed ongoing service within 14 days of a non-emergent assessment with a professional?

Study Question Example

Does providing members with needed follow-up care, and increasing the percentage of discharges with follow-up appointments, improve the rate of members receiving follow-up care within 7 days after discharge from an inpatient stay for a mental illness?

Activity III: Selected Study Indicators

HSAG Evaluation Criteria

The study indicator(s):• Is well defined, objective, and measurable.• Is based on practice guidelines, with sources

identified.

Activity III: Selected Study Indicators

HSAG Evaluation Criteria (cont.)

The study indicator(s):• Allows for the study question to be answered.• Measures changes (outcomes) in health or

functional status, member satisfaction, or valid process alternatives.

Activity III: Selected Study Indicators

HSAG Evaluation Criteria (cont.)

The study indicator(s):• Has available data that can be collected on

each indicator.• Is a nationally recognized measure such as

HEDIS®, when appropriate.• Includes the basis on which each indicator

was adopted, if internally developed.

HEDIS® is a registered trademark of the National Committee for Quality Assurance (NCQA).

Study Indicator Example

Study Indicator Describe the rationale for the selection of the study indicator: Based on the expertise of the State and the Quality Improvement Committee, the indicator was chosen due to a high number of PIHPs having difficulty meeting the department’s established standard.

Numerator The number of new children with serious emotional disturbance starting any needed ongoing service within 14 days of a non-emergent assessment with a professional.

Denominator All new children with serious emotional disturbance starting any needed ongoing service after a non-emergent assessment with a professional.

First Measurement Period Dates

January 1, 2006 – December 31, 2006

Benchmark 95% within 14 days based on the state average (91.1%)

Source of Benchmark

State based standard

Baseline Goal 95% within 14 days

Study Indicator ExampleStudy Indicator Describe the rationale for the selection of the study indicator:

Because of the high risk to both members and the health plan, XYZ has attempted to improve the rate of follow-up visits post-hospitalization and thereby prevent acute exacerbations from recurring by improving timely and appropriate coordination between systems of care, in the transition from inpatient to outpatient levels of treatment. XYZ calculates NCQA’s HEDIS measure of Follow-Up After Hospitalization for Mental Illness at 7 days to monitor progress in this area annually.

Numerator Number of ambulatory or intermediate encounters with a mental health practitioner up to 7 days after a psychiatric hospital discharge of an eligible member.

Denominator Number of hospital discharges on or before December 1st of the calendar year for all eligible members 6 years of age or older as of the date of discharge.

First Measurement Period Dates

Discharge from January 1 through December 1 with a date of service through December 31 of the measurement year.

Benchmark 50th percentile: 38.4%

Source of Benchmark

National Medicaid HEDIS 2005

Baseline Goal HEDIS 50th percentile for the measurement calendar year

HSAG Evaluation CriteriaThe data analysis:• Is conducted according to the data analysis plan in the study design.• Allows for generalization of the results to the study population if a sample was selected.• Identifies factors that threaten internal or external validity of findings.• Includes an interpretation of findings.

Activity VIII: Data Analysis and Interpretation of Study Results

HSAG Evaluation Criteria (cont.) The data analysis:• Is presented in a way that provides accurate, clear, and easily understood information.• Identifies initial measurement and remeasurement of study indicators.• Identifies statistical differences between initial measurement and remeasurement.• Identifies factors that affect the ability to compare initial measurement with remeasurement.• Includes the extent to which the study was successful.

Activity VIII: Data Analysis and Interpretation of Study Results

Data Analysis Plan Example

Data analysis process:

The data will be collected and reported on an annual basis.

Statistical analysis will be employed to determine whether

statistically significant progress is occurring.

The resulting baseline and remeasurement proportions will be compared using a chi-square to determine if the difference was statistically significant, with p<.05 as the generally accepted standard for significance.

Activity VIII: Interpretation of Study Results Example

Baseline Interpretation

The baseline rate was 14.1% of members having had follow-up within 7 days after a discharge from an inpatient stay for mental illness during the time period 1/1/03-12/31/03.

Activity VIII: Interpretation of Study Results Example

Remeasurement 1 Interpretation

The baseline rate of 14.1% of members who had follow-up within 7 days after a discharge from an inpatient stay for mental illness increased to 21.7% for the first remeasurement (1/1/04-12/31/04). This represents a statistically significant increase (p=.005).

Activity VIII: Interpretation of Study Results Example

Remeasurement 2 Interpretation

• The second remeasurement rate increased to 27.6% from 21.7%. The increase was not statistically significant (p=0.125).

• The rate of members who had follow-up within 7 days of a discharge has increased each year, with no decline in performance.

• The PIP has been successful in increasing follow-up after discharge and will be continued. Third remeasurement data will be collected to assess for sustained improvement.

Activity VIII: Interpretation of Study Results Example

• Factors that threatened the validity of the study included acquiring ABC’s health plan members and a change in staff members during the data collection phase of the study.

• Acquiring ABC’s health plan members also affected the ability to compare initial measurement with remeasurement.

* If there were no factors that threatened the validity of the study or affected the ability to compare measurements, include that in

the documentation.

Activity XI: Real Improvement Achieved

HSAG Evaluation Criteria• The remeasurement methodology is the

same as the baseline methodology.• There is documented improvement in

processes or outcomes of care.• The improvement appears to be the result of

intervention(s).• There is statistical evidence that observed

improvement is true improvement.

Activity IX: Study Summary Results and ImprovementExample

#1 Quantifiable Measure: Ambulatory or Intermediate Follow-Up After An Inpatient Stay Within 7 Days

Time Period Measurement

Covers

Baseline Project Indicator

Measurement Numerator Denominator Rate or

Results Industry

Benchmark Statistical Test and

Significance*

CY 2003 BASELINE 62 440 14.1% 38.7% N/A

CY 2004 Remeasurement 1: 136 627 21.7% 40.3% p=0.005*

CY 2005 Remeasurement 2: 227 822 27.6% 39.2% p=0.125

CY 2006 Remeasurement 3: 240 740 32.4% 39.2% p=0.248

New PIP Submissions

• New PIPs were not submitted to HSAG for the 2006-2007 validation cycle.

• For new PIP submissions, it is important to contact HSAG to obtain the most current PIP Summary Form.

How to Submit Continuing PIPs

• On-going PIPs are those that were submitted to HSAG for the 2006-2007 validation cycle.

• Highlight, bold, or add text in a different color, and date any new information that is added to the existing PIP Summary Form.

• Strikethrough and date any information that no longer applies to the PIP study submission.

• Ensure all Partially Met and Not Met evaluation elements from the previous validation cycle have been addressed in the documentation.

PIP Tips

1. Complete the demographic page before submission.2. Notify HSAG when the PIP documents are uploaded to the secure FTP site and state the number of documents uploaded. 3. Label ALL attachments and reference them

in the body of the PIP study.4. HSAG does not require personal health

information to be submitted. Submit only aggregate results.

5. Document, document, and document!!6. Look for a frequently asked questions on myfloridaeqro.com.7. If you have additional questions, contact HSAG.

Deliverables

September 7th: PMHPs notified electronically of submission date with

instructions

October 5th: Submit PIP studies to HSAG

HSAG will be validating two PIPs per PMHP; one clinical and one nonclinical. If the collaborative PIP is clinical, the other PIP chosen for validation will be nonclinical.

Resources

• Frequently asked Questions and PIP information myfloridaeqro.com

• NCQA Quality Profiles http://www.qualityprofiles.org/index.asp

• Institute for Healthcare Improvement – www.ihi.org• Center for Healthcare Strategies – www.chcs.org• Health Care Quality Improvement Studies in

Managed Care Settings – A Guide for State Medicaid Agencies www.ncqa.org/publications

• National Guidance Clearinghouse www.guidelines.gov

• Agency for Healthcare Research and Quality http://www.ahrq.gov/

HSAG Contacts

For questions contact:Cheryl Neel

– cneel@hsag.com– 602.745.6201

Denise Driscoll– ddriscoll@hsag.com– 602.745.6260

Questions and Answers