Acting on Your NSQIP Data

What it’s All aboutPeter DorisSQAN

Responding to Your Data

Identify areas for improvement

Gather the Team

Develop a Plan

Institute Changes

Monitor Outcomes

Revise the plan

Keep Going! (PDSA)

Using Non Risk Adjusted Data

• Can Do All Steps Outlined• Data nearly “Real Time”• Clearly “Our Data”• Positive Feedback encourages the Team

There are problems thoughCommon cause variationUnstable process?My patients are different!!

Using Risk Adjusted Data

• Identify Areas needing Improvement• Minimizes the “my patients are different” discussion• Clarifies true improvement• Stimulates our competitive instincts

– If they can do it – we can too!– I don’t like being on the wrong end of the curve

NSQIP SAR

• “Gold Standard” of Surgical Outcomes• “Validates” improvements

• However:– Only received twice a year– Data is at least 6 months old– Not helpful for PDSA cycles

BC Risk Adjustment for NSQIP?

• More frequent reports• Compare just Canadian (BC) Outcomes

• However– We need a Validated Model!

LOCAL MODEL INFORMATION

Purpose

A local risk-adjusted model would allow British Columbian sites to:• a) identify local predictors, which could then be

used to compare across North America; that is, are BC patients different, and if so, in what ways?

• b) receive timely reporting, about three months ahead of NSQIP’s SAR; it would serve as a supplement to, not a replacement for, the full SAR.

• c) carry out direct comparison of local sites more easily, while still retaining site anonymity.

• d) develop a consistent province-wide reporting template that meets the needs locally. It will be more responsive and faster than NSQIP, will allow identification and sharing of best practices, and it thus is anticipated to lead to improved outcomes.

What will it look like?

• The local model will use O/E ratios – similar to the SAR previous SAR (NSQIP now uses odds ratios)

• Sites will only see their own risk adjusted information

• Comparison information will maintain anonymity of the other local sites

Model Validation

• The results of the risk adjusted local model will be compared with the NSQIP SAR to see that the local model produces similar results

• NSQIP does not disclose some information about their modelling process so we are not able to replicate their process exactly

• But we do know what NSQIP considers when they create their model

What is needed from the SCR?

• Raw data will be download for designated time periods and submitted securely

• SCR time investment is minimal• Feedback from the data analysis group around data

cleanliness may lead to improved data integrity and reliability

How will this help the SCR?

• More frequent risk adjusted data will provide confirmation of trend noticed in the non-risk adjusted data

• QI projects will have more consistent evaluation especially when teams are waiting on risk adjusted data to make decisions – feedback is vital to keep projects moving forward

• Less ‘wait and see’ time• Risk adjustment may improve ‘buy in’ from staff and

administrators

WHERE ARE WE NOW?

PIA

• There was an addendum created for the NSQIP PIA to address the local model

• The Local Model addendum has been approved• Some sites still need to sign the revised letter of

agreement

Update Timeline

• January 2013: Sites submit data to BCPSQC for July 1 – June 30, 2012, which is the same data submitted to ACS to produce the SAR. BCPSQC will use this data to evaluate the BC Local Risk-Adjusted Report by comparing it to the February 2013 SAR.

• March 2013: Sites submit data for January 1, 2012 – December 31, 2012 and this data will be used to produce a BC Local Risk-Adjusted Report to be released in April 2013 if the initial test report was successful in comparison to the ACS NSQIP SAR. Subsequent reports will be released bi-annually if the statistical model is successful.