Partnering to improve data quality and birth outcomes

Extending our reach through partnershipsJune 2-6, 2013, Phoenix AZ

From Good to Great – Collaborating to Advance Data Quality

The Birth Data Quality Workgroup

Extending our reach through partnershipsJune 2-6, 2013, Phoenix AZ

MembersBirth Data Quality Workgroup

Co-chairs Isabelle Horon (MD), David Justice (NCHS), Joyce Martin (NCHS)

MembersSukhjeet Ahuja (NAPHSIS), Sally Almond (MN), Marie Aschliman (UT), Karyn Backus (CT), Karin Barrett (MA), Krista Bauer (MN), Mary Chase (CO), Bruce Cohen (MA), Cindy Coleman (MN), Greg Crawford (KS), Cheri Denardo (MN), Judy Edwards (MN), Claudia Fabian (IL), Mark Flotow (IL), Coleen Fontana (RI), Kelly Friar (OH), Melissa Gambatese (NYC), Gloria Haluptzok (MN), Cindy Hooley (VT), Renata Howland, (NYC)Priya Kakkar (KS), Larry Nielsen (NAPHSIS), Lloyd Mueller (CT), Ann Madsen (NYC), Carol Moyer (KS), Laura Ninneman (WI), Michelle Osterman (NCHS), Sharon Pagnano (MA), Mariah Pokorny (SD), Phyllis Reed (WA), Matthew Rowe (WY), George Rudis (IL), Margarita Segundo (CO), Neeti Sethi (MN), Shae Sutton (SC), Marie Thoma (NCHS), Elaine Tretter (MD), Regina Zimmerman (NYC)

Origins Birth Data Quality Workgroup

NAPHSIS/NCHS Good to Great collaboration

VSCP Directors’ conference calls

Workgroup establishedone year ago at 2012 Joint Meeting

Mission Birth Data Quality Workgroup

Assess and improve the quality of vital statistics birth and fetal death data

Focus on improving data at the sourcei.e., at the hospital

Priorities Birth Data Quality Workgroup

SubgroupsHospital reportsKaryn Backus (CT)

Engaging hospitalsKelly Friar (OH), Colleen Fontana (RI)

Prenatal care itemsIsabelle Horon (MD)

New subgroupsE-learning training for hospital staffIdentify items to drop from the national birth data file Working with the Good to Great to develop process for dropping

items

Birth Data Quality Workgroup

Meeting Wednesday @ 7:15 amPlease join us!

Data Quality WorkgroupSubgroup 1

Members:• Karyn Backus (CT)• Ann Madsen Straight (NYC)• Phyllis Reed (WA)• Elaine Tretter (MD)• Greg Crawford (KS)• Carol Moyer (KS)• Mary Chase (CO)• Sharon Pagnano (MA)• David Justice (NCHS)• Marie Thoma (NCHS)• Sukhjeet Ahuja (NAPHSIS)

• Charge: Recommend a process (metrics, means of communication, actions) for vital records offices to improve hospital reporting of birth certificate data

• Aim 1: Survey jurisdictions on their current activities to• Measure hospital-level data quality

• Communicate findings to hospitals

• Act on information

Background

NAPHSIS survey participantsCharacteristics of Jurisdictions n/N (%)

Response rate 41/52 (78.8%) – United States2/5 (40.0%) – Territories

2003 Birth Certificate 33/41 (80.5%) – United States1/2 (50.0%) – Territories

Data quality responsibility

Registration Staff 14/41 (34.1%)

Registration and Statistics (shared) 18/41 (43.9%)

Statistics Staff 9/41 (22.0%)

Current Practices:Standardized Worksheets

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

76%61% 58% 64%

30%

24%39% 42% 36%

70%

Yes No

Jurisdictions using the 2003 birth certificate (n=33)

Current Practices: Evaluating Data Quality

Real-time communica-tion (< 1 wk)

Logic checks Regular completeness reports

Medical record audits (rarely or regular)

Other detailed analyses (i.e., trends, patterns)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

37%

88%

40%

22%

78%

63%

7%

60%

78%

20%

Yes No Unknown

U.S. jurisdictions (n=41)

Current Practices: Evaluating Data Quality

• Does your jurisdiction maintain a desire to perform a more extensive quality review?• 90% indicated “Yes”• 95% indicating limited resources as main impediment

Current Practices: Evaluating Data Quality

Yes63-71%29-37% 100%

Evaluated Impact Detected Improvement

No Yes!Completeness Reports, 63% (n=19)Medical Record Audits, 67% (n=9)Other Detailed Analyses, 71% (n=17)

Current Practices: Raising Awareness

Hospital Staff Only40%

Hospital Staff and Stakeholders

33%

Stakeholders Only5%

No efforts23%

Hospital Staff (total): 72%

Stakeholders (total): 38%

U.S. jurisdictions (n=40)*

* No response for 1 jurisdiction

• Produce Final Summary Report

• Subgroup 1 - Goals:

• Develop metrics to evaluate birth facility data quality that are feasible for jurisdictions to implement

• Develop means of communications to promote birth data quality

• Prepare best practice recommendations for data quality management

• Support overall data quality workgroup and ongoing national initiatives

Future Directions

Data Quality WorkgroupSubgroup 2

Members:• Kelly Friar, Ohio• Colleen Fontana, Rhode Island• Melissa Gambatese, New York City• Matthew Rowe, Wyoming• Sharon Pagano, Massachusetts• Elaine Tretter, Maryland• Shae Sutton, South Carolina• Joyce Martin, NCHS

• Overall Charge• Develop approaches in engaging hospitals to improve

the quality of the birth certificate data provided to the jurisdictions’ Vital Records office

• Goal• Develop a set of standard guidelines for jurisdictions

to follow to engage hospital staff to improve the quality of the birth data.• Can be tailored to any jurisdiction• Can be self-sustaining

Engaging Hospitals

Deliverables• Short List of Deliverables• Recommendations to Engage Hospitals• Introduce intent of project/campaign to high level

hospital key players• Ways to identify key data elements that could/should

be of concern to hospital• Key points to consider when addressing hospital staff• Such as…

A Few Key Points of Consideration• Purpose of birth health data• Contributing factors of birth outcomes• How does quality data help the hospital with the community

it serves• What could be the “burning platform” for the hospital (QI –

PDSA model)• Develop ways for the hospital to access the data• Standardization of definitions• Training• Metrics for vital records offices to improve hospital

reporting – Subgroup 1

Action Items

1) Collaborate with national partners2) Steal shamelessly from other jurisdictions that may

have done this!3) Determine why a hospital should care and how to

convince them they should care4) Determine who in the hospital needs to be convinced5) Develop methods of communication – what and how

1) Collaborating with National Partners• American Congress of Obstetricians and

Gynecologists (ACOG)• March of Dimes• Association of Women’s Health, Obstetric and

Neonatal Nurses (AWHNN)

2) Steal shamelessly from other jurisdictions – we did!

• Ohio – Kelly Friar• Ohio statewide initiative• Improve birth outcomes and reduce infant mortality• Key data elements of focus• Letter of introduction to hospital executives• Dr. Jay D. Iams, Ohio State University Wexner Medical Center• Joined our subgroup conference call to gain doctor’s perspective

– very enlightening and encouraging

• California Maternal Quality Care Collaborative - Dr. Elliott Main

We would love to hear from other jurisdictions – just contact any

member of the workgroup

Thank You !

Members:

• Sukhjeet Ahuja (NAPHSIS)

• Karyn Backus (CT)

• Bruce Cohen (MA)

• Isabelle Horon (MD)

• Renata Howland (NYC)

• Michelle Osterman (NCHS)

• Elaine Tretter (MD)

Prenatal Care Data ItemsSubgroup 3

• To assess the quality of prenatal care data items collected on the U.S. Standard Certificate of Live Birth; and

• Recommend changes for improvement

Charge

1. Determine which prenatal care data items should be studied

2. Prepare summary of why these data items were selected for inclusion on the certificate, and how they are used for public health purposes

3. Compile information available on data quality and barriers to collection of accurate data

4. Compile information on state efforts to improve data quality

5. Prepare best practices document

6. If warranted, recommend changes to data items on certificate

Workgroup plan

• Date of first prenatal care visit

• Date of last prenatal care visit

• Number of prenatal care visits

Data items selected for study

History of collection of PNC data

•1968• Month PNC began to be collected

•1972• Number of PNC visits added

•2000—Panel recommendations• Month PNC began Date of 1st PNC visit• Slight revision to number of PNC visits question

•2001—Addendum to Panel Report• Date of last PNC visit added

Barriers to collecting accurate data

• Missing data• PNC records not sent to hospital • Mothers may change PNC providers

• Inaccurate data• PNC records sent well before delivery and don’t contain current data

• Birth clerks may estimate date of last PNC visit and total number of visits• Some states have placed PNC items on mother’s worksheet• Difficult to evaluate what constitutes:

• A first PNC visit• A PNC visit

• Very time-consuming for birth clerks to collect data items• Providers may provide inaccurate data

Data AccuracyRange of level of agreement in hospitals in 3 states

Data itemLevel of agreement

Medical records and birth certificates

Date of first PNC visit 30% - 90%

Date of last PNC visit 10% - 80%

Number of PNC visits 5% - 90%

**Accuracy rates probably even poorer, since data in medical records often incorrect**

Jurisdictions with efforts aimed at improving PNC data quality

No ef-forts

in place58%

Efforts in

place42%

79% believe efforts aremaking a difference

2013 NAPHSIS Survey

Does poor data quality prevent PNC data items from being useful for public health purposes?

Date of first PNC visit Date of last PNC visit Number of PNC visits0

5

10

15

20

25

30

35

40

45

24

3639

Perc

ent

% of respondents indicating “yes”

2013 NAPHSIS Survey

Summary• Problems with data accuracy• Limited efforts underway to improve data accuracy• Data quality limits usefulness for public health

purposes• Collection/correction requires considerable effort • Medical staff• Birth clerks• Vital Records staff

• Next steps• Can the data be improved?• Should changes be made to certificate?