State EMS Data Collection, Evaluation, and Quality ... · EMERGENCY MEDICAL SERVICES AUTHORITY 1930 9th STREET SACRAMENTO, CA 95811-7043 (916) 322-4336 FAX (916) 324-2875 EMS System

STATE OF CALIFORNIA – HEALTH AND HUMAN SERVICES AGENCY ARNOLD SCHWARZENEGGER, Governor

EMERGENCY MEDICAL SERVICES AUTHORITY 1930 9th STREET SACRAMENTO, CA 95811-7043 (916) 322-4336 FAX (916) 324-2875

EMS System Guidelines

EMSA #160

State EMS Data Collection, Evaluation, and Quality

Improvement System Overview

EMSA #160 Commission on EMS December 5, 2007

State EMS Data Collection, Evaluation, and Quality Improvement System Overview

2

Prepared by:

The California Emergency Medical Services Authority

AS DEVELOPED BY: State EMS Data Committee

Arnold Schwarzenegger Governor

~

Kim Belshé Secretary

California Health and Human Services Agency

~

Cesar A. Aristeiguieta, M.D. Director

California EMS Authority

~

Bonnie Sinz, RN Chief, EMS Systems Division

California EMS Authority


3

ACKNOWLEDGEMENTS

EMS DATA COMMITTEE

Steve Andriese, Administrator Mountain Valley EMS Agency Barbara Stepanski, MPH Epidemiologist San Diego County EMS Agency Glen Youngblood EMS Coordinator/Data Analyst San Mateo County EMS Agency Linda Combs Sierra-Sacramento Valley EMS Agency Craig Stroup Emergency Medical Sciences Training Institute Allen N. Norman, MPA, EMT-P, Battalion Chief Commander, Quality Improvement Section Los Angeles Fire Department Che-Chuen Ho Data Systems Manager/Trauma Data Analyst Santa Clara EMS Agency Cindy Wilmshurst Special Projects Coordinator National EMSC Data Analysis Resource Center Mike Denton, RN Lawrence Livermore National Laboratory Ginger Ochs, RN Quality Management Coordinator San Diego Fire Department

Kristy Harlan Data Coordinator American Medical Response Luanne Underwood, RN QI Program Director Los Angeles County Fire Department Ric Maloney, RN CQI Manager/Paramedic Instructor Sacramento Metro Fire Department Maureen Hasbrouck, RN, MS EMS Data System Coordinator Los Angeles County EMS Agency Adele Pagan Information Systems Analyst Alameda County EMS Agency Bob O’Brien, Division Chief Fremont Fire Department Leif Juliussen CSFA EMS Committee Milpitas Fire Department David Lindberg Todd Hatley Project Consultants HealthAnalytics

EMS Authority Representatives: Bonnie Sinz, RN Ed Armitage Charla Jensen


4

1 2 3 4

5

6

7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

EMS System Guidelines

EMSA #160


“The ability of EMS to optimally meet communities’ and

individual patients’ needs in the future is dependent on evaluation processes that assess and improve the

quality of EMS. Continuous evaluation is essential and should pervade all aspects of every EMS system.”

Theodore R. Delbridge MD, MPH

Introduction In the above quotation from the 1996 National Highway Traffic Safety Administration’s (NHTSA) publication, Emergency Medical Services, Agenda for the Future1, Dr. Delbridge stresses the importance of an effective evaluation and improvement system if EMS systems in this country are to be successful in the future. This statement can be interpreted as containing both a “quality-of-care,” as well as an “economic” warning. When economies tighten at the federal, state, and local levels, it becomes extremely difficult to compete for limited healthcare dollars if EMS systems cannot definitively show their worth to the community. More importantly, the fact that EMS systems are administering medications and practicing invasive medical procedures carries with it a fiduciary responsibility to show that those systems are safe and effective for the patient being served, and to demonstrate what affect the EMS system has on the final outcome of the patient. In an effort to encourage standardized data collection and system evaluation on a national level, NHTSA developed their original standardized EMS data standards in 1993, which most states, including California, adopted. However, these data standards lacked many of the necessary elements required for effective system evaluation, did not contain clear and concise definition for all data elements, and

1 NHTSA, Agenda for the Future, 1996; p57


5

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

did not include standardized quality indicators for which the data elements would be used. Because the original NHTSA data standards had its limitations, and because many states did not have the resources or capability for large scale participation in EMS data collection programs, no real effective data were produced on a national level. In July 1997, NHTSA released “A Leadership Guide to Quality Improvement (QI) for EMS Systems” which has become a standard for EMS QI nationwide. In addition, in April 2004, following a two-year development process, NHTSA announced the release of a complete rewrite of their EMS data standards entitled, NHTSA Uniform Prehospital Dataset, Version 2.1. Even though this document still does not contain standardized quality indicators for EMS, it does, along with the 1997 QI Guide, provide a level of standardization in EMS evaluation and improvement that has never existed before. The leadership of NHTSA is important since their work finally provides a standard for local EMS systems and EMS software vendors nationwide to utilize in their data system designs. Without this leadership at the national level, software vendors and state/local EMS offices would continue to develop new, non-comparable and non-compatible data systems. With this new data set as a foundation, those who choose to follow these standards will be able to produce and compare data across local borders and state lines. Consistent with this vision is the project by the State EMS Officials’ Association, in conjunction with NHTSA, to implement the National EMS data repository and reporting system (National EMS Information System [NEMSIS]). NHTSA is also supporting the National EMS Performance Indicators project for development of a standardized national set of EMS performance indicators, now under development. With the foundation for a standardized approach to EMS system evaluation and improvement in place at the national level, it has moved the challenge of implementing these standards to the state level. History of EMS System Evaluation and Improvement in California By 1999, California had not been much more successful than the rest of the nation in creating an effective EMS evaluation and improvement system. The California EMS Vision Project, which was established by the EMS Authority and EMS Commission in 1999, addressed several areas of perceived deficiency in the EMS system in California, and identified the establishment of an effective, standardized, data collection and quality improvement system among the top six EMS needs within the state. The 1999 National Highway Traffic Safety Administration’s (NHTSA) evaluation of the California EMS system reiterated this


6

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

finding when they reported that there is, “a lack of an integrated, statewide information system that (has) the capability to monitor, evaluate and elucidate emergency medical services and trauma care in California.”2 Much like the rest of the nation, California has had some success stories in local system evaluation and improvement. In the 1980’s and early 1990’s, local EMS agencies (LEMSAs) were encouraged, through the EMS System Guidelines and special project funding, to develop data collection systems. However, without a clear standard for how these systems should be designed and utilized, LEMSAs developed their data collection systems independent of each other. While a state data standards document (based on the original NHTSA data standards did exist, it was void of any accompanying performance indicators, effective definitions, benchmarks, or standardized reporting capabilities. The LEMSAs that did develop data systems utilized their own definitions and data parameters. Consequently, they were only able to measure their performance against themselves. With the exception of some basic descriptive or structural data reporting capabilities, no comparative analysis of performance from LEMSA to LEMSA, or on a statewide basis was possible. In addition, even in areas where data was collected, it was not always used effectively for quality improvement purposes. Through both the Vision Process, and the NHTSA assessment, this was identified as an unacceptable system deficit. The Vision Process ran from 1999 to 2003. During that period, several multi-agency groups, committees and task forces assisted in the development of the EMS System Evaluation and Quality Improvement System outlined in EMSA Series 160. Under the overall coordination of the “Vision Work Group D, System Evaluation and Improvement,” the Paramedic Task Force, EMS Data Committee and many EMS Constituency groups were instrumental in creating the referenced documents below. The ultimate goal of the project was to create an outline for a comprehensive EMS system evaluation and improvement system in California that was compatible with all national standards in force at the time. The results of those years of work make up EMSA Guidelines #160 -168. Purpose and Format of the EMSA-160 Series Guidelines In reviewing the 160 series guidelines, the reader will notice that the concepts of EMS system evaluation and EMS quality improvement are treated as a continuum.

2NHTSA, Assessment of Emergency Medical Services in California; August, 1999; p30


7

1 2 3 4 5 6 7 8 9

10 11 12

System Evaluation, which consists of data collection, data analysis, and system research, is futile from a quality of care perspective if that evaluation is not reported and utilized for system improvement. Likewise, EMS Quality Improvement cannot be effective if it is not based upon sound data and research. Therefore, these two concepts are presented as a single process, broken down into the components of Guidelines #161-168 The guidelines contained in the 160 series should provide the step-by-step guidance necessary for a local EMS service provider, a LEMSA, or the State EMS Authority to identify their specific roles and responsibilities in a standardized, statewide EMS evaluation and improvement system. While some of the documents necessary for a comprehensive program have not yet been completed (these are identified in the Table of Contents in gray italic print), the documents that are currently included provide the key elements necessary to establish and implement effective data collection, and quality improvement programs at the provider, LEMSA, base hospital and state levels.

13 14 15 16 17 18 19 20

Listed below is a brief description of each of the Guideline documents included in the Table of Contents. I. Evaluations of EMS Regulatory Agencies 21

22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

EMSA # 161 - State EMS Authority Assessment Guidelines (Not yet Developed) EMSA # 162 - Local EMS Agencies Assessment Guidelines (Developed but not validated or tested) Besides stressing the needed for a sound evaluation and improvement system, the NHTSA EMS Agenda for the Future, referenced in the introduction of this document, also conveys the importance of assessing “all aspects” of the EMS system. A comprehensive EMS CQI program must include evaluation and ongoing improvement within all components of the system, not just the medical care provided in the field and designated receiving facilities. To this end, guidelines are being developed that include evaluation and improvement of the services provided by the state’s EMS administrative agencies. The evaluation would include the State EMS Authority and local EMS agencies, to ensure that planning, implementation, and monitoring functions are being performed efficiently and effectively, and that the day-to-day duties of these agencies are supportive of the care being provided in the field.


8

1 II EMS System Evaluation 2

3 4 5 6

7 8 9

10 11 12 13 14

15 16 17 18 19 20 21 22 23 24 25 26

EMSA # 163 – State EMS Core Quality Indicators (Appendix M to the Emergency Medical Services System Quality Improvement Program Model Guidelines) Historically, EMS data collection system design has begun with the establishment of a data set which included those elements that the creators felt would be worthwhile for future reporting. It hasn’t been until after the systems were developed that the reporting questions were actually asked of the system. Very often, because the software was not designed to answer a particular question, system modification and additional reprogramming would be necessary. As new questions were asked, new modifications were required. To ensure that the amount of system modifications was kept to a minimum, prior to finalizing the California State EMS Data Standards, a different approach was taken. Instead of building the data collection system, and then asking the questions, the process was reversed. The Vision Work Group D spent more than two years determining key questions that must be answered to ensure that all key components of the EMS system could be effectively evaluated. These questions were then translated into the State EMS Core Quality Indicators (EMSA # 163). Only after these questions were carefully established was the State EMS Data Standards (EMSA # 164) finalized. This approach has helped ensure that the necessary data elements to answer key system questions can be answered once the system is fully functional. However, even with the amount of time spent on the development of the Core Quality indicators, the Work Group was not able to complete all the performance indicators they set out to develop. Work on the development of new core quality indicators will be on-going as EMS providers, LEMSAs and the State EMS Authority continue to refine and evolve their approaches to EMS performance improvement. When the initial version is fully completed, EMSA #163 should include standardized indicator definitions and minimum benchmark values to facilitate comparative analysis of local system performance, quality of patient care, customer satisfaction, and system cost on a local and state level. Once EMS Core Quality indicators have been established at the national level, it is anticipated that the California indicators will be revised to comply with those indicators.

27 28 29 30 31 32 33 34 35 36 37 38 39

40 41 42


9

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

EMSA # 164 - State EMS Data Standards The State EMS Data Standards consists of a comprehensive list of minimum data elements and definitions consistent with two main sources. First it will always be in compliance with the NHTSA Uniform Prehospital Dataset, National EMS Information System [NEMSIS]). Secondly, it also contains the elements necessary for monitoring and evaluating the State EMS Core Quality Indicators (EMSA # 163). The State EMS Data Standards contains the names of each data element, the source(s) from which the data can be obtained, and a detailed, practical operational definition of the element. Additionally, validation criteria have been developed to evaluate data quality and integrity at multiple points in the data collection, transfer, storage, and analysis process. The entire list of State EMS Data Standards is categorized as ‘Minimum,’ which means they must be collected for a given incident in circumstances where applicable. As an example of applicability; the date that the call was received would be considered to be a data element that would be applicable to all incidents. In contrast, the name of the patient may not be applicable because in some circumstances, there may not be a ‘patient’ identified for a particular EMS response. This may occur with an automobile crash where the vehicles and their occupants cleared the scene before any emergency responders arrived on-scene.

EMSA # 165 - State Data Collection and Reporting Guidelines (Under Development) The State EMS Authority has established a statewide EMS data collection reporting system which has been named the California EMS Information System or CEMSIS. Once fully online, this system will be able to collect standardized data from all participating LEMSAs in the state and provide a web-based reporting capability for the public, LEMSAs, and service providers to utilize. This data collection system will develop over time and is to be based on available resources at the LEMSA and provider agency level. At first, this system will be able to collect Patient Care Record (PCR) only in accordance with the EMS Data Standards. Once the system has matured, Computer-Aided Dispatch (CAD) data will be included with linkage capabilities with the National Fire Incident Reporting System (NFIRS). This will initially limit reporting capabilities to field operations and


10

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17

18

19 20

21 22 23

24

field care. However, the system is designed to be expanded to link with the State Trauma Registry, Office of Statewide Hospital Planning and Development (OSHPD) which stores hospital and emergency department outcome data. Further expansion to include the Department of Health Services vital statistics data (e.g., death statistics), and the California Highway Patrol CRASH data is also included in the long term expansion plan. With these vital linkages, California will be well positioned to provide many different types of EMS patient outcome data that have never before been available. EMSA # 165 is designed to provide the timeline, instructions and format for LEMSAs and local providers to follow in order to successfully up-load local data into CEMSIS as well as how to utilize the web-based reporting features. The system features will ultimately include:

• Process performance feedback via data reports to all participating EMS agencies

• Maintenance of data confidentiality and security

• Mechanisms for feedback to prehospital personnel on the diagnosis and disposition of their patients

• Collection and sharing of data among EMS system participants, to include integration and linkage of data with other private, state, and federal agencies, and organizations as appropriate.

III. EMS System Quality Improvement 25

26

27

28 29 30 31 32 33 34 35 36 37 38 39 40

EMSA # 166 – State EMS System QI Program Model Guidelines EMSA # 166 follows the guidelines published in NHTSA’s “Leadership Guide to Quality Improvement (QI) for EMS System’s.” It includes mechanisms to ensure that EMS performance data is utilized at the state, local and provider level for continuous quality improvement aimed at improving EMS services and quality of patient care, decreasing death and disability, and reducing costs. It also establishes linkages with EMS training and prevention programs to ensure that needs identified through the evaluation process are integrated into EMS training curriculum and prevention efforts. Finally, it provides an organizational structure and standard operating procedures necessary to ensure maintenance of a statewide EMS data collection, evaluation and quality improvement process.


11

IV. System Evaluation and Improvement Training 1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17

EMSA #167 – State EMS System Evaluation Training Guidelines (To be developed) Once completed, the System Evaluation and Improvement Training Guidelines will contain the recommended training modules for all levels of EMS system personnel to ensure proper documentation, data entry, analysis, utilization of data, and an understanding of the principles of quality improvement and research. It is anticipated that there will be a module developed for EMT-Is, EMT-Ps, and MICNs and a separate module for LEMSA and State EMSA staff. Guidelines will also be developed to assist provider agencies in developing agency specific training. Ultimately, the curriculum developed for field personnel under this section, should be added to their initial training requirements.

V. EMS System Research 18

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43

EMSA # 168 - EMS Research Guidelines (To be developed) Once completed, the EMS System Research component of the California EMS QI Program will provide a response to the State EMS Commission’s frequent expression of the need for guidelines for conducting and funding of State supported, or State required EMS research. The National EMS Research Agenda will be integrated into this process. To that end, EMSA #168 should establish mechanisms to:

• Identify the various types of state required or state funded research that may be conducted to include: trial studies, treatment guideline effectiveness research, EMS system cost evaluation studies, patient outcome studies, etc.

• Identify the various research design methodologies practical for EMS such as quantitative, qualitative, survey, observation, historical, experimental, evaluation, etc.

• Identify current requirements and standards for conducting EMS research for each research design methodology identified.

• Establish a statewide IRB Process • Develop criteria for EMS research based upon above standards

which take into practical consideration research in rural areas with low study populations.

• Identify the types of research which should be given priority for state funded research projects.


12

1 2 3 4 5 6 7 8 9

10 11 12

• Develop minimum standards criteria for research projects to be funded by the State EMS Authority.

• Develop strategies which would encourage statewide EMS research projects that meet the minimum standards developed above.

• Identify all current barriers to conducting EMS research in California such as restrictions for prehospital human subject review

• Identify legislative/regulatory changes in bullet point format required to reduce identified barriers

• Identify areas of research outside of the realm of EMS that would benefit EMS

Future Needs 13

14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42

As mentioned before, the Table of Contents includes documents that are not yet completed. The unfinished documents were included in the Table of Contents to ensure that the vision of the comprehensive structure was not lost. As time and funding becomes available, those unfinished sections will be completed. Listed below, is a suggested order of priority for those projects in need of completion. 1. Identify a standing State EMS System Evaluation and Improvement

Oversight Body to coordinate implementation of the data, and QI efforts and coordinate future revisions to the statewide evaluation and improvement process.

2. Completion of the State Data Collection and Reporting Guidelines

(EMSA #165) and submit to the State EMS Commission for adoption. 3. Implementation of the State EMS System Quality Improvement

Program Model Guidelines (EMSA #166) throughout the state. 4. Completion of the State EMS System Evaluation Training Guidelines

(EMSA # 167) to facilitate #1 above. 5. Conduct testing and validation of the LEMSA Assessment Tool

(EMSA #162) and submit final tool to the State EMS Commission for adoption.

6. Completion of the uncompleted quality indicators (EMSA #163) to

ensure a complete and comprehensive list of EMS evaluation indicators.


13

1 2 3 4 5 6 7 8 9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

7. Completion and full utilization of the statewide CEMSIS data collection and web-based reporting capabilities.

8. Establishment of data linkages between CEMSIS and a) OSHPD

Hospital Outcome and E.D. Data, b) DHS Vital Statistics Data, c) CHP/FEDERAL Crash Data.

9. Establishment of data downloads to the National EMS Information

System. 10. Development of the State EMS Authority Assessment Guidelines

(EMSA 161). 11. Development of EMS Research Guidelines (EMSA 168). Conclusion By following the guidelines established in this Section, it is anticipated that EMS service providers, LEMSAs, and the State EMS Authority will be able to, for the first time, truly measure the effectiveness and worth of EMS systems in California, and will be able to provide a comparative analysis between systems that was never before possible. The documents contained in this section are a best effort by all those involved. It is a starting point. These documents will need to remain dynamic and be revised as we collectively gain experience and knowledge on what constitutes effective evaluation and quality improvement methodologies.

Documents

State EMS Data Collection, Evaluation, and Quality ... · EMERGENCY MEDICAL SERVICES AUTHORITY 1930 9th STREET SACRAMENTO, CA 95811-7043 (916) 322-4336 FAX (916) 324-2875 EMS System