Quality Assurance ProgramQuality Assurance ProgramPresenter: Erin MustainPresenter: Erin Mustain

11

22

Recommendation 5Benchmarks

Data quality issues have been categorized and quantified.

A detailed plan exists for addressing sources of continuing errors and correcting historical errors.

The plan has been validated with representative data samples

Substantive progress has been made toward correcting major categories of errors.

The Steering Committee agrees that progress is being made and that there is a high probability that existing data problems will be resolved.

3333

Progress Institutionalized Quality AssuranceBusiness RulesError chartSteering committee and User groups

collaborationEliminated system-generated

violations for paper tracking SMRs

44

Total Study Error

Data Population

Data Generation

Migration Manual Data Entry

System Limitations

Training SMRs

Lab errors

Sampling Errors

Intentional manual errors

Field not populated

Field auto-populated incorrectly

Field not appearing

Data doesn’t follow

business rules

Data entered into SWIM incorrectly

Non-numeric data (ND, QND, etc.) not handled by eSMR

System can’t handle unique orders (several

facilities under one permit)

No place to store data (enrollee

history)

Training Manuals

don’t follow Business

Rules

Instruction not

consistent

Doesn’t enforce all of the Business

Rules

Lack of Training

Manual

Errors in data entry

form

Calculation errors

Can’t easily delete records

System generated duplicates (SMARTS)

Business Rules not followed

Typos

Difficulty Data

Mining

Selecting the wrong

link

Duplicate entry

5555

Current vs. Historical Data

66

Data Validation

QA Reports

Audits

SOPs QAP

DQOs

What decisions will bemade with this

data?

CIWQSQA

Program

66

TrainingData

Verification

Data Cleanup

Business rules

Corrective Action Integration with

State Water Board QMP

Communication

7777

Plan and ProceduresQuality Assurance Plan

Scope Roles and responsibilities Data quality indicators Quality objectives Assurance activities Problem reporting and corrective action Audits Migration and future projects

88

Plan and Procedures Standard Operating Procedures

Data Cleanup Training Document Management Corrective Action Quality Assurance Reports Audit End-user-layer enhancements and testing Database enhancement prioritization Report prioritization

99

Plan and ProceduresDIT Standard Procedures Document

Maintenance & Documentation Requirements System Environment Data model, database, data integration, &

maintenance Application source code integration &

maintenance Application and database source code & scripts

repository CVS Database tools & scripts standards

Issue routing Maintenance implementation

10101010

External AuditOnsite audit of Regional Boards and State Board programs

Onsite audit of Division of Information Technology

Data audit using stratified random design approach

Policies and procedures for data entry

Security, performance, and policies and procedures

Accuracy of records

11111111

Next Steps Quantifying the data quality issues – Audit

Correcting historical data – recommendations to management after audit results

Validate the QAP and SOPs with representative data samples

Implement training program

Continuing process improvement at all levels (QA Program is not static)

Establish a mechanism for communication between QA Program and panel