Embed Size (px)
DESCRIPTION
3 To address issues discussed in the Panel’s subcommittee presentation (June 2007 ): Enforce meaningful referential integrity Enforce meaningful data uniqueness Purpose
Citation preview
Fix Implementation Deficiencies of Database Structure
Presenter: Rueen-Fang Wang, Ph.D., P.E.Presenter: Rueen-Fang Wang, Ph.D., P.E. 11
22
TopicsPurpose of this taskDeficiencies identified Steps to resolve deficienciesCurrent statusRemaining tasks
22
33
To address issues discussed in the Panel’s subcommittee presentation (June 2007 ): Enforce meaningful referential
integrity Enforce meaningful data
uniqueness
Purpose
44
Deficiencies Identified in the Initial Implementation
Related to referential integrity:1. Arc relationship2. Logical deletion3. List Item reference
Related to meaningful uniqueness4. Database Uniqueness rules
55
Identified Deficiency 1 –Arc Implementation
Deficiency Scale of Problem
Arc Usage Rules have not been enforced during the initial implementation => allowing incorrect references between tables
5 (<2%) rule violations out of 261 arc usage rules, involving approximately 15,000 (<0.1%)records out of 17,000,000
Referential integrity of arc relationship has not been enforced during the initial implementation => allowing creation of orphan records
About 480,000 (~1.5%)orphan records exist via arc implementation out of 32,000,000
66
Identified Deficiency 2 –DELETED_DATETIME column
Deficiency Scale of ProblemApplication performed incomplete logical deletion of data records by setting DELETED_DATETIME in the parent records only => bypassing FK check and allowing orphan records to be created at logical level
More than 300,000 (~4%)orphan records exist out of 69,000,000
77
Identified Deficiency 3 –List Item Implementation
Deficiency Scale of ProblemList Item Usage Rules have not been enforced during the initial implementation => allowing creation of data records with incorrect lookup values (i.e. Party record refers to Place Type, etc.)
5 (~2%) rule violations out of 234 list item usage rules, involving approximately 26,000 (<0.05%)records out of 52,000,000
88
Identified Deficiency 4 –Meaningful Data Uniqueness
Deficiency Scale of ProblemMeaningful Data Uniqueness Rules have not been enforced for many tables in the initial implementation => allowing creation of duplicate data records
Approximately 114 (80%) tables out of 142 tables being used do not contain rules to enforce meaningful uniqueness
99
Review and develop adequate database rules as necessary
Identify existing problematic data and the associated causes
Develop solutions to clean up the data and fix application
Develop and impose DB constraints and triggers to enforce database rules
Steps to Resolve Deficiencies
1010
Current StatusArc Deleted
Date timeList Item Meaningful
Uniqueness
Develop Rules
completed as of 9/30/07
completed as of 8/27/07
completed as of 8/31/07
to be completed by 3/31/08
Identify Problem
completed as of 10/11/07
completed as of 9/20/07
completed as of 9/13/07
begin in Feb, 2008
Enforce Rules
completed as of 12/31/07
completed as of 12/31/07
to be completed by 3/31/08
begin in Feb, 2008
Fix App & Clean Data
to be completed by 6/30/08
to be completed by 6/30/08
to be completed by 6/30/08
begin in Feb, 2008
1111
Remaining Tasks
Arc Fix application and clean up existing data
Deleted Date time
Clean up existing data marked as deleted, including linked child records
List Item Cleanup 113 existing data records that violate a specific list item usage rule
Meaningful Uniqueness
Implement uniqueness rules for each table (including fixing application, clean up data, and impose rules)
1212
Questions?
