GIS Data Quality Evaluator Version 4.0 DataLOGIC, Inc. DataLOGIC Corporation 72 Dartmouth Avenue Avondale Estates, GA 30002 404-289-4050

GIS Data Quality EvaluatorVersion 4.0

DataLOGIC, Inc.

DataLOGIC Corporation72 Dartmouth AvenueAvondale Estates, GA 30002404-289-4050www.datalogic-systems.com

DataQE – Data Quality Evaluator • 2

Overview

• Concepts• Types of QA Checks Performed• Evaluation Process• Future Development


What is DataQE?

• GIS Data Quality Evaluation Tool• Evaluations based on user-specified rules• Focuses on evaluations of attribute fields

and values• Includes:

– Layer Checks

– Data Structure Checks

– Field Checks

– Query Checks


What Can You Evaluate with DataQE?

Compliance• Is data compliant with established standards, such as a

Data Dictionary?

Completeness• Has required data been appropriately populated?

Suitability• Is a data source appropriate for a given purpose?• Does data make sense from a real-world perspective?


Who Can Use DataQE?

• Anyone who needs to evaluate the quality or suitability of a GIS data layer or tabular data– Data Entry Staff

– Data Migration Specialists

– GIS Managers

– Quality Assurance Staff

– Analysts


Requirements

• ESRI ArcMap 9.3 or higher• Data Layers or Tables in the following formats

are currently supported:– Geodatabases (SDE, File, Personal)

– Shapefiles

– Coverages

– Database Tables (SQL Server, Oracle, MS Access, MS Excel, DBF, CSV)


The Data Evaluation Process

• Create Rule Sets containing the rules your data should follow

• Assign a Rule Set to the data source being evaluated

• Launch an Evaluation of the data source using the selected Rule Set

• View Evaluation Results and identify data errors

• Correct any errors as necessary using ArcMap or other editing tools

• Re-evaluate as necessary


Layer Checks

• Data Source/File Name Check– Verifies that the name of the evaluated data source name matches

a list of valid names

• Layer Type Check– Verifies that the layer type of the evaluated data source matches

specified requirements (GDB vs. Shapefile vs. Coverage, etc.)

• Projection Check– Verifies that a layer’s current projection matches established

projection requirements

Verify commonly used properties within a spatial data source.


Data Structure Checks

• Field Exists Check– Verifies that required fields exist in the data source being

evaluated

• Field Type Check– Verifies that each field is the correct type (Text, Date, etc.)

• Field Width Check– Verifies that each field is the correct width

Verify that a data source’s field structure matches appropriate requirements.


Field Checks

• Required Value Check– Verifies that each record in a field contains a value

• Unique Value Check– Verifies that each record in a field contains a unique value

• List of Values (LOV) Check– Verifies that each record in a field contains a value that matches a

specified list of values (domain)

• Valid Range Check– Verifies that each record in a numeric field contains a value that

falls within a specified range

Verify that attribute values within a field match established standards.


Query Checks

• Used to perform in-depth evaluations of a data source• Query Rules can be created using the DataQE Query

Builder, or copied from existing ArcMap or database queries

• Allows queries to be stored permanently in Rule Sets• Can be used to expand basic compliance evaluations to

determine whether data meets established Business Rules• Can be used by analysts to evaluate whether data is

appropriate for a particular purpose• Can be used to validate whether data “makes sense”, as

opposed to simply meeting data dictionary standards

User-defined rules based on customizable queries.


Data Quality Evaluator Window

• View Rule Set Assignments and properties

• Review Evaluation Results

• Used to perform evaluations of selected data sources

Data Quality Evaluator

ResultsWindowResultsWindow

Data SourcesWindow

Data SourcesWindow

Rule Set ExplorerRule Set Explorer


Rule Set Manager

• Create Rule Groups and Rule Sets

• Add/Edit rules within a Rule Set

• Rule Sets can be shared among users

Used for managing and editing Rule Sets

Rule Set Manager – Properties Window


PropertiesWindow

PropertiesWindow

Rule Set Manager – Layer Rule Properties


LayerRulesLayerRules

Rule Set Manager – Field Rule Properties


Field RulePropertiesField RuleProperties

Rule Set Manager – Field Rules Grid


Field RulesGrid

Field RulesGrid


Query Checks

• Allow users to build custom Query Rules.

• Query Builder window helps users build a query statement that can be used to evaluate a field or combination of fields.

Query Check

Rule SetExplorerRule SetExplorer

Query RulePropertiesQuery RuleProperties


LOV Manager

• Create/Import New LOV’s

• Add/Edit Values within an LOV

• Assign LOV’s to Rule Sets

Used for managing and editing Lists of Values (LOV’s) for Domain Checks

LOV Manager – Valid Value Lists

Value ListsValue Lists Valid ValuesValid Values

LOV Manager – Rule Set Assignments

Value ListsValue Lists Rule SetAssignments

Rule SetAssignments

LOV Manager – Create LOV’s from Existing Data

Value ListsValue ListsCreate LOV’sFrom Existing

Data

Create LOV’sFrom Existing

Data

Evaluation Manager

• Allows creation of Evaluations with pre-assigned properties

• Easy selection of Data Sources and Rule Sets

• Evaluation Definitions can be stored & shared

• Allows definitions to be saved and performed at any time, individually or in batches


Used to manage and edit predefined Evaluation Definitions to support batch processing

EvaluationDefinitions

Window

EvaluationDefinitions

Window

PropertiesWindow

PropertiesWindow

Evaluation Manager


Future Development

Coming in future versions of DataQE:

• Support for Citrix implementations

• Password Protected Public Rule Sets

• Associated Tables/Orphan Records Check

• Metadata Check

• User Definable Field Rules– Business Logic Check

– Spatial Overlay Check

Documents

GIS Data Quality Evaluator Version 4.0 DataLOGIC, Inc. DataLOGIC Corporation 72 Dartmouth Avenue Avondale Estates, GA 30002 404-289-4050