Leveraging ESRI Data Models in Your Leveraging ESRI Data Models in Your Custom Solution Custom Solution

Steve GrisSteve GrisééJoe BremanJoe Breman

ESRI Developer Summit 2006ESRI Developer Summit 2006

TopicsTopics

•• IntroductionIntroduction•• ApplicationApplication--Driven DesignDriven Design

•• Leveraging Data Model TemplatesLeveraging Data Model Templates

•• Data model program updatesData model program updates•• Opportunities for Developers at ArcGIS 9.2Opportunities for Developers at ArcGIS 9.2

……and some information on:and some information on:

GeodatabasesGeodatabases•• More closely represent real world objects than previous More closely represent real world objects than previous

GIS data modelsGIS data models•• Provide a starting point for more sophisticated data Provide a starting point for more sophisticated data

models based on a more natural approachmodels based on a more natural approach•• Rely on geography and rules to define relationships, Rely on geography and rules to define relationships,

not just primary and foreign keysnot just primary and foreign keys•• Users and Programmers donUsers and Programmers don’’t need to manage atomic t need to manage atomic

data elements/topology, they manage the objects and data elements/topology, they manage the objects and relationships insteadrelationships instead

Bridge

BridgeIDBridgeNameConstructionDateLinearLocation...

Feature

Designing Designing GeodatabasesGeodatabases

ImagesImages

VectorsVectors

TopologyTopology

NetworksNetworks

TerrainTerrain

SurveysSurveys

CADCADDrawingsDrawings

AddressesAddresses

27 Main St.27 Main St.

AttributesAttributes

ABCABC

3D Objects3D Objects

107107’’

DimensionsDimensions

AnnotationAnnotation

Geodatabases Model and Manages Geodatabases Model and Manages All Geographic Data Types All Geographic Data Types

Social FactorsSocial Factors

Roads/InfrastructureRoads/Infrastructure

WaterWater

Land Use/Land CoverLand Use/Land Cover

EnvironmentEnvironment

ImageryImagery•• GeoreferencingGeoreferencing•• Digital ProcessingDigital Processing•• Map OverlayMap Overlay•• Spatial AnalysisSpatial Analysis•• VisualizationVisualization

Key ConceptsKey Concepts

GIS Integrates All Types of DataGIS Integrates All Types of Data

. . . Integrating Disciplines, Organizations and Activities. . . Integrating Disciplines, Organizations and Activities

Base MapsBase Maps

Geography is a Geography is a ““KeyKey””

Architectural Overview Architectural Overview –– n Tiersn Tiers

PresentationPresentationTierTier

(multiple viewers)(multiple viewers)

Globe Services

Map &Chart

Services

Image-Processing

Services

Open Web

Services

TrackingServices

Serving/Serving/PublishingPublishing

TierTier

AuthoringAuthoringTierTier

22--D D ViewersViewers

33--DDViewersViewers

Catalog Services

Geo-Processing

Services

Network/Enterprise Service Bus

COPCOP

Geodatabase DesignGeodatabase Design

•• Object oriented analysis and design (OOAD) methods Object oriented analysis and design (OOAD) methods work well for application developmentwork well for application development–– I.e., Agile or UML development processesI.e., Agile or UML development processes–– Useful for designing applications, software, and web services Useful for designing applications, software, and web services

that use serversthat use servers•• They donThey don’’t work well for GIS Servers and t work well for GIS Servers and GeodatabasesGeodatabases

–– Need to accommodate multiple applications, multiple usesNeed to accommodate multiple applications, multiple uses–– Must anticipate how to support evolving needsMust anticipate how to support evolving needs–– OOAD methods usually result in GIS server designs that donOOAD methods usually result in GIS server designs that don’’t t

workwork•• We need a different method based on traditional GIS We need a different method based on traditional GIS

design techniquesdesign techniques–– Application Driven, MultiApplication Driven, Multi--purpose systemspurpose systems–– Based on Best Practices: Based on Best Practices:

•• Domain KnowledgeDomain Knowledge•• PerformancePerformance•• Design patternsDesign patterns

–– Addresses patterns and tradeoffs early in the development cycleAddresses patterns and tradeoffs early in the development cycle

Role of UML DiagramsRole of UML Diagrams•• Need to concentrate on the information productsNeed to concentrate on the information products

required for the system to develop a complete designrequired for the system to develop a complete design•• Many developers/project teams jump to using UMLMany developers/project teams jump to using UML

as a way to implement a designas a way to implement a design–– Problem is that UML is a conceptual language;Problem is that UML is a conceptual language;

it can be used to express anything at any levelit can be used to express anything at any levelof abstractionof abstraction

•• Implementation of a data model in a Geodatabase Implementation of a data model in a Geodatabase requires a level of logical and physical modeling requires a level of logical and physical modeling to build a working systemto build a working system

DomainsDomains

FeaturesFeaturesPackagesPackages RelationshipsRelationshipsObjects/TablesObjects/Tables

TopicsTopics

•• IntroductionIntroduction

•• Data model program updatesData model program updates•• Opportunities for Developers at ArcGIS 9.2Opportunities for Developers at ArcGIS 9.2

•• ApplicationApplication--Driven DesignDriven Design•• Leveraging Data Model TemplatesLeveraging Data Model Templates

……and, some information on:and, some information on:

Design ApproachDesign Approach

•• ApplicationApplication--Driven Design means Driven Design means –– Understanding the GeoUnderstanding the Geo--information needs information needs

required to support business processesrequired to support business processes–– Documenting the information products and Documenting the information products and

layers requiredlayers required–– Creating Application/Data matrices to Creating Application/Data matrices to

document the highdocument the high--level information needslevel information needs

GeoGeo--Information System DesignInformation System Design

LayersLayers

Applications & Applications & Information ProductsInformation Products

DatasetsDatasets

Reporting Reporting Databases, Maps, Databases, Maps,

Reports, Reports, Web ServicesWeb Services

Layer Files, Layer Files, Map Elements, Map Elements, Presentation Presentation

RulesRules

GeodatabasesGeodatabases, , other Files and other Files and

DatabasesDatabases

Information Information NeedsNeeds

ApplicationApplication--Driven DesignDriven Design•• ServerServer--side solutions require this approach to address things like:side solutions require this approach to address things like:

–– What is the map on the server?What is the map on the server?–– Labels vs. anno in the databaseLabels vs. anno in the database–– Relationship classes Relationship classes vsvs relates and joinsrelates and joins–– Dynamic vs. static/preDynamic vs. static/pre--cooked viewscooked views–– Data Management design vs. publication designData Management design vs. publication design–– Multiple scalesMultiple scales

DataModels

LocalTransactions

Local Data SetsLocal Data Sets

•• TransformationTransformation•• ConversionConversion•• IntegrationIntegration

National Data SetsNational Data Sets

DistributionDistribution

Example: GIS for the NationExample: GIS for the Nation

HighHigh--Quality Data Compiled from Local SourcesQuality Data Compiled from Local Sources

NationalTemplates

Products and ServicesProducts and Services

•• GIS for the Nation GIS for the Nation concept requires a concept requires a complex backcomplex back--end end architecture that is not architecture that is not seen in the information seen in the information productsproducts

•• Challenge is to Challenge is to develop best practices develop best practices for building these for building these types of systemstypes of systems

•• Workflows and Workflows and methods involve much methods involve much more than a data more than a data modelmodel

World Trade Center RecoveryWorld Trade Center Recovery

Base of Base of OperationsOperations

Public Public Transportation Transportation

StatusStatus

LidarLidar ImageImage

Urban Urban Search and Search and

RescueRescue

Traffic Traffic RestrictionsRestrictions

Hurricane Hurricane Response and Response and

RecoveryRecoveryAgriculture in North CarolinaAgriculture in North Carolina

Perimeter ControlPerimeter Control

Map It Map It ToolTool

Disease Disease Cluster Cluster

MapMap

NC NC OneMapOneMap

DelawareDelaware

National Mapping ProductsNational Mapping Products

KentuckyKentucky

North CarolinaNorth Carolina

ResultsResults

Orange spots indicate new ideas, orange/grey indicates datasets Orange spots indicate new ideas, orange/grey indicates datasets used by used by information productsinformation products……

DatasetsDatasets

Info

rmat

ion

Prod

ucts

Info

rmat

ion

Prod

ucts

Defining New Products and ServicesDefining New Products and Services

DataData

Data Data MaintenanceMaintenance

ServiceService ProductProduct

ContractorContractor PartnerPartner

CustomerCustomer

• Information product focus needed, not just publishing

• Information Products and Services requirements must drive design process

• A major challenge for most organizations trying to leveraging GIS technology

TopicsTopics

•• IntroductionIntroduction

•• Data model program updatesData model program updates•• Opportunities for Developers at ArcGIS 9.2Opportunities for Developers at ArcGIS 9.2

•• ApplicationApplication--Driven DesignDriven Design•• Leveraging Data Model TemplatesLeveraging Data Model Templates

……and some information on:and some information on:

Geodatabase Data ModelsGeodatabase Data ModelsTemplates for Many FieldsTemplates for Many Fields

. . . Help Users Get Started. . . Help Users Get Started. . . Support . . . Support Consistent StandardsConsistent Standards

•• AddressAddress•• AgriculturalAgricultural•• AtmosphericAtmospheric•• Base MapBase Map•• BiodiversityBiodiversity•• CensusCensus--AdminAdmin•• BoundariesBoundaries•• DefenseDefense--IntelIntel•• Energy UtilitiesEnergy Utilities•• Environmental Environmental •• ForestryForestry•• GeologyGeology•• GroundwaterGroundwater•• HealthHealth

•• Historic Preservation Historic Preservation & Archaeology& Archaeology

•• Homeland SecurityHomeland Security•• HydroHydro•• IHOIHO•• Land ParcelsLand Parcels•• Local GovernmentLocal Government•• MarineMarine•• National CadastreNational Cadastre•• PetroleumPetroleum•• PipelinePipeline•• TelecommunicationsTelecommunications•• TransportationTransportation•• Water UtilitiesWater Utilities

ArcHydroArcHydro

Simple feature classHaul Contains Z values

Contains M valuesGeometry Point

NoNo

Data typeField namePrec-ision Scale LengthDomainDefault value

Allownulls

OBJECTID Object IDSHAPE Geometry Yes

HAULJOIN Long integer No 0VESSELCODE Long integer No 0

CRUISE Long integer No 0HAUL Long integer No 0

STATION_ID String No 7SSTEMP Float No 0 0

BOTTOMTEMP Float No 0 0EFFORT Float No 0 0

DataSourceID String Yes 50

TableDataSource

Data typeField namePrec-ision Scale LengthDomainDefault value

Allownulls

OBJECTID Object IDDESC_ String Yes 35

SOURCE String Yes 35CONTACT String Yes 20

PREC String Yes 20CREATE_DATE Long integer Yes 0

LAST_UPDATED Long integer Yes 0DataSourceID String Yes 50

TableFishCatch

Data typeField namePrec-ision Scale LengthDomainDefault value

Allownulls

OBJECTID Object IDHAULJOIN Long integer No 0

SPECIESCODE Long integer No 0LATINNAME String No 28

COMMONNAME String No 21WEIGHTCPUE Double No 0 0NUMBERCPUE Double No 0 0

Relationship class

NamePrimary keyForeign key

TypeCardinalityNotification

Forward labelBackward label

No relationship rules defined.

DataSourceHasHaul

Origin table Destination feature class

SimpleOne to manyNone

HaulDataSource

DataSourceDataSourceIDDataSourceID

HaulName

Relationship class

NamePrimary keyForeign key

TypeCardinalityNotification

Forward labelBackward label

No relationship rules defined.

HaulHasFishCatch

Origin feature class Destination table

SimpleOne to manyNone

FishCatchHaul

HaulHAULJOINHAULJOIN

FishCatchName

Role of Data Model TemplatesRole of Data Model Templates

Data ContentData Content

•• Multiple Multiple Scales of UseScales of Use–– NeighborhoodNeighborhood–– CityCity–– RegionalRegional–– State and NationalState and National

•• Emphasis on Data Emphasis on Data Sharing/Publication Sharing/Publication Models and Best Models and Best PracticesPractices

The Data Modeling CycleThe Data Modeling Cycle

ArcGIS Data Models Web siteArcGIS Data Models Web sitehttp://http://support.esri.com/datamodelssupport.esri.com/datamodels

•• Over 25 industryOver 25 industry--specific data modelsspecific data models•• Conceptual and logical diagramsConceptual and logical diagrams•• Case studies, Tips and Tricks documentsCase studies, Tips and Tricks documents

Data Models that ESRI ProducesData Models that ESRI ProducesCreated and owned by user groups and industriesCreated and owned by user groups and industries

Forestry

Census

IHO S-57Defense-Intel Local Govt

Marine

Health

Agriculture

TransportationWater Utilities

Hydro

Telecom-munications

Address

Archaeology

Energy Utilities

Geology

Land Parcels

Petroleum

Basemap

Biodiversity

Env RegFacilities

Atmospheric

Pipeline

What is a Schema?What is a Schema?

•• In the GIS it is a list of the In the GIS it is a list of the names of contents and names of contents and rules stored in the rules stored in the Geodatabase.Geodatabase.

•• Organized approach of Organized approach of grouping knowledge.grouping knowledge.

•• The concepts and actions The concepts and actions that can be revised and that can be revised and used to create a database.used to create a database.

How Do I Use a Template?How Do I Use a Template?

•• You may only need a small part of the over all You may only need a small part of the over all schema and template to match your dataschema and template to match your data

Physical ModelPhysical ModelDatabase SchemaDatabase Schema

Business RulesBusiness Rules

Conceptual ModelConceptual ModelSketches, Flow Diagrams, etc.Sketches, Flow Diagrams, etc.

The Layer StackThe Layer StackThematic groupings of data setsThematic groupings of data sets

TemplatesTemplates

OwnerParcelOwnerParcel

ParcelIDParcelIDParcelLocalLabelParcelLocalLabelParcelTypeParcelTypeParcelNameParcelNameOwnerClassificationOwnerClassificationOwnerOrManagingAgencyOwnerOrManagingAgencyLocalAreaLocalAreaAreaTypeAreaType

3D Polygon3D Polygon

3D Geometry3D GeometryHeightHeight

**EncumbranceEncumbrance

EncumbranceIDEncumbranceIDEncumbranceTypeEncumbranceTypeEncumbranceOwnerEncumbranceOwnerLegalAreaLegalAreaAreaTypeAreaType

SeparateRightSeparateRight

SeparateRightIDSeparateRightIDRightOwnerRightOwnerTenureTenureRightTypeRightTypeRightMineral[nRightMineral[n]]LegalAreaLegalAreaAreaTypeAreaType

VerticalParcelVerticalParcel

ParcelIDParcelIDPrimaryParcelIDPrimaryParcelIDVerticalParcelOwnerVerticalParcelOwnerLegalAreaLegalAreaAreaTypeAreaTypeSequence[n]Sequence[n]

3D Model/CAD file3D Model/CAD file

Document/Image URLDocument/Image URL

ApartmentComplexApartmentComplex

ComplNumComplNum

LegalLandObjectLegalLandObject

LegalTopicLegalTopic

ApartmentUnitApartmentUnit

ApartmentNumApartmentNum

* ** *

**

0..10..1

2..*2..*

•• WellWell--defined data types and elements defined data types and elements •• Can be used as a starting point to create a geodatabaseCan be used as a starting point to create a geodatabase

Using a Design TemplateUsing a Design TemplateSchema Wizard reads repository or template to create a Schema Wizard reads repository or template to create a geodatabasegeodatabase

WhatWhat’’s in a Data Model Template?s in a Data Model Template?•• A preA pre--designed schema of Objectsdesigned schema of Objects•• Feature classesFeature classes•• TablesTables•• RelationshipsRelationships•• DomainsDomains•• SubtypesSubtypes

Geoprocessing ModelsGeoprocessing ModelsModel Builder diagrams for workflowModel Builder diagrams for workflow

Raster in WGS84

extract_west Shifted_west

Outputgrid name

Extract_east

Raster in WGS84

Extract byRectangle (2)

OutputExtent

OutputExtent

OutputExtent

Shift

Extract byRectangle (3)

Extract byRectangle

Mosaic

OutputExtent

Geodatabase Geodatabase DiagrammerDiagrammerCreate graphical representation of geodatabase once completeCreate graphical representation of geodatabase once complete

TopicsTopics

•• IntroductionIntroduction

•• Data model program updatesData model program updates•• Opportunities for Developers at ArcGIS 9.2Opportunities for Developers at ArcGIS 9.2

•• ApplicationApplication--Driven DesignDriven Design•• Leveraging Data Model TemplatesLeveraging Data Model Templates

……and some information on:and some information on:

Updates on Data Model ProgramsUpdates on Data Model Programs•• 30 data models on 30 data models on http://http://support.esri.com/datamodelssupport.esri.com/datamodels•• New contentNew content

–– Data Model Beta Program for GroundwaterData Model Beta Program for Groundwater–– Data Model Data Model ““ToolsTools”” section on Downloads section on Downloads

page page •• Recent updatesRecent updates……•• Planned activitiesPlanned activities……•• PublicationsPublications……

Opportunities at 9.2Opportunities at 9.2–– ETL Extract Transform and Load tools for ETL Extract Transform and Load tools for

managing DB designmanaging DB design–– Tools for updating test and production systemsTools for updating test and production systems

GeoWebGeoWeb

Interoperability Procedure (ETL)Interoperability Procedure (ETL)

Semantic Semantic TranslationTranslationStatesStates NationalNational

Data TransformationData Transformation•• FormatsFormats•• Data ModelsData Models•• SchemaSchema•• Projections / DatumProjections / Datum

Opportunities at 9.2Opportunities at 9.2–– NetCDFNetCDF tools tools –– Transportation applications and ways to Transportation applications and ways to

use network analyst toolsuse network analyst tools–– Groundwater/Earth Sciences modelingGroundwater/Earth Sciences modeling

PointsPoints

TablesTables

RastersRasters

YY

XX

TimeTime

Opportunities at 9.2Opportunities at 9.2

•• 9.2 provides new capabilities that have some 9.2 provides new capabilities that have some impact on data modelsimpact on data models–– Short transaction modelShort transaction model

•• For simple features onlyFor simple features only•• Developers need to handle undo/redo Developers need to handle undo/redo

and transaction scopeand transaction scope–– History in the GeodatabaseHistory in the Geodatabase–– Workgroup and Personal SDEWorkgroup and Personal SDE

For More InformationFor More Information

•• ESRI Virtual CampusESRI Virtual Campus

–– Using CASE Tools (for Using CASE Tools (for ArcEditorArcEditor and ArcInfo)and ArcInfo)–– Intro to data models Live Training Seminar (LTS)Intro to data models Live Training Seminar (LTS)–– Creating, Editing, and Managing Geodatabases Creating, Editing, and Managing Geodatabases

for ArcGIS 9for ArcGIS 9–– Creating and Editing Geodatabase FeaturesCreating and Editing Geodatabase Features

•• ESRI Instructor ESRI Instructor -- Led TrainingLed Training–– Modeling Geodatabases Using CASE ToolsModeling Geodatabases Using CASE Tools–– Geodatabase Design ConceptsGeodatabase Design Concepts–– Building Geodatabases IBuilding Geodatabases I–– Building Geodatabases IIBuilding Geodatabases II

For More InformationFor More Information

•• ArcGIS Data Models Web siteArcGIS Data Models Web sitehttp://http://support.esri.com/datamodelssupport.esri.com/datamodels

•• ESRI Press BookESRI Press Book

–– Modeling Our World: The ESRI Guide Modeling Our World: The ESRI Guide to Geodatabase Design to Geodatabase Design

–– Designing Designing GeodatabasesGeodatabases: Case : Case Studies in GIS Data ModelingStudies in GIS Data Modeling

•• Product BookProduct Book–– Building a GeodatabaseBuilding a Geodatabase

Session Evaluations ReminderSession Evaluations Reminder

Session Attendees:Session Attendees:Please turn in your session evaluations.Please turn in your session evaluations.

. . . Thank you. . . Thank you

Thanks for ParticipatingThanks for Participating

Joe Breman Joe Breman jbreman@esri.comjbreman@esri.comSteve GrisSteve Griséé sgrise@esri.comsgrise@esri.com