11

W. Fredrick LimpUniversity Professor

Robert HarrisDBA

Center for Advanced Spatial TechnologiesUniversity of Arkansas

Spatially Enabling the Spatially Enabling the DBMSDBMS

Design, implementation, Design, implementation, operation and implications for operation and implications for

the futurethe future

22

Outline Review of geospatial data structures and Review of geospatial data structures and

their relationship to “traditional” DBMS their relationship to “traditional” DBMS data typesdata types

Feature and feature-class based spatial Feature and feature-class based spatial data modelsdata models

Data storage and indexing in enterprise Data storage and indexing in enterprise geospatial systemsgeospatial systems

OpenGIS and enterprise systemsOpenGIS and enterprise systems SQL operations in 10GSQL operations in 10G Examples of on-line systemsExamples of on-line systems

33

Initial spatial data model

Paradigm was a mapParadigm was a map Map layer theme fileMap layer theme file Data model corresponded to Data model corresponded to

technology drivertechnology driver Developed in response to “slow” Developed in response to “slow”

display hardware/software display hardware/software performance and need to be performance and need to be responsiveresponsive

44

The attributed graphics model

Graphic files were Graphic files were linked to attributes in linked to attributes in a separate file and a separate file and later DBMS tablelater DBMS table

Allowed user to Allowed user to conduct searches on conduct searches on and get graphic and get graphic element(s) that was element(s) that was associatedassociated

Graphic Info element 1 Road 2 water line 3 valve

1

2

55

Replaced by - blob-based entity spatial model

Software developers take advantage of a Software developers take advantage of a special type of record in database - the special type of record in database - the blobblob– a blob is a chunk of disk space that the DBMS a blob is a chunk of disk space that the DBMS

keeps track of but otherwise just knows keeps track of but otherwise just knows “something is there”“something is there”

Changes view of basic “thing” recordedChanges view of basic “thing” recorded Spatial entities become the “element” Spatial entities become the “element”

– a particular road segment, a particular parcela particular road segment, a particular parcel

66

Database storage in this model

Lot 2

Lot 1

Tract layer

Tract 1 population an attribute Geometric stuff another attribute

94102

94103

234

567

100

100

blob

blob

Columbia

Jefferson

All geometric stuff in a “blob”

77

BLOB problems (1)

BLOB contents are not managed BLOB contents are not managed by DBMSby DBMS– BLOB contents are unknown to DBMSBLOB contents are unknown to DBMS– Managed by 3-rd party softwareManaged by 3-rd party software– All operations external to DBMSAll operations external to DBMS

BLOBs do not reflect known data BLOBs do not reflect known data typestypes

88

BLOB problems (2) Definition of a thingDefinition of a thing

– This is now clear within a single This is now clear within a single themetheme

– but still problem in confusion between but still problem in confusion between themesthemes

District 1

The City

Smith RoadWhat’s this ???

What happens if the road is moved?

99

Theme and element confusion

What if the “road” changes - and it is also What if the “road” changes - and it is also the legal definition of the city limit and the legal definition of the city limit and voting district boundary?voting district boundary?

If the road is an entity then perhaps the If the road is an entity then perhaps the road department updates itroad department updates it– then the city limit and voting district then the city limit and voting district

boundaries are now out-of-dateboundaries are now out-of-date BUT maybe that’s correct - the road BUT maybe that’s correct - the road

changes but the district boundary doesn’t!changes but the district boundary doesn’t!

1010

Spatial operations in this model

How do you find a geographic thingHow do you find a geographic thing– find all old street segments within Alderman find all old street segments within Alderman

Smith’s districtSmith’s district old is an attributeold is an attribute ““district” is a geographic areadistrict” is a geographic area

““IN”, “BESIDE” etc. are viewed as IN”, “BESIDE” etc. are viewed as topological relationshipstopological relationships

Requires specialized “GIS” operations Requires specialized “GIS” operations and possibly specialized data structureand possibly specialized data structure

DBMS simply the data storeDBMS simply the data store– All operations in client or middle-wareAll operations in client or middle-ware

1111

Enterprise systems and interoperability

Spatial data has (to date) been Spatial data has (to date) been stored in many different internal stored in many different internal formatsformats

Common that each software vendor Common that each software vendor has a different spatial data formathas a different spatial data format

Different formats made exchange Different formats made exchange of data difficult (if not impossible)of data difficult (if not impossible)

How to solve this in an enterprise How to solve this in an enterprise context?context?

1212

Solutions Old solutionOld solution

– Everybody got the same software from the same Everybody got the same software from the same vendor vendor OROR

– ORGANIZATIONAL RULE was passed ORGANIZATIONAL RULE was passed – Problems with thisProblems with this

one size doesn't fit allone size doesn't fit all limits technological innovationlimits technological innovation

New solutionNew solution– Open GIS Open GIS (TM) (TM) consortium and interoperability consortium and interoperability

specificationsspecifications– Multiple vendors = competitionMultiple vendors = competition– Lower cost and increased capacityLower cost and increased capacity– Flexibility by unitsFlexibility by units

1313

Definition of OpenGIS®

Open and interoperable geoprocessing:

The ability to share heterogeneous geodata and geoprocessing resources transparently in a network environment.

1414

OpenGIS Specification is an Interface Standard (not a data or transfer)

standard

• OpenGIS-compliant clients can be understood by OpenGIS-compliant servers:– Set or get geodetic datum and projection– Create, modify, copy, delete features in the

database– Select features from the database by query– Execute GIS commands– And many others...

– Adopted by ISO (TC211)

1515

Simple features model

With development led, in large With development led, in large part, by DBMS vendors, a new part, by DBMS vendors, a new spatial data model is createdspatial data model is created

Requires some new algorithms Requires some new algorithms But otherwise can utilize “normal” But otherwise can utilize “normal”

relational and/or object DBMSrelational and/or object DBMS

1616

Basic Data ModelBasic Data Model

P o in t L in e P o lyg on

E le m e n ts

G e o m e tries

S p a tia l L a yer

P o in t L in e P o lyg on

E le m e n ts

G e o m e tries

S p a tia l L a yer

1717

Base Elements

Basic building blocks of a Basic building blocks of a geometrygeometry

Element typesElement types– Point (1)Point (1)– Line (2)Line (2)– Polygon (3)Polygon (3)

Constructed using Constructed using coordinates coordinates

No limit on number of No limit on number of coordinatescoordinates

1818

Extended Elements

PointsLine/Circular Arc StringsPolygonsRectangle optimized for spaceCirclesCompound elements without

areaCompound elements with area

1919

Geometry

Represents a Represents a spatial featurespatial feature

Consists of an Consists of an ordered set of ordered set of primitive elementsprimitive elements

Uniquely identified Uniquely identified with a geometry with a geometry identifieridentifier

Geometry 1Florida, USA

Geometry 2Hawaii, USA

2020

Layers

Consists of Consists of geometries that geometries that share a common share a common set of attributesset of attributes

Stored using a set Stored using a set of tablesof tables

State layer

2121

Conversion from map tile to feature - element in ORDBMS

Map tile by “theme” Enterprise system’s feature classes

2222

Database properties Allows storage of individual elementAllows storage of individual element

– each segment of road is potentially accessibleeach segment of road is potentially accessible– a geometric element serve multiple purposesa geometric element serve multiple purposes

road segmentroad segment city limit segmentcity limit segment voting district segmentvoting district segment

““Automatic” updates of all associated Automatic” updates of all associated “themes” because they are simply DBMS “themes” because they are simply DBMS linkageslinkages

2323

Potential problems with this approach

Zillions of elementsZillions of elements How do you keep track of where they How do you keep track of where they

livelive– rapid access to a record in a DBMS provided rapid access to a record in a DBMS provided

by INDEXby INDEX– index means that all records do not need to index means that all records do not need to

be sequentially accessed and checkedbe sequentially accessed and checked How to index spatial elementsHow to index spatial elements

– attributes are X, Y (and maybe Z)attributes are X, Y (and maybe Z)

2424

Quad tree indexing

A full tessellation of spaceA full tessellation of space

2525

Morton Codes (HHcode) for Tiles

.---

32

10

02

00

03

01 10 11

13

3121 30

12

20

22 23 32 33

002

000

003

001

012

010

013

011

033032

031030

023022

021020

102

100

103

101

112

110

113

111

133132

131130

123122

121120

302

300

303

301

312

310

313

311

333332

331330

323322

321320

202

200

203

201

212

210

213

211

233232

231230

223222

221220

2626

Use quad trees to spatially index

002

000

003

001

012

010

013

011

033032

031030

023022

021020

102

100

103

101

112

110

113

111

133132

131130

312

310

313

311

333332

331330

323322

321320

202

200

203

201

233232

231230

223222

221220

30

12

21

GROUP CODE

03 031

03 033

12 12

21 21

23 230

23 231

30 30

32 320

32 321

<layer>_SDOINDEX

SDO_GID SDO_CODE SDO_MAXCODE

1013 T2 T2FFFF…F

1013

1013

T320

T302

T320FF…F

T302FF…F

2727

Spatial indexing implications

Spatial joins (Spatial joins (a a VERY BIG DEAL!VERY BIG DEAL!))– IF same extent is IF same extent is

used to create a used to create a quad tree index quad tree index then the same index then the same index number corresponds number corresponds to the same place to the same place on the earthon the earth

Address layer

Other layer

Geocode returns Index location

2828

R-tree index

Commercially introduced by Commercially introduced by Informix Spatial CartridgeInformix Spatial Cartridge– From Illustra technologyFrom Illustra technology– Data (not space)is partitionedData (not space)is partitioned

Bounding boxes (MBR) around each Bounding boxes (MBR) around each element are createdelement are created

MBR encompassing lower groupsMBR encompassing lower groups Index of MBR “values”Index of MBR “values”

2929

R-trees

3030

R-trees

Variety of formsVariety of forms Are balanced index structuresAre balanced index structures Appear to be much fasterAppear to be much faster Probably not effective as quad-tree Probably not effective as quad-tree

in a very dynamic data environmentin a very dynamic data environment More readingMore reading

– http://www.geog.ubc.ca/courses/klink/http://www.geog.ubc.ca/courses/klink/gis.notes/ncgia/u37.htmlgis.notes/ncgia/u37.html

3131

Spatial queries

Two stagesTwo stages– Stage 1 - IndexStage 1 - Index

Based on spatial indexBased on spatial index RapidRapid Can dramatically reduce data sizeCan dramatically reduce data size

– Stage 2 - ExactStage 2 - Exact Uses spatial operatorsUses spatial operators Performs calculationPerforms calculation

3232

Query ModelQuery Model

LayerData

PrimaryFilter

INDEX

ReducedData Set

SecondaryFilter

QueryFunctions

ExactResult

Set

Table wherecoordinates are stored

Index retrieves area of interest (window)

Proceduresthat determineexact relationship

3333

Spatial Queries in SQL Based on Egenhofer Relationships Based on Egenhofer Relationships

Between GeometriesBetween Geometries– InsideInside– ContainsContains– CoversCovers– Covered ByCovered By– TouchTouch– Overlap Boundaries IntersectOverlap Boundaries Intersect– Overlap Boundaries DisjointOverlap Boundaries Disjoint– EqualEqual– DisjointDisjoint– AnyInteractAnyInteract

3434

INSIDE AND CONTAINS

B INSIDE AA CONTAINS B

A

B

3535

COVERS AND COVERED BY

B COVEREDBY AA COVERS B

A

B

3636

TOUCH

A TOUCHES BB TOUCHES A

A

B

3737

OVERLAP BOUNDARY INTERSECT

A OVERLAPBDYINTERSECT BB OVERLAPBDYINTERSECT A

A

B

3838

OVERLAP BOUNDARY DISJOINT

A OVERLAPBDYDISJOINT BB OVERLAPBDYDISJOINT A

A

B

3939

EQUAL

A EQUALS BB EQUALS A

A blue B green

4040

DISJOINT

A DISJOINT BB DISJOINT A

A

B

4141

ANYINTERACT AND DETERMINE

ANYINTERACT ANYINTERACT - Given two - Given two geometries, returns true if the two geometries, returns true if the two geometries are not disjointgeometries are not disjoint

DETERMINEDETERMINE - Given two - Given two geometries, returns their geometries, returns their relationship to each other.relationship to each other.

4242

Define an Area Of Interest

Area Of Interest

4343

Features Inside Area of Interest

INSIDE

4444

Client or server queries

Many applications that connect Many applications that connect with enterprise systems have with enterprise systems have option for queryoption for query– Server sideServer side– Client sideClient side

Will return to this question laterWill return to this question later

4545

Queries and indexing

PerformancePerformance– Retrieval of ALL valuesRetrieval of ALL values– Retrieval of selected setRetrieval of selected set– Data set sizeData set size

Under 100,00 featuresUnder 100,00 features 100,000 – 1,000,000 features100,000 – 1,000,000 features More than 1,000,000 featuresMore than 1,000,000 features

4646

Relationships

Formal “connections” between Formal “connections” between elements in two tables in a databaseelements in two tables in a database

ExamplesExamples– Transformer is “on” power poleTransformer is “on” power pole

Implies that geographic location of transformer is Implies that geographic location of transformer is dependant on location of pole – move pole move dependant on location of pole – move pole move transformertransformer

– Water meter is at end of water lineWater meter is at end of water line– Bridge is connected to street segmentBridge is connected to street segment– City limit is boundary of voting districtCity limit is boundary of voting district

4747

What happens in relationships

Messaging and/or “triggers”Messaging and/or “triggers” User performs an action on a feature that User performs an action on a feature that

has a relationshiphas a relationship Software recognizes existence of Software recognizes existence of

relationship and executes defined relationship and executes defined operationoperation

May require user input may be automaticMay require user input may be automatic Extensive use of relationships has Extensive use of relationships has

performance implicationsperformance implications Use has design complexitiesUse has design complexities

4848

Transactions and versioning

Enterprise systems provide methods to Enterprise systems provide methods to allow features to be updated while under allow features to be updated while under useuse– User 1 locks individual records are locked for User 1 locks individual records are locked for

edit/update purposesedit/update purposes– User 2 can access data but nor changeUser 2 can access data but nor change– If multiple users are updating rules define If multiple users are updating rules define

how multiple changes are appliedhow multiple changes are applied Versioning allows data configuration to Versioning allows data configuration to

be date definedbe date defined

4949

Metadata Spatial metadata vs applications metadataSpatial metadata vs applications metadata Spatial metadataSpatial metadata

– E.g.E.g. ISO TC 211 19115ISO TC 211 19115 FGDCFGDC

– Originally file orientedOriginally file oriented DateDate SourceSource

– Enterprise systems are dynamic feature/element Enterprise systems are dynamic feature/element structuresstructures

Date = date of last updateDate = date of last update

Movement to element based, dynamic Movement to element based, dynamic metadatametadata