Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Brian Marwick
Manager
Spatial Information Services
Several Projects undertaken by
LogicaCMG using Oracle Locator
Overview of Presentation
� LogicaCMG
� Building and maintaining PSMA Australia’s
national databases
� G-NAF and Transport
� Integrating Oracle Spatial into Smallworld
environments
3
LogicaCMG - Global Presence
United Arab Emirates
United Kingdom
Netherlands
New Zealand
Switzerland
Sweden
United States
Saudi Arabia
Indonesia
Canada
Germany
Belgium
Italy
Czech Republic
Australia
Hong Kong
Malaysia
Japan
France
Luxembourg
India
Ireland
Brazil
Norway
• Formed in December 2002 through the merger of Logica and CMG, the company has over 40 years of combined experience in the IT services arena.
• Listed on the London (FTSE 250 &
techMARK 100) & Amsterdam (AEX)
Stock Exchanges.
• A major international force in IT services and wireless telecoms with a leading position in Europe.
• 28,000 people in 36 countries.
• A broad portfolio of offerings across key industry sectors.
• A worldwide client base of blue-chip organisations.
4
LogicaCMG - Australia
Projects & Services
• IT services with YVW, CWW, TXU, Aurora, Transend,
Hydro, Eraring Energy, Pacific Power, Integral Energy…..
• SMARTBuy and e Health, NSW
• Gas Retail Market NSW, ACT, SA and WA
• Metering for TCA and Infomet
• G-NAF and Roads for PSMA Australia
• Customer information & billing at IE, AE & YVW
• SAP implementation support for >35 companies
• GIS at over ten utilities ( S’world, Intergraph, ESRI )
• Vicmap Property data management
• Department of Education and Training, Victoria
• nemPOWER with six utilities
• SCADA & SMS in electricity, gas and water
• Victorian Gas Market with OEAM/Envestra
• Alinta Gas Networks and Sales
• Commenced operations in 1974
• Industry Sector Focus
• Financial Services
• Government
• Energy and Utilities
• Industry, Distribution and Transport
• Telecommunications
• Strategic Growth
• Acquired MITS in late 2000
• Acquired eGlobal in 2002
• Merger with CMG on 1/1/2003
• Local Staff about 850
• IT skills in
• Systems Integration
• Managed Services
• Application development and management
• Field & Equipment Maintenance
5
� PSMA Australia Limited (Public Sector Mapping Agencies Australia) is an unlisted public company, established under the Corporations Act (2001), wholly owned by the state, territory and Australian Governments.
� PSMA Aust. draws on the public sector’s extensive, geospatial data resources to co-ordinate the assembly of, and facilitate access to fundamental national spatial datasets that deliver economic, environmental and social benefits to Australia
� G-NAF / Transport / Cadastral / Administrative boundaries databases
Client - PSMA Australia Limited
6
PSMA Australia’s Transport dataset
PSMA Australia Transport Dataset
ROADNAME
ROAD
TYPE
PERSISTENT
ID
CREATE
DATE
RETIRE
DATE
EXPERSISTENT
ID
SURFACE
TYPE
GROUND
RELATIONSHIP LOCALITY_1 LOCALITY_2
RIVERINA HIGHWAY 301 NSW501565482 19981029 501565482 1 3 ALBURY EAST ALBURY
UNNAMED 305 NSW501565025 19981029 501565025 3 3 ALBURY EAST ALBURY
WILSON STREET 305 NSW501565321 19981029 501565321 1 3 ALBURY EAST ALBURY
DELANY STREET 305 NSW501563446 19981029 501563446 1 3 ALBURY GLENROY
NORTH STREET 301 NSW501565782 19981029 501565782 1 3 ALBURY NORTH ALBURY
NORTH STREET 305 NSW501565811 19981029 501565811 1 3 ALBURY NORTH ALBURY
NORTH STREET 301 NSW501565773 19981029 501565773 1 3 ALBURY NORTH ALBURY
EDMONDSON AVENUE 305 NSW501565595 19981029 501565595 1 3 ALBURY NOT REQUIRED
WILCOX STREET 305 NSW501565168 19981029 501565168 1 3 ALBURY NOT REQUIRED
THURGOONA STREET 305 NSW501565307 19981029 501565307 1 3 ALBURY NOT REQUIRED
DEAN STREET 305 NSW501565169 19981029 501565169 1 3 ALBURY NOT REQUIRED
KIEWA STREET 305 NSW501565657 19981029 501565657 1 3 ALBURY NOT REQUIRED
UNNAMED 305 NSW501565073 19981029 501565073 3 3 ALBURY NOT REQUIRED
CRISP STREET 305 NSW501565573 19981029 501565573 1 3 ALBURY NOT REQUIRED
QUEEN STREET 305 NSW501565801 19981029 501565801 1 3 ALBURY NOT REQUIRED
GOLF VIEW DRIVE 305 NSW501565834 19981029 501565834 1 3 ALBURY NOT REQUIRED
STANLEY STREET 305 NSW501565286 19981029 501565286 1 3 ALBURY NOT REQUIRED
THURGOONA STREET 305 NSW501565477 19981029 501565477 1 3 ALBURY NOT REQUIRED
AVONDALE DRIVE 305 VIC5897193 20041209 5897193 1 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 305 VIC5897180 20041209 5897180 1 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 305 VIC5897257 20041209 5897257 1 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 305 VIC5897270 20041209 5897270 1 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 305 VIC5897333 20041209 5897333 1 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 308 VIC10075713 20041209 10075713 3 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 308 VIC9929444 20041209 9929444 3 3 WODONGA NOT REQUIRED
AVONDALE DRIVE 305 VIC5897083 20041209 5897083 1 3 WODONGA NOT REQUIRED
AWBURN STREET 305 VIC5894068 20041209 5894068 1 3 WODONGA NOT REQUIRED
BAELON STREET 305 VIC5891540 20041209 5891540 1 3 WODONGA NOT REQUIRED
PSMA Australia: G-NAF
• G-NAF is Australia’s first authoritative index of locality, street, number and geographic coordinate (geocode)
• Addresses in G-NAF have been supplied by the States and Territories of Australia, AEC and AUSP
• G-NAF has been built and now maintained using a
methodology developed by PSMA and implemented by LogicaCMG and Geometry
• The methodology uses a range of spatial and aspatial validation techniques
• First released in March 2004
• Updated every three months – 7 updates completed
9
CONFIDENCE GNAF_PID
LEGAL_
PARCEL_ID
FLAT_
NUMBER
NUMBER_
FIRST
STREET_
NAME
STREET_
TYPE
LOCALITY_
NAME
STATE_
ABBREVIA
TION
DATE_GEOCODE
_CREATED LATITUDE LONGITUDE
2 GANSW705382383 DP/28//708269 4 ILUKA AVENUE MOAMA NSW 20040415 -36.10550 144.75700
2 GANSW705382384 DP/27//708269 5 ILUKA AVENUE MOAMA NSW 20040415 -36.10540 144.75600
2 GANSW705382385 26//708269 6 ILUKA AVENUE MOAMA NSW 20040415 -36.10530 144.75600
2 GANSW705382410 DP/3//801078 11 KULGOA AVENUE MOAMA NSW 20040415 -36.10410 144.75600
2 GANSW705374532 DP/1//195161 13 SIMMS STREET MOAMA NSW 20040415 -36.11480 144.76100
2 GANSW705374534 DP/5/29/758686 16 SIMMS STREET MOAMA NSW 20040415 -36.11420 144.76200
2 GANSW705374536 DP/2//504342 19 SIMMS STREET MOAMA NSW 20040415 -36.11500 144.76200
2 GANSW705374537 DP/5//719506 2 SIMMS STREET MOAMA NSW 20040415 -36.11420 144.76000
2 GANSW705374538 SP/1//19738 1 20 SIMMS STREET MOAMA NSW 20040415 -36.11430 144.76200
2 GANSW705374539 SP/2//19738 2 20 SIMMS STREET MOAMA NSW 20040415 -36.11430 144.76200
2 GANSW705374540 SP/3//19738 3 20 SIMMS STREET MOAMA NSW 20040415 -36.11430 144.76200
PSMA Australia : G-NAF
10
Statistics – G-NAF (Feb 2006 release)
PSMA Australia : G-NAF
Reliability No of Addresses %
1 (GPS derived address level) 0 0%
2 ( Within address site boundary) 10,662,127 87%
3 (gap geocoded ) 195,951 2%
4 (street Level) 1,278,035 10%
5 (locality level) 207,429 2%
6 (topo level) 1,817 0%
no geocode 0 0%Total 12,305,359 100%
SUMMARY OF GEOCODE RELIABILITY OF PRINCIPAL ADDRESSES
Confidence level No of Addresses %
Confidence of 0 3,925,756 32%
Confidence of 1 2,253,763 18%
Confidence of 2 6,125,840 50%12,305,359 100%
SUMMARY OF CONFIDENCE OF PRINCIPAL ADDRESSES
11
Statistics – PSMA Data used for G-NAF
PSMA Australia : G-NAF
STATE
Roads
(segments)
G-NAF
(Principal Addresses)
Total Addresses from
Contributors LOCALITIES
NSW 1,016,489 4,226,893 8,803,421 4646
VIC 685,484 2,862,919 6,741,026 2948
QLD 312,739 2,519,369 5,144,668 3404
WA 508,047 1,217,910 2,743,021 1684
SA 506,780 919,865 2,049,991 1610
TAS 261,325 293,303 710,765 1107
ACT 24,344 178,089 433,747 104
NT 36,084 85,465 182,506 113
OT 1,917 1,546 1,657 0
3,353,209 12,305,359 26,809,145 15616
12
Technology Environment
• Dell 2650
• Windows 2003 Server
• Dell 2650 • Dell 2650 • Dell 2850
• Oracle Locator
• G-NAF Java Application
FME AutoCAD Map AutoCAD Map
Applications
13
Example of Processing for PSMA Roads
14
Example of Processing for PSMA Roads
Determining localities in which road segments are located.
15
Example of Processing for PSMA Roads
• Caulfield South
Ormond
10mm
• Road centreline
• Locality Boundary
16
Example of Processing for PSMA Roads
• Caulfield South
Ormond
• Road centreline
• Locality Boundary
Buffer added to each locality to allow inclusion of road segments not exactly aligned to the locality.
Currently investigating the options to enable vertical topology
10mm
17
Data Processing for G-NAF
Datasets used
• G-NAF (addresses)
• Roads (road segments)
• Localities (polygons)
• CadLite (land parcels)
• Place Names
Gazetteer (all gazetted
Places names in Aust.)
Oracle Tables
Pre-processing
phase primarily
using FME
Core G-NAF
processing using
java app.
Post processing
phase primarily
using FME
18
Overview of G-NAF Methodology
2. Locality Name
1. Initial Validation
Is the address complete and
does it comply with the required
format? Rectify if possible
Are there any known address
inconsistencies? Apply rules
19
Overview of G-NAF Methodology
2. Locality Name
3. Street Name
1. Initial Validation
Is the address complete and
does it comply with the required
format? Rectify if possible
Are there any known address
inconsistencies? Apply rules
20
Overview of G-NAF Methodology
2. Locality Name
3. Street Name4. Street Number
1. Initial Validation
Is the address complete and
does it comply with the required
format? Rectify if possible
Are there any known address
inconsistencies? Apply rules
21
Examples of Spatial processing
• PSMA roads database stores individual segments between road intersections
22
Examples of Spatial processing
ROADNAME ROADTYPE PERSISTENTID CREATEDATE LOCALITY_1
RIVER ROAD 302 NSW500168676 19980702 GREENWICH
RIVER ROAD 302 NSW500168691 19980702 GREENWICH
RIVER ROAD 302 NSW500168960 19980702 GREENWICH
RIVER ROAD 302 NSW505141094 20010501 GREENWICH
RIVER ROAD 302 NSW505139241 20010501 GREENWICH
RIVER ROAD 302 NSW500168821 19980702 GREENWICH
RIVER ROAD 302 NSW500168900 19980702 GREENWICH
RIVER ROAD 302 NSW500168948 19980702 GREENWICH
RIVER ROAD 302 NSW500168902 19980702 GREENWICH
RIVER ROAD 302 NSW500168911 19980702 GREENWICH
RIVER ROAD 302 NSW500168862 19980702 GREENWICH
RIVER ROAD 302 NSW500168800 19980702 GREENWICH
RIVER ROAD 302 NSW505193609 20010604 LANE COVE
RIVER ROAD 302 NSW505139264 20010501 LANE COVE
RIVER ROAD 302 NSW505139263 20010501 LANE COVE
RIVER ROAD 302 NSW505193610 20010604 LANE COVE
RIVER ROAD 302 NSW505193609 20010604 LANE COVE
RIVER ROAD 302 NSW505139264 20010501 LANE COVE
RIVER ROAD 302 NSW505139263 20010501 LANE COVE
RIVER ROAD 302 NSW505193610 20010604 LANE COVE
RIVER ROAD 302 NSW505193593 20010604 NORTHWOOD
RIVER ROAD 302 NSW500168808 19980702 ST LEONARDS
RIVER ROAD 302 NSW500168865 19980702 ST LEONARDS
RIVER ROAD 302 NSW505139191 20010501 ST LEONARDS
RIVER ROAD 302 NSW500168729 19980702 ST LEONARDS
RIVER ROAD 302 NSW500168863 19980702 ST LEONARDS
RIVER ROAD 302 NSW505139166 20010501 WOLLSTONECRAFT
RIVER ROAD 302 NSW500168568 19980702 WOLLSTONECRAFT
RIVER ROAD 302 NSW500168461 19980702 WOLLSTONECRAFT
RIVER ROAD 302 NSW500168696 19980702 WOLLSTONECRAFT
RIVER ROAD 302 NSW505139179 20010501 WOLLSTONECRAFT
RIVER ROAD 302 NSW505139190 20010501 WOLLSTONECRAFT
RIVER ROAD 302 NSW500168577 19980702 WOLLSTONECRAFT
RIVER ROAD 302 NSW505139180 20010501 WOLLSTONECRAFT
RIVER ROAD 302 NSW500168707 19980702 WOLLSTONECRAFT
23
Examples of Spatial processing
• Individual segments for the entire length of River road are joined to create PSMA Road line complete
• Road line complete is then segmented into the respective locality polygons in which exists
24
Examples of Spatial processing
• Creation of Street Locality Geocode for Street within Locality
25
Examples of Spatial processing
NSW
• Locality polygons – 4646
• Addresses – 2.9 million
• Road Segments – 1.1 million
Point in polygon of each Jurisdictional address to ensure each address has the correct locality
26
Examples of Spatial processing
Many complex polygons as a locality boundaries following cadastral boundaries, road centrelines and coastlines
27
Examples of Spatial processing
Example of locality fix
Use Oracle to sub-set addresses and then undertake point in polygon using FME for each locality polygon.
28
Summary – PSMA Datasets
• Maintenance of PSMA datasets involves considerable spatial data processing
• Considerably flexibility has been required to address various data quality issues as they have been identified.
• The size of the larger State datasets initially impacted significantly processing speeds however the spatial functionality of Oracle Locator together with FME has allow the development of cost effective solutions
• Oracle Locator has provided a reliable platform to build and maintain the G-NAF and Transport datasets for PSMA using a range of software applications of our choosing.
29
Oracle in a Smallworld Environment
Case 1: Replacing Smallworld VMDS with Oracle Schema
• Replacing a Smallworld native database with an Oracle Schema using Smallworld’s Spatial
Object Controller (SOC) to treat Oracle as if it were a native dataset
– Richer range of tools for manipulating data in an Oracle Database.
– More accessible skills in the Oracle development space.
– Data could be maintained in-house using standard PL/SQL skills.
Smallworld GIS
SOC VMDS SOC OracleSOC VMDS
Data to replace
(cadastre)
New Dataset
(cadastre)
Asset data
30
Requirements
• Create New Schema in Oracle for cadastral data
• Load up Oracle Schema with New version of data
• Re-configure Smallworld Spatial Object Controller for new Dataset
• Decommission old Smallworld VMDS data partition
• Smallworld Features to be retained
–Richer Geometry types for Points and Text Labels
–Fast Drawing within the Smallworld Application
• Smallworld Features not required in this instance
–Geometry Topology
–Version Management
Case 1 : Replacing Smallworld VMDS with Oracle Schema
31
Solution:
• Store properties as extra fields in table.
• Use extra code (style methods) to render SDO points in Smallworld.
• Smallworld Point
–Coordinate (x,y,z)
–Angle
–Scale
–Mirror
• Oracle SDO_Point
–Coordinate (x,y,z)
Modelling Smallworld Points in Oracle
Case 1 : Replacing Smallworld VMDS with Oracle Schema
Other Possibility:
•Extend SDO Object model to handle other attributes
–Not compatible with the Smallworld interface to Oracle Geometry. Only standard types supported.
32
Modelling Smallworld Text in Oracle
Solution:
• Store missing properties as extra fields in Oracle.
• Use extra code (style methods) to render SDO points in Smallworld using the extra fields.
• Smallworld Label
–Coordinate (x,y,z)
–Angle
–Scale
–Draw Direction
–Justification Point 25 point
–String
• Oracle SDO_Point
–Coordinate (x,y,z)
Text
TextText
Text
Case 1 : Replacing Smallworld VMDS with Oracle Schema
Other Possibility:
•Extend SDO Object model
33
Speed of Refresh in GIS• Challenge
– Time to perform a screen refresh of Oracle data in Smallworld GIS similar to native geometry in VMDS.
• Observations
– Time to perform spatial query in Oracle faster than same query in VMDS.
– Creating Materialised Views of restricted data to be normally displayed in GIS created a significant performance improvement.
– Recreated R-Tree after large updates.
– Unable to cache the Oracle data in the GIS as data could have changed with each redraw
• Result:
– Achieved performance close to drawing native data
Case 1 : Replacing Smallworld VMDS with Oracle Schema
34
Case 2: Smallworld Mirror in Oracle
• Requirement to have all corporate databases available in Oracle.
• Other Benefits:
– Able to perform comparative queries of data in GIS and other corporate systems more efficiently
– Able to use other tools to access GIS data (eg MapInfo)
– Some types of queries are more efficiently handled by Oracle.
– Data became more accessible to a wider range of users.
35
Technology
• InSync
– Connects to Oracle using a Smallworld Spatial Object Controller
– Uses a checkpoint to note the differences between the current database version and when the last Synchronisation occurred
• Checkpoints: similar to a save point in Workspace Manager
– A mapping database is created to describe a translation between Oracle and Smallworld, tables and fields
Smallworld GIS VMDS SOCOracle SOC
current synch
last synch
Inserts, Updates, Deletes
Case 2: Smallworld Mirror in Oracle
36
Data Modelling
• Differences between Oracle and Smallworld table naming
– 30 character name length, special characters, reserved words
• Hidden tables in Smallworld
– Intermediate Tables, Discrete Lists, Topology, Configuration, Datamodel metadata
– Auto-generated table names and fields (eg. int!miscellaneous_miscellaneous5)
• Differences between geometry engines
– self intersecting sectors and other geometry errors, topology, extra properties of text and point geometry
Case 2: Smallworld Mirror in Oracle
37
Modelling Smallworld Text and Points
Solution:
• Store missing properties as extra fields in Oracle using a convention that can be handled by code.
• InSync modified to update the extra property fields in Oracle.
• Smallworld Label
–Coordinate (x,y,z)
–Angle
–Scale
–Draw Direction
–Justification Point 25 point
–String
• Oracle SDO_Point
–Coordinate (x,y,z)
Text
TextText
Text
Case 2 : Smallworld Mirror in Oracle
Other Possibility:
• Extend SDO Object model to handle other properties (new geometry types)
–Not compatible with the Smallworld/Oracle Access
methods
38
Summary – Oracle in Smallworld Environment
• Organisations are successfully implementing a variety of solutions to enable either:
– improved access by corporate databases to their established GIS’s.
– the incorporation of externally maintained data
• Some issues do exist with regard the management of text and other model issues however once identified these can be resolved
39
The End