Upload
haru
View
34
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Netlobs Manipulating Gridded Data in a Relational World Neil Stamps Technical Architect. Agenda. Introduction to Lost Wax The problem framed Oracle 10g capabilities NetLobs – NetCDF meets database. Introduction to Lost Wax. Server-side Systems J2EE / MS.NET SOA and Web Services - PowerPoint PPT Presentation
Citation preview
Netlobs Manipulating Gridded Data in a
Relational World
Neil Stamps
Technical Architect
AgendaAgenda
Introduction to Lost Wax
The problem framed
Oracle 10g capabilities
NetLobs – NetCDF meets database
Introduction to Lost Wax
AdvancedSoftware
Engineering
Multi-agent SystemsContinual R&DRoles based business analysisAgent frameworkProducts (SPL)Innovation projects
Server-side SystemsJ2EE / MS.NETSOA and Web ServicesLegacy integration / wrappingGIS Mapping Solutions
MobileDistributed computingJava PDAs / phones
The problem framedThe problem framed
DTI DEWS research project partners:
Met Office, Reading University, BADC, BMT, IBM
Provides web services to multiple domains
GADS provides marine services
Oracle target platform
InternetWeb Service
InternetWeb Service
Forecast Data
SQL
GADS
Oracle – Blob supportOracle – Blob support
Oracle 10g deployment platform
Large object support Blob – max size (4GB –1)*block
e.g. 32k block = 128TB maxClob, nClob – max size as per Blob
Extension supportJava, C extensionsJava stored proceduresCustom data types (cartridges)
Remote symbolic debugging (JDeveloper)
Oracle - Custom data typesOracle - Custom data types
Provide encapsulation of attributes and methods
Introduce OO capabilities into relational world
Allow unstructured data to be queried
Extensions to indexes allow efficient queries
Nested tables provide collection capability
NetLobs – NetCDF ‘SmartLobs’NetLobs – NetCDF ‘SmartLobs’
Provides NetCDF file capability to Oracle
Encapsulates data and meta-data in single type
Physical implementation agnostic
Automatic extraction and storage of meta-data
Interrogate meta-data without blob enquiry
Extraction over single or multiple Netlobs
NetLobs – NetCDF ‘SmartLobs’NetLobs – NetCDF ‘SmartLobs’
NetCDF 2.2 open source Java libraries
NetLob wraps Oracle for NetCDF Files
Extraction interfaces based upon current GADS requirements:
SubsetReductionConcatenation
Higher level interfaces to be layered over basic functionality
Cartridge invocationCartridge invocation
PL/SQL interface maps to Java call (or C)
Oracle instantiates NetLob object
Object implements SQLData interface
Blob pointer passed to Java, Random access provided via ‘internal’ JDBC
NetLobs – Data ingestionNetLobs – Data ingestion
Upload using Oracle SQL*Loader
Upload in two–phase method
Validation at Netlob creation
System optimisation based upon once-only performance hit at extraction of meta-data
Meta-data ‘chunks’ will facilitate query by value
NetLobs – PerformanceNetLobs – Performance
Reference GADS system provides predictable, linear extraction performance
NetLob cartridge aims to achieve similar performance characteristic over large data extractions
Optimisation tailored to larger extractions
Moving forwardMoving forward
Storage and retrieval of rotated data
Pluggable interpolation framework
Offloading processing to GRID
Enhanced meta-data to meet community needs
Query by-value enhancements
Any Questions?Any Questions?