1EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

EarthServer Ahead!

Open Source Park, INTERGEO 2013

Peter Baumann

Jacobs University | rasdaman GmbH

Supported by EU FP7 eInfrastructure, contract 283610 EarthServer

2EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

A Bit of OGC: Features & Coverages

The basis of all: geographic feature

• = abstraction of a real world phenomenon [OGC, ISO]

• associated with a location relative to Earth

Special kind of feature: coverage•= space-time varying multi-dimensional phenomenon

•Classic: 2-D raster image

•...but there is more!

Often Big Geo Data are coverages

3EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

MultiSolid

Coverage

n-D "space/time-varying phenomenon"

• [ISO 19123, OGC AT6]

OGC Coverage

«FeatureType»AbstractCoverage

[OGC 09-146r1]

MultiPoint

Coverage

MultiCurve

Coverage

MultiSurface

Coverage

Grid

Coverage

Referenceable

GridCoverage

Rectified

GridCoverage

4EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

Web Coverage Service (WCS)

Core: Simple & efficient access to n-D spatio-temporal coverages

• plus format encoding

subset = trim | slice

Extensions add functionality facets

• Scaling, CRS transformation, ...up to ad-hoc analytics

4

5EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

WCS Core & Extensions

for illustration only – not normative!

Legend:

- available

- under work

- planned

WCS CoreGML Application Schema -Coverages

SWE Common OWS CommonGML

ServiceData Model UsabilityProtocol Binding

Range Subsetting

Processing

WCS-TQuality GET-KVP

POST-XML

SOAP

Multilinguality

Interpolation

Scaling

CRS

EO-WCS MetOc-WCS

REST

JSON

Abstract Topic 6

Hierarchies

Format Encoding

GeoTIFF

netCDF

JPEG2000

etc.

GRIB2

GMLJP2

data service

6EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

Scalable On-Demand Processing for the Earth Sciences

• EU FP7-INFRA, 3 years, 5.85 mEUR

100+ TB databases for Earth & Planetary sciences

• Platform: rasdaman; strictly open standards interfaces (OGC, W3C, X3D)

EarthServer: Big Earth Data Analytics

RTD partners:

7EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

raster data manager

= Array DBMS for massive n-D raster data

• SQL + imaging operators

Flexibility query language

Scalability „tile streaming“ architecture,

parallelization

In use by DWD, VITO/BE, ...

www.rasdaman.org

select img.green[x0:x1,y0:y1] > 130

from LandsatArchive as img

where avg_cells( img.nir ) < 17

8EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

EarthServer Anatomy

RTD Activity: establish & extend operational service software

• Lead: Jacobs Uni

Services Activity: establish 6 operational Lighthouse Applications

• Lead: EOX

Networking Activity: outreach, standardization

• Lead: CNR

9EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

WP320: Coverage Services

Jacobs Uni, rasdaman, CNR, BGS

WCS 2.0 service components (JacobsUni)

• Done: WCS Core, protocols (KVP, POX, SOAP), range subsetting, CRS, ...

plus manifold fixes & additions to rasdaman

• TBD: WCS-T 2.0, WCPS 2.0; REST

Extend WCPS to nonuniform grids (JacobsUni, RAS)

• Done: nonuniform grids in WCS

Extend WCPS to nongridded data (JacobsUni, RAS)

• Done: point clouds in WCS

Harmonize NetCDF and WCS/WCPS wrt. new coverage types (CNR-IIA)

10EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

WP330: QL Integration Data/Metadata

ATHENA, Jacobs Uni

WCPS 2.0 = WCPS + XQuery

• Most metadata in XML – XPath/XQuery natural choice

Advantages:

• Convenience: integrated data + metadata search

• Performance: less ping-pong between client and server(s)

Implementation:

• eXist + rasdaman

12EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

WP340: GIS Integration

EOX

MapServer accessing rasdaman as data source

GDAL accessing rasdaman as data source

Integration of existing archives

• Traditionally: data imported into database

• Problem: Large-scale data centers sometimes object to copying

• Approach: reference external files, use as tiles

13EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

WP350: Scalability

Jacobs Uni, rasdaman, COMETA

select

max((A.nir - A.red) / (A.nir + A.red))

- max((B.nir - B.red) / (B.nir + B.red))

from A, B

Array A

select

max((B.nir - B.red) / (B.nir + B.red))

from B

select

max((A.nir - A.red) / (A.nir +A.red))

from A

Array B

14EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

WP360: 3D Database Visualization

Fraunhofer IGD, Jacobs Uni, COMETA

• Mobile WCS client, plus...

select

encode(

struct {

red: (char) s.b7[x0:x1,x0:x1],

green: (char) s.b5[x0:x1,x0:x1],

blue: (char) s.b0[x0:x1,x0:x1],

alpha: (char) scale( d, 20 )

},

"png"

)

from SatImage as s, DEM as d

[JacobsU, Fraunhofer 2012]

[data courtesy BGS, ESA]

15EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

Ex: Climate Data Service, MEEO

[MEEO 2013]

16EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

Ex: Plymouth Marine Laboratory

[PML 2013]

17EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

Ex: British Geological Service

[BGS 2013]

18EarthServer :: INTERGEO Open-Source Park :: ©20123 P. Baumann

Conclusion

EarthServer establishing on-demand mix&match of Big Earth Data

• Transatlantic – only FP7 project with NASA on board

• Significant impact on standards

rasdaman: flexibility, scalability, information integration, open source

• GEOSS key component

rasdaman hits 2013+vote for rasdaman!