Overview of OSGeo + OGC + Neo4J Spatial + GeoServer

© 2004 – 2009 , Meruvian Foundation. All rights reserved. Proprietary and Confidential

Graph BigData – Neo4J SpatialGraph BigData – Neo4J Spatial

Frans ThamuraFrans [email protected]@meruvian.com

© 2004 – 2009 , Meruvian Foundation. All rights reserved. Proprietary and Confidential - Not to be Copied or Distributed.

• BlueOxygen Marketplace (www.blueoxygen.net)‏ - Founder

- Open Source Project on Java, since 2001.

• JUG Indonesia (http://www.jug.or.id) - Founder

• Meruvian Group (http://www.meruvian.org) - Founder

• jTechnopreneur (http://www.jtechnopreneur) - Founder

• JENI Advisory Board (http://www.jeni.or.id) -

• Oracle Ace Director

• Sun Java Champion

• CodeAndroid Indonesia Chapter (www.codeandroid.or.id)

• OpenStack Indonesia Chapter (www.openstack-id.org)

• OSGeo Indonesia (www.osgeo.or.id)

Who is Frans Thamura?Who is Frans Thamura?

http://www.blueoxygen.net/

http://www.jug.or.id/

http://www.meruvian.org/

http://www.jtechnopreneur/

http://www.jeni.or.id/

http://www.openstack-id.org/

http://www.osgeo.or.id/


MeruvianMeruvian

• Meruvian is Development and Services Institute based on Java and Enterprise OpenSource.

• Found as a gap solution between education and industry

• Meruvian start operates at March 28, 2006

• Founder: Frans Thamura

• Knowledge Management Oriented

Industry

Education

Meruvian


JENI 2.0JENI 2.0


Open Source Geospatial FoundationOpen Source Geospatial Foundation

http://www.osgeo.or.idhttp://www.osgeo.or.id


OSGeo's MissionOSGeo's Mission

Support and promotethe highest quality

Open Source Geospatial Software


OSGeo's StructureOSGeo's Structure

CommitteesCommittees

Committees

Board of 9 Directorsand President

LocalChaptersLocal

ChaptersLocalChapters

OfficersOfficers25 Officers

LocalChaptersLocal

ChaptersFoundationProjects

CommitteesCommittees

Committees

elected by Membership 147 Charter Members

Charter Members vote

represent

Sponsors


OpenStreetMap

UMN MapServer

MapFish

web mapping

GRASS GIS

Quantum GIS OSSIM

desktop GIS

metadata

GEOS

GDAL/OGR

GeoTools

libraries

MetaCRS

PostGIS

GeoMajas

Quantum GIS MapServer

deegree

OpenLayers GeoServer

MapGuide OS

deegree GIS

MapNik

Viking

KOSMO

gvSIG

SpatialLite

uDig

Geopublisher

Mapbender

GeoNetwork

GeoKettle

GMT

ZOO Project

FDO


The Open Geospatial Consortium (OGC)The Open Geospatial Consortium (OGC)

• The Open Geospatial Consortium (OGC) is an international industry consortium

• 481 members (as of March 2013), companies, government agencies and universities

• Operates under a consensus process to develop publicly available interface standards.

• OGC Standards support interoperable solutions that "geo-enable" the Web, wireless & location-based services and mainstream IT.

• The standards empower producers to make complex spatial information and services accessible

Slide by courtesyof Athina Trakas


GeoServerGeoServer

• GeoSpatial enterprise gateway – Java Enterprise

– Management and Dissemination of raster and vector data

• Standards compliant – OGC WCS 1.0, 1.1.1 (RI), 2.0 in the pipeline

– OGC WFS 1.0, 1.1 (RI), 2.0

– OGC WMS 1.1.1, 1.3

– OGC WPS 1.0.0

• Google Earth/Maps support – KML, GeoSearch, etc..


GeoServer: Format & ProtocolsGeoServer: Format & Protocols


GeoServer ArchitectureGeoServer Architecture


NoSQL NoSQL


Neo4j – Nodes, Relationships, PropertiesNeo4j – Nodes, Relationships, Properties

• Nodes have different properties– Matrix characters: People vs. Programs

• Build structure as you go– Who loves Neo?



Neo4j SpatialNeo4j Spatial

• Core– Storage

– Search

– Operations (GeoProcessing)

– I/O (Import / Export)

• Extensions– Geotools & GeoServer

– uDig

– OpenStreetMap

Page 17© 2004 – 2009 , Meruvian Foundation. All rights reserved. Proprietary and Confidential - Not to be Copied or Distributed. 17

diamond butterfly bullstar

franklin horton hall-jankorobertson

Graph PatternGraph Pattern

Page 18© 2004 – 2009 , Meruvian Foundation. All rights reserved. Proprietary and Confidential - Not to be Copied or Distributed.

Trends in BigData & NOSQLTrends in BigData & NOSQL

18

• Increase data size (bigdata)– “Today we create as much information as we did up to 2003” - Eric

Schmidt

• Increasingly connected data (graph data)– for example, text documents to html

• Semi-structured data – individualization of data, with common sub-set1. increasing data size

(big data)

– “Every 2 d

• Architecture - a facade over multiple services– from monolithic to modular, distributed applications


Graph EverywhereGraph Everywhere

• Relationships in – Politics, Economics, History, Science, Transportation

• Biology, Chemistry, Physics, Sociology– Body, Ecosphere, Reaction, Interactions

• Internet– Hardware, Software, Interaction

• Social Networks– Family, Friends

– Work, Communities

– Neighbours, Cities, Society


Good RelationshipsGood Relationships

• The world is rich, messy and related data

• Relationships are as least as important as the things they connect

• Graphs = Whole > Σ parts

• Complex interactions

• Always changing, change of structures as well

• Graph: Relationships are part of the data

• RDBMS: Relationships part of the fixed schema



A graph database...A graph database...

• NO: not for charts & diagrams, or vector artwork

• YES: for storing data that is structured as a graph– remember linked lists, trees?

– graphs are the general-purpose data structure

• “A relational database may tell you the average age of everyone in this session,

• but a graph database will tell you who is most likely to buy you a beer.”

Page 23© 2004 – 2009 , Meruvian Foundation. All rights reserved. Proprietary and Confidential - Not to be Copied or Distributed. 23

foo barfoo_bar

You know relationalYou know relational

now consider relationships...


We're talking about a Property GraphWe're talking about a Property Graph

24

Properties (each a key+value)+ Indexes (for easy look-ups)


Looks different, fine. Who cares?Looks different, fine. Who cares?

• a sample social graph

• with ~1,000 persons

• average 50 friends per person

• pathExists(a,b) limited to depth 4

• caches warmed up to eliminate disk I/O

# persons query time

Relational database

1,000 2000ms

Neo4j 1,000 2ms

Neo4j 1,000,000 2ms


Neo4j is a Graph DatabaseNeo4j is a Graph Database

• A Graph Database:

• a schema-free Property Graph

• perfect for complex, highly connected data

• A Graph Database:

• reliable with real ACID Transactions

• scalable: 32 Billion Nodes, 32 Billion Relationships, 64 Billion Properties

• fast with more than 1M traversals / second

• Server with REST API, or Embeddable on the JVM

• higher-performance with High-Availability (read scaling)

26


Create Graph with CypherCreate Graph with Cypher

CREATE (steve {name: “Steve Vinoski”}) -[:PRESENTED_WITH {date:{day}}]-> (michael {name: “Michael Hunger”})


BindingsBindings

28

REST://


Connecting and SplittingConnecting and Splitting

• Routing

• OSM Dynamic Layers


FutureFuture

• Editing– Fine Grained Geotools Feature Editing

– OSM Editor

• OSM– Caching sub-graphs (desktop & mobile)

– More Dynamic Layers & Shapefile export

• Topology– Persist all topological results in graph

• Benchmarking & Performance– Improved indexing

– Composite index


OpenStreetMapOpenStreetMap


OpenStreetMapOpenStreetMap


Snappr - OpenStreet /Neo4JSnappr - OpenStreet /Neo4J

https://github.com/peterneubauer/snappstr

© 2004 – 2009 , Meruvian Foundation. All rights reserved. Proprietary and Confidential

Thank YouThank You

Frans ThamuraFrans [email protected]@meruvian.com

https://www.slideshare.net/flatburger/overview-of-osgeo-ogc-neo4j-spatial-geoserver