Embed Size (px)
Citation preview
PublicaMundiScalable and Reusable Open Geospatial
Data
, IMIS AthenaAngelos TzotsosScientific & Technical Manager
OSGeo Charter MemberOGC Member
GeoDataCamp 2014, Athens, 30 May 2014
Genesis of the projectConsortium established on active research, commercial, andpersonal interactionsConsortium members are active contributors to Free and OpenSource Geospatial SoftwareIdea based on practical experience (2010-) from geodata.gov.gr andour interaction with users,publishers and SMEs
Why PublicaMundi?Imago Mundi
Image of the World, Babylonian world mapMaps, created by cartographers and geographers
PublicaMundiImage of the World, based on Open DataMaps, created by open knowledge and FOSS
Consortium
Athena IMIS
Rasdaman
Geolabs
GET
AthenaAthena Research and Innovation Center in Information,Communication and Knowledge TechnologiesNon-profit, research organization, governed by public lawInstitute for the Management of Information Systems (IMIS)
Rasdamanrasdaman GmbHR&D centric SME, established in 2003 (MBO in 2010)Areas of business: commercial support for rasdaman Array DBMS;consultancy on SDIs & standardsGeospatial World Innovation Award (2013)
GeoLabsGeoLabs SARLR&D centric SME focused on FLOSS GISSenegalease Land register (2007)3D module development for Terra Explorer (2008) in IGN 3DGeoportalDevelopment of the MapMint SDI using WPS, other OGC WebServices and OASIS
GETGeospatial Enabling Technologies LtDSME focused on GeoInformaticsSuccessful design and implementation of projects for public andprivate sector related to geospatial data production, management,curation, geospatial applications (desktop, mobile, web)One of the first Greek private companies invested in Open SourceGIS technology
Experiences and ProblemsReoccurring practical problems in open geospatial data reuse:
Data formats/CRSWeb mapsInterlinkTranslationsPublishingOGC/INSPIRE documents not for all
GoalsResearch and develop methodologies, as well as scalable, reusabletools to facilitate:
the publicationdiscoveryand reuse
of open geospatial data
GoalsOpen data catalogues fully supporting publishing, curation andmanagement lifecycle of geospatial dataInterlinking of geospatial data and multilinguality support in across-boundary contextScalable technologies and services to create and reuse on-demandmaps from open geospatial dataAnalytics to accurately monitor the usage of open geospatial dataScalable technologies and reusable data APIs supporting querying,processing, and analysis of open geospatial data
Free and Open Source Software(FOSS)
It's a prerequisite for Big Data and Cloud ComputingEverybody uses FOSS today, even without knowing it
CKANAn abbreviation for “Comprehensive Knowledge Archive Network”Open Source web platform for publishing and sharing data withimpressive deployment history:
EU Open Data PortalUSA data.govUK data.gov.ukAustralia data.gov.auand many many others...
PublicaMundi and FOSSBased on CKAN open data cataloguePublicaMundi will spatially extend CKAN using OGC standardsDevelopment is based exclusively on the OSGeo stack
Open Source GeospatialFoundation (OSGeo)
Since 2006 A Non Profit Umbrella for:GeoSpatial Free and Open Source SoftwareEducationOpen Data
OSGeoLive
GNU/Linux distribution60+ Open Source Geospatial ApplicationsSample DatasetsConsistent Overviews & QuickstartsTranslationsUnfortunately Big Data do not fit on a 4GB DVD...
OGC standards and INSPIREDiscovery ServicesView ServicesDownload ServicesProcessing Services
Earth Observation Big DataIntegration with rasdamanWCPS and WPS support
CKANCKAN is a powerful data management system that makes dataaccessible – by providing tools to streamline publishing, sharing,finding and using data.CKAN is aimed at data publishers (national and regionalgovernments, companies and organizations) wanting to make theirdata open and available.
CKAN FeaturesPublish and find datasetsStore and manage dataFederated nodesHarvestingMetadata Editing/ManagementAPIs and Extensions
ckanext-spatial - Geo pluginsfor CKAN
A spatial field on the default CKAN dataset schema, that usesPostGIS as the backend and allows to perform spatial queries and todisplay the dataset extent on the frontendHarvesters to import geospatial metadata into CKAN from othersources in ISO 19139 format and othersCommands to support the CSW standard using pycswPlugins to preview spatial formats such as GeoJSON
ckanext-spatial FeaturesSpatial SearchSpatial HarvestersCSW interfaceWMS Preview
pycsw is a OGC CSW server implementation written in Python is an Open Source project released under the MIT license
pycswpycsw
What is Metadata?Metadata is often described as “data about data”, or the who, what,where, and when.In the geospatial world, for each dataset we maintain, we shouldrecord information about the data such as:
general descriptionlocationusage restrictionsprojectiontechnical contacttime perioddate createddate modifiedversion
Metadata StandardsDublin Core: established a core/common group of 15 metadataelementsFGDC CSDGM: approved by the U.S. Federal Geographic DataCommittee originally in 1994 and composed of Sections, CompoundElements, Data ElementsISO 19115: International Standards Organization’s TC211 committeecreated this in 2003 and is composed of more than 400 “Core”,“Mandatory”, and “Optional” elementsISO 19139: The XML implementation schema for ISO 19115specifying the metadata record format
OGC CSW SpecificationThe Open Geospatial Consortium (OGC)
, currently at version 2.0.2, is a standardfor discovering and retrieving spatial data and metadata.
OpenGIS Catalogue ServiceImplementation Specification
Catalogue Services for the Web (CSW) is the HTTP protocol binding ofthe Catalogue Service Implementation Specificaton that allows forpublishing and searching of metadata.
CSW OperationsGetCapabilities (mandatory) - allow clients to retrieve informationdescribing the service instanceDescribeRecord (mandatory) - allows a client to discover elements ofthe information model supported by the target catalogue serviceGetRecords (mandatory) - get metadata recordsGetRecordById (optional) - get metadata records by IDGetDomain (optional) - obtain runtime information about the rangeof values of a metadata record element or request parameterHarvest (optional) - references the data to be inserted or updated inthe catalogTransaction (optional) - defines an interface for creating, modifyingand deleting catalogue records
Example RequestsGetCapabilitiesDescribeRecordGetRecordsGetRecordByIdGetDomain
pycswpycsw fully implements the OpenGIS Catalogue ServiceImplementation Specification [Catalogue Service for the Web]pycsw allows for the publishing and discovery of geospatialmetadata
pycswThe project is certified , and is an OGC Reference
ImplementationOGC Compliant
pycsw is currently under OSGeo Incubation
FeaturesHarvesting support for WMS, WFS, WCS, WPS, WAF, CSW, SOSImplements ISO Metadata Application Profile 1.0.0Implements FGDC CSDGM Application Profile for CSW 2.0Implements INSPIRE Discovery Services 3.0Supports ISO, Dublin Core, DIF, FGDC and Atom metadata modelsStandalone of embedded deployment (CGI or WSGI)Transactional capabilities (CSW-T)Flexible repository configuration (SQLite, PostgreSQL, PostGIS,MySQL)Federated catalogue distributed searching
More features...Simple configurationExtensible plugin architecture (profiles, repositories/backends)Seamless integration with Python environments (e.g. GeoNode,Open Data Catalog)Includes commandline utility to administer the metadata repositoryImplements the Search/Retrieval via URL (SRU) search protocolImplements OpenSearchRealtime XML Schema validation
Standards SupportOGC CSW 2.0.2OGC Filter 1.1.0OGC OWS Common 1.0.0OGC OpenSearch Geo/TimeOGC GML 3.1.1OGC SFSQL 1.2.1Dublin Core 1.1SOAP 1.2ISO 19115 2003ISO 19139 2007ISO 19119 2005NASA DIF 9.7FGDC CSDGM 1998SRU 1.1A9 OpenSearch 1.1
ZOO Project WPSZOO is a WPS (Web Processing Service) open source project releasedunder a MIT/X-11 style licenseIt provides an OGC WPS compliant developer-friendly framework tocreate and chain WPS Web services
ZOO OverviewZOO is made of three parts:
ZOO Kernel: A powerful server-side C Kernel which makes it possibleto manage and chain Web services coded in different programminglanguagesZOO Services: A growing suite of example Web services based onvarious Open Source librariesZOO API: A server-side JavaScript API able to call and chain the ZOOServices, which makes the development and chaining processeseasier
RasdamanRasdaman ("raster data manager") is a domain-neutral ArrayDatabase System: it extends standard relational database systemswith the ability to store and retrieve multi-dimensional raster data (arrays) of unlimited size through an SQL-style query language.It provides reference implementation of OGC WCS and WCPSinterfacesRasdaman embeds itself smoothly into PostgreSQLThe Petascope component of rasdaman provides service interfacesbased on the OGC WCS, WCPS, WCS-T, and WPS
Rasdaman featuresRasdaman makes it easy to search in large, multi-dimensional rasterdataRASQL languageTiling policiesParallel server processingOGC interfaces
RASQLThe rasdaman query language, rasql, offers raster processingformulated through expressions over raster operations in the style ofSQL.Consider the following query: "The difference of red and greenchannel from all images from collection LandsatImages wheresomewhere in the red channel intensity exceeds 127"
select ls.red - ls.greenfrom LandsatImages as lswhere max_cells( ls.red ) > 127
Other Geospatial TechnologiesInvolved
PostGIS - Spatial Database
OpenLayers – Browser MappingLibrary
Leaflet – Mobile FriendlyInteractive Maps
GeoServer – Web Services
MapServer – Web Services
MapProxy – Proxy WMS & tileservices
GDAL/OGR – Geospatial DataTranslation Tools
MetaCRS - CoordinateReference SystemTransformations
Thank you for your attention!
Questions?
http://publicamundi.eu
