Extending GDAL/OGR and UMN MapServer - KNMIadaguc.knmi.nl/contents/documents/workshop2008/presentations/ADAGUC_WS2_20081205...• ESRI Shapefiles, PostGIS, Oracle Spatial • Used

Maarten Plieger

Nils de Reus

Maarten Plieger

Nils de Reus

http://adaguc.knmi.nl/

This project is sponsored bySpace for Geo-Information

www.rgi.nl

Extending GDAL/OGR andExtending GDAL/OGR and

UMN UMN MapServerMapServer

Providing the building blocks for a Providing the building blocks for a

geospatial infrastructuregeospatial infrastructure

http://adaguc.knmi.nl

[email protected]

ADAGUC Workshop 2008

IntroductionIntroduction

• Overview on the datasets

• UMN MapServer and GDAL / OGR

• Extending the GDAL driver

• Extending the OGR driver

• Patching UMN MapServer

• ADAGUC Web Portal

• Overview on the datasets

• UMN MapServer and GDAL / OGR

• Extending the GDAL driver

• Extending the OGR driver

• Patching UMN MapServer

• ADAGUC Web Portal

This project is sponsored bySpace for Geo-Information

www.rgi.nlhttp://adaguc.knmi.nl/


[email protected]


DatasetsDatasets

GOME OGOME O3 3 in ArcGIS Explorer in ArcGIS Explorer -- using WMSusing WMS

• Datasets

• Soil Moisture

• FRESCO

• NO2, CH4, CO, O3

• Datasets

• Soil Moisture

• FRESCO

• NO2, CH4, CO, O3

• Goal: Provide data through OGC Services

• Instruments

• AMSR

• GOME

• Sciamachy

• Instruments

• AMSR

• GOME

• Sciamachy


[email protected]


Datasets -> GISDatasets -> GIS

DATA PRODUCTS SERVICES

BinaryBinaryBinary

BinaryBinary

BinaryBinaryOther

WMS

WCS

WFS

ADAGUCContainerFormat

SCIAMACHYSCIAMACHY

radar

GOMEGOME

MSGMSG

GDAL/OGRADAGUCContainerFormat

ADAGUCContainerFormat

ASCII

ArcGIS

WWW

uDIG

gvSIG

GoogleEarth

…

& many others

IExplorer

AMSR-E


[email protected]


UMN MapServer and GDAL/OGRUMN MapServer and GDAL/OGR

• UMN MapServer

• Developed by the University of Minnesota (UMN)

• Supports WMS, WFS, WCS, WMC, Filter Encoding, SLD, GML, SOS

• Data can be accessed using GDAL/OGR

• Fast renderer

• Geospatial Data Abstraction Library (GDAL)

• Raster datasets; over 70 formats supported

• Used by many software products:• ESRI ArcGIS, GRASS, OpenEV, Quantum GIS

• Used in UMN Mapserver for raster access

• OGR Simple Features Library

• Vector datasets; over 20 formats supported

• ESRI Shapefiles, PostGIS, Oracle Spatial

• Used in UMN Mapserver for vector access

• Part of the GDAL library

• UMN MapServer

• Developed by the University of Minnesota (UMN)

• Supports WMS, WFS, WCS, WMC, Filter Encoding, SLD, GML, SOS

• Data can be accessed using GDAL/OGR

• Fast renderer

• Geospatial Data Abstraction Library (GDAL)

• Raster datasets; over 70 formats supported

• Used by many software products:• ESRI ArcGIS, GRASS, OpenEV, Quantum GIS

• Used in UMN Mapserver for raster access

• OGR Simple Features Library

• Vector datasets; over 20 formats supported

• ESRI Shapefiles, PostGIS, Oracle Spatial

• Used in UMN Mapserver for vector access

• Part of the GDAL library


[email protected]


Services with UMN MapServerServices with UMN MapServer

UMN MapServer

ADAGUC Raster Files ADAGUC Vector Files

WMSWMS WFSWFS WCSWCS

GDALGDAL OGROGR

PNGPNG

GIFGIF

JPEGJPEG

GML2GML2

GML3GML3

ADAGUCADAGUC

AAIGridAAIGrid

GeoTiffGeoTiff

Etc..Etc..

ImagesImages VectorVector RasterRaster

extendedextended


[email protected]


GDAL ADAGUC Raster driverGDAL ADAGUC Raster driver

• Based on existing NetCDF3 driver from Denis Nadeau and Frank Warmerdam

• Convert between ADAGUC files and formats supported by GDAL

• Uses NetCDF4/HD5 and UDUNITS libraries

• Climate and Forecast 1.2 conventions

• Features the ADAGUC metadata model

• Metadata is copied and updated where needed

• Additional metadata can be supplied using an NCML file

• For backwards compatibility: NetCDF3 output

• The driver is available from the OSGEO wiki on http://trac.osgeo.org/gdal/wiki/ADAGUC

• Based on existing NetCDF3 driver from Denis Nadeau and Frank Warmerdam

• Convert between ADAGUC files and formats supported by GDAL

• Uses NetCDF4/HD5 and UDUNITS libraries

• Climate and Forecast 1.2 conventions

• Features the ADAGUC metadata model

• Metadata is copied and updated where needed

• Additional metadata can be supplied using an NCML file

• For backwards compatibility: NetCDF3 output

• The driver is available from the OSGEO wiki on http://trac.osgeo.org/gdal/wiki/ADAGUC


[email protected]


• gdalinfo file.nc

• gdalinfo ADAGUC:file.nc:TMTNO2

• gdalinfo ADAGUC:file.nc:TMTNO2:time=10

• gdal_translate –of AAIGRID ADAGUC:file.nc:TMTNO2:time=10 out.asc

• gdalinfo file.nc

• gdalinfo ADAGUC:file.nc:TMTNO2

• gdalinfo ADAGUC:file.nc:TMTNO2:time=10

• gdal_translate –of AAIGRID ADAGUC:file.nc:TMTNO2:time=10 out.asc

GDAL: Variables and timeGDAL: Variables and time


[email protected]


OGR ADAGUC Vector driverOGR ADAGUC Vector driver

• Reads the ADAGUC vector format

• Convert ADAGUC to ESRI shapefiles:• ogr2ogr -f "ESRI Shapefile" out.shp adagucinputfile.nc

• ADAGUC files can not be created from OGR formats:• The ADAGUC format expects four vertices for each tile

• Most vector files contain polygons with lots of vertices

• The driver is available from the OSGEO wiki on

http://trac.osgeo.org/gdal/wiki/ADAGUC

• Reads the ADAGUC vector format

• Convert ADAGUC to ESRI shapefiles:• ogr2ogr -f "ESRI Shapefile" out.shp adagucinputfile.nc

• ADAGUC files can not be created from OGR formats:• The ADAGUC format expects four vertices for each tile

• Most vector files contain polygons with lots of vertices

• The driver is available from the OSGEO wiki on

http://trac.osgeo.org/gdal/wiki/ADAGUC


[email protected]


UMN MapServerUMN MapServer

• Supports WMS and WCS time• Although not supported by most GIS clients…

• Very active and helpfull mailing list• Questions and problems are usually answered within the hour

• High performance• One of the fastest renderers

• A few minor issues solved• Filenames with a ‘:’ in the filename were incorrectly interpreted when using WCS

• Metadata needed by GDAL was not passed by UMN MapServer

• Patches will be submitted on the UMN MapServer site

• Current issues• Currently no information can be queried about a certain locationwhen using WMS Time• This will be solved by the developers soon

• Supports WMS and WCS time• Although not supported by most GIS clients…

• Very active and helpfull mailing list• Questions and problems are usually answered within the hour

• High performance• One of the fastest renderers

• A few minor issues solved• Filenames with a ‘:’ in the filename were incorrectly interpreted when using WCS

• Metadata needed by GDAL was not passed by UMN MapServer

• Patches will be submitted on the UMN MapServer site

• Current issues• Currently no information can be queried about a certain locationwhen using WMS Time• This will be solved by the developers soon


[email protected]


ADAGUC Web PortalADAGUC Web Portal

• Features:

• Combining services from different institutes

• KNMI and VU data are accessible in the same portal

• Overlay layers from different services

• WMS Time support

• For compatibility with other clients used as little as possible

• GetFeatureInfo

• Support for Legends

• Map Resizing

• WCS or WFS access where applicable

• Direct access to the original ADAGUC files

• Used techniques:

• HTML, CSS, XML, PHP, JSON, AJAX and JavaScript

• Uses JavaScript libraries

• ExtJS

• WebmapJS• Custom made

• Features:

• Combining services from different institutes

• KNMI and VU data are accessible in the same portal

• Overlay layers from different services

• WMS Time support

• For compatibility with other clients used as little as possible

• GetFeatureInfo

• Support for Legends

• Map Resizing

• WCS or WFS access where applicable

• Direct access to the original ADAGUC files

• Used techniques:

• HTML, CSS, XML, PHP, JSON, AJAX and JavaScript

• Uses JavaScript libraries

• ExtJS

• WebmapJS• Custom made


[email protected]


ADAGUC Web PortalADAGUC Web Portal

WMS GetCapabilities

WMS GetMap

WMS GetFeatureInfo

WMS GetLegendGraphic


[email protected]


Download data from the PortalDownload data from the Portal

• Download the original file,

or convert to your favorite GIS format using OpenGIS

• Download the original file

• Raster files are downloaded using WCS

• Vector files are downloaded using WFS

• Depending on the datatype, which service is needed?• The right service is automatically selected

• Download the original file,

or convert to your favorite GIS format using OpenGIS

• Download the original file

• Raster files are downloaded using WCS

• Vector files are downloaded using WFS

• Depending on the datatype, which service is needed?• The right service is automatically selected

WCS DescribeCoverage

WFS MetadataURL

WFS or WCS?


[email protected]


WFS ServiceWFS Service

• Select Bounding Box

• Viewing window or Custom bounding box?

• Only GML available as output format

• Researching the possibility to export directly into ESRI shapefiles

• Select Bounding Box

• Viewing window or Custom bounding box?

• Only GML available as output format

• Researching the possibility to export directly into ESRI shapefiles


[email protected]


WCS ServiceWCS Service• Optimal parameters are automatically set

• Select a Bounding Box• native bounding box• view window• custom?

• Resolution• Use width and height • or resx and resy?

• Support for WCS time

• Choose your favorite GISformat

• Optimal parameters are automatically set

• Select a Bounding Box• native bounding box• view window• custom?

• Resolution• Use width and height • or resx and resy?

• Support for WCS time

• Choose your favorite GISformat

Parameters derived from DescribeCoverage


[email protected]


Open IssuesOpen Issues

• GDAL ADAGUC driver

• Supports currently only the dimensions lat, lon and time

• Support for other dimensions like ‘height’ can be added

• OGR ADAGUC driver

• Reads the whole source NetCDF file in to memory

• Should only read the parts that are needed…

• UMN MapServer

• WMS GetFeatureInfo does not yet return data when querying on time layers

• Support for x and y (2D) data is easy

• Support for x, y, time and height is tricky

• Not many GIS clients out there that support WMS time

• GDAL ADAGUC driver

• Supports currently only the dimensions lat, lon and time

• Support for other dimensions like ‘height’ can be added

• OGR ADAGUC driver

• Reads the whole source NetCDF file in to memory

• Should only read the parts that are needed…

• UMN MapServer

• WMS GetFeatureInfo does not yet return data when querying on time layers

• Support for x and y (2D) data is easy

• Support for x, y, time and height is tricky

• Not many GIS clients out there that support WMS time


[email protected]


• ADAGUC NetCDF4 used with GDAL/OGR and UMN MapServer

• Web Portal communicates with the server using purely OpenGIS services

• Third parties can setup their own services using the building blocks created within the ADAGUC project

• The demonstrated atmospheric datasets are now very well accessible to GIS users!

• ADAGUC NetCDF4 used with GDAL/OGR and UMN MapServer

• Web Portal communicates with the server using purely OpenGIS services

• Third parties can setup their own services using the building blocks created within the ADAGUC project

• The demonstrated atmospheric datasets are now very well accessible to GIS users!

SummarySummary


[email protected]


Questions?Questions?

Documents

Extending GDAL/OGR and UMN MapServer - KNMIadaguc.knmi.nl/contents/documents/workshop2008/presentations/ADAGUC_WS2_20081205...• ESRI Shapefiles, PostGIS, Oracle Spatial • Used