Scientific and Multidimensional Raster Support in ArcGIS•Visualizing and Analyzing •Disseminating and Consuming •Demo: Application(s) and Use case Scientific ... stores processing

Copyright © Esri. All rights reserved.

Sudhir Raj Shrestha

[email protected]

Scientific and Multidimensional Raster Support in ArcGIS


Brief breakdown

• Scientific Multidimensional data

• Scientific MultiDim Data in ArcGIS

• Ingesting and Managing

• Visualizing and Analyzing

• Disseminating and Consuming

•Demo: Application(s) and Use case

Scientific Computing

Ingesting

Managing

AnalyzingVisualizing

Sharing


Scientific Multidimensional Data


. . . Successfully Integrated into Thousands of Systems

Open Standards

and Formats

Product

Integration

Open Software

AutoCAD

SAP HANAIBM Cognos

Adobe Creative Cloud

AWS

AzureMS Office

Oracle

SQL Server

TeradataNetezza

REST

OPeNDAP

WMSWCS

CSW

GMLWFS

KML

WMTS

WPS

SLD

GeoPackage

SOAP

IFC

LAS

WWW

ISONetCDF

SQL

JSON

WaterML

CityGML

OneGeology

LERC

Shapefiles

Web Scene (I3S)

INSPIRE

Extensible Architecture

Open-Source Contributions (500+)

Embeddable Components

Open APIs & SDKs

Open Data Access

Open-Source Integration

ArcGISAn Open, Interoperable, and Standards-Compliant Platform

SharePoint

Any SystemMany OGC

Certifications

GRIB

HDF


Diverse Scientific Multidimensional Data

Oceanographic

• Salinity

• Sea surface temperature

• Ocean current

Meteorological

• Temperature

• Water Vapor/Precipitation

• Wind speed/direction

Terrestrial

• Soil moisture

• NDVI

• Land cover


mosaic dataset

geoprocessing tools

optimization

consumption

multidimensional filter

raster functions

Interactive/dynamic

scalable/extensible

dynamic web services

web maps & apps

web based analytics

Analyze ShareManage VisualizeIngest

raster types

crawl disk

link to pixels

animate over slices

sophisticated renderers

vector & scalar fields

GIS workflows that scaleDesigned for the characteristics of Earth Observations scientific data

Data Service


Ingesting and Managing Scientific Data


Multidimensional Mosaic Dataset

Representing multivariate collection of multidimensional rasters in ArcGIS

spatially-indexed catalog

multi-resolution, multivariate, multidimensional

reduce storage redundancy & pixel resampling

defines information products

on-demand processing

mosaic

table

raster pixels

mosaic dataset

HDF

GRIB

netCDF

d-aware rasters


Multidimensional Data ModelMultidimensional Mosaic Dataset in Geodatabase

• Ingest variables from netCDF, HDF & GRIB using raster types

- Aggregate multiple variables, multiple files

• Support on-the-fly processing


Raster Types for Multidimensional data

Format-agnostic direct ingestion of rasters into a mosaic dataset

crawls disk

identifies rasters

extracts metadata

attaches processing

raster typedata on disk mosaic dataset

netCDF

HDF

GRIB

d-aware rasters

stores no pixels

references rasters

stores processing & metadata


Creating a Multidimensional Mosaic Dataset

Using Geoprocessing Tools

• Create a empty mosaic dataset

• Add select variables


Demo

Create multidimensional Mosaic Dataset

Populate Mosaic dataset using NetCDF Raster type

Ingest Multidim data using Make NetCDF raster layer GP tool

Ingest data using Multidimensional raster layer


Visualizing and Analyzing Scientific Data


Slicing your data

•Slicing

• By variable, using variable selector

• Processing Templates

• By dimension using Select by Dimension Geoprocessing tool

•Visualizing

• Time slider

• Range slider

• Vector field renderer


Demo

Visualize multidimensional data using time slider and range slider

Slicing your Mosaic dataset with Select by Dimension GP Tool

Parse data using Variable filter and Processing templates

Visualize Wind data with vector field renderer


Raster Analysis

• Geoprocessing Tools (GP)

• Tons of GP tools

• GP tools, Python scripting and ModelBuilder perfect automation of data

management and analysis.

• Dynamic On-the-Fly Processing using Raster Functions

• Manage and analyze large collection of rasters on the fly as the data is accessed

and viewed

• Quick and save time by not required to write the processed product to disk

• Functions can be applied to various rasters (images) including:

• Raster dataset layers

• Mosaic datasets

• Image service layers

http://pro.arcgis.com/en/pro-app/tool-reference/data-management/an-overview-of-the-raster-toolset.htm


Choose from dozens of built-in functions or implement your own algorithm using Python

Learn more at: github.com/Esri/raster-functions

raster-valued transformation

piecewise, scale-aware

local / focal

on-demand processing

unary, binary, …, n-ary, polyadic

raster function

Raster Function: Transforming Raster Data


Modeling and Chaining Raster Functions

… to compose a complex analytic model

Elevation

Slope

Conservation area

Soil Type

Soil Acidity


Raster Function Templates

A portable & reusable chain of raster functions

Variable 1

Variable 2

raster variables


Demo

Transforming your scientific data using Raster Functions


Disseminating and Consuming


Access / Identity

Apps

Services

Desktop Web Device

Online

Content and

Services

Server

multivariate multidimensional

mosaic dataset

Disseminating

professional

geospatial analysts


Serving Multidimensional DataMultidimensional Image Services

Serve data (not a picture)• LERC compression to maintain data quality

• Perform analysis and analysis through the web

Multiple clients:• Desktop clients

• Web Map Viewers

• JavaScript etc.

• WMS/WCS

• Time=t1&Elevation=z1

• DIM_Time=t1&DIM_depth


Sharing data & information products

· Mosaic Dataset ≻ Share As Web Layer

Enable access to a dynamic representation of your information product as an image service


Consuming your services

• In any ArcGIS application or any WMS client

• In a web map

• Identify web services driven by maps or datasets

•Bring service layers into a web map

• In a map-based application

•Configurable apps

• Story Maps

•Web AppBuilder

•Custom web apps using ArcGIS API for JavaScript


Create

Mosaic Dataset

Publish

Mosaic Dataset

Ingest:

Metadata

References to Source

Approx 30 records/Sec

Server

Accessible

HRRR Dataset

(GRIB)

ArcGIS

Image ServerDownloadHRRR

Dataset

(GRIB)

Model

Generation

Time t

Time for HRRR 2D Surface Levels data: 2.2GB 2376 rec/MD

Standard Workflow 80sec 123Sec 30sec Total= 233 Sec

Pre-Generate Workflow 80sec 0Sec 30sec Total= 110sec

Download Build Mosaic Restart Service

Near real time High Resolution Rapid Refresh (HRRR) Data

Pre-Generate Workflow 0sec 0Sec 30sec Total= 30sec

http://imagerydemo.s3.amazonaws.com/hrrrexplorer/index.html


Demo

Publish the scientific data (Image, WCS, WMS capability)

Dynamic scientific computing in web

Using image service layer in ArcGIS Online map viewer

Web Map Application: HRRR, Hycom


3 Take Away

1. Mosaic Dataset is a robust data model that allows you to manage your large

collections of scientific multidimensional data and is a quick way to build the live

web service

2. Raster function(s) can help with your efficient on the fly computing that saves

Time and Resources

3. Make your scientific data and research output usable with repeatable workflow to

your larger community


Join our session on Analysis of Earth Science Data

Analyzing Multidimensional Scientific Data in ArcGIS

(Room 144 B)

When: March 21, 2018 11:00-12:00 PM.


Select the session

you attendedScroll down to find

the feedback sectionComplete Answers

and Select “Submit”

Download the Esri Events

app and find your event

Please Take Our Survey in the Esri Events App

Print Your Certificate of AttendancePrint stations located in the 140 Concourse

Tuesday Wednesday12:30 pm – 6:30 pm GIS Solutions Expo Hall B

5:00 pm – 6:30 pm GIS Solutions Expo SocialHall B

10:30 am – 5:15 pm GIS Solutions Expo Hall B

6:30 pm – 9:00 pm Networking ReceptionSmithsonian National Portrait Gallery


Documents

Scientific and Multidimensional Raster Support in ArcGIS•Visualizing and Analyzing •Disseminating and Consuming •Demo: Application(s) and Use case Scientific ... stores processing