Upload
others
View
23
Download
0
Embed Size (px)
Citation preview
Marten Hogeweg, Adam Martin
Creating, Using and Managing Metadata across the Enterprise
Adam Martin
Good Metadata Matters
Good Metadata Cultivates Trust (with)in the Organization
Data Creation
Map
Analysis
Derived App
Decisions
What were the sensor specs?
What’s the level of uncertainty?
When was this data last updated?
What models were used to process the data?
How would I recreate them?
Which layers are used in the map and who created them?
Whose model are we using?
How can I reproduce and validate this analysis for myself?
Outcomes
…and is important at each step as data become decisions.
Imagery Lidar
3DVector
Tabular
Real-Time
(IoT)
Big Data
Unstructured
Enabling Search, Discovery and Use
Hub Sites Exploratory
AppsDCAT
Catalog Feed
Effectively and freely sharing your derivative work with any machine or person
Open Cloud Big Data
Stores
Metadata
Hub Sites Exploratory
AppsCatalog
Feed
Public Web Services
Census – Small Business Builder
Imagery Comparison Viewer
Metadata
Big Data Stores
on an Open Cloud
HarvestersCrawlers
Bots
DCATCSW
JSON-LD
GeoServices REST APIWFS, WMTS
DHS – HIFLD Open
USDA - Forest
FAA - UAV
ArcGIS
Enabling Search, Discovery and Use
> 4,500 sites
> 264,000 public datasets
> 1,500 organizations
DCAT
/data.json
ArcGIS Hub
Catalog Interoperability: Publish Once!
How do users find your data?User Acquisition Sources
• 35% Search Google
• Click on a trustworthy looking source
• 60% of search-based users land on datasets!
• 26% Referred from a trusted source
• From an already trusted .gov website
Source: Hub Google Analytics, Jan-Dec 2019
Are people looking for “authoritative” data or sources?
The Tools Help You Create Metadata, You Are the One Creating Good Metadata
• Fill out all metadata fields on data portals
• Keep page titles no longer than 50-60 characters
• Optimize the content of abstracts
• Do not include lists of keywords [in the abstract]
• Check whether you can influence the URL
• Avoid special characters where these are not displayed correctly
• Keep the same URL if your data is updated
• Remove out-of-date pages
• Avoid duplication of metadata
• Use tools and tests to understand users
• Use tools to identify the best keywords for your subject
• Apply these recommendations to all your pages
https://www.gov.uk/government/news/new-guide-helps-find-and-classify-geospatial-datasets-across-search-engines
Marten Hogeweg
Creating Metadata in ArcGIS
Metadata is Everywhere
Portal
Author Serve Discover Use
Metadata Lifecycle in ArcGIS
• All items have metadata
• Metadata in ArcGIS is stored in ArcGIS XML
• Metadata styles determine how you display, edit, validate, export metadata
ArcMap ArcGIS Pro Portal for ArcGIS / ArcGIS Online
ArcGIS Platform
Read Write
Stored as ArcGIS XML
Standard formats
Standard formats
Full ArcGIS Metadata
Metadata in ArcGIS
Dataset Properties
& Syncing
Default for
Portal and
ServicesOwner (CitationContacts) |
Spatial Reference | Extent | Scale
Range | Coverage | Culture [en-
us] | GP History | Type [Map
Service] | TypeKeywords |
Distribution Information
System generated
defaults to support application use; can be manually edited or
synced in Desktop
Item Info /
Item Description
Title | Tags | Summary (Snippet) |
Description | Credits | Use
Limitations | Thumbnail (jpg)
Thumbnail (binary) |
Resource Maintenance |
Data Quality |
Data Lineage |
Resource Point of Contact |
Metadata Contact |
Temporal Extent |
Security Constraints…
Human readable,
scanned for minimum viable quality.
Support a higher
descriptive standard to support reuse, legal records and faster
discovery, internally and externally.
Usually defined by org and/or mandated by gov
Stored in ArcGIS Metadata Format v1.0
Flow of Metadata Publishing to ArcGIS Enterprise
• Create metadata for data in registered enterprise geodatabase
• Add data to an ArcGIS Pro map
- Layer reference’s data’s metadata
• Publish map to ArcGIS Enterprise as a feature layer
- Data with metadata is referenced
• Individual layers in the feature layer reference the data’s metadata
• Update metadata in the enterprise geodatabase, immediately available online
Flow of Metadata Publishing to ArcGIS Online
• Create metadata for data
• Add data to an ArcGIS Pro map
- Layer reference’s data’s metadata
• Publish map to ArcGIS Online as a hosted feature layer
- Data with metadata is copied
• Individual layers in the feature layer reference the copied data’s metadata
• Update metadata locally, overwrite item to make updates available online
Metadata Support in ArcGIS Desktop
ArcMap 10.x
PRO 2.0
PRO 2.1 PRO 2.2
PRO 2.3
PRO2.4
PRO 2.5
Portal/Online
Hub
Metadata styles for geospatial metadata standards
✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ -
Display and edit full metadata ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ -
Map layers have metadata - ✓
(limited)✓ ✓ ✓ ✓ ✓ ✓ -
Import and export metadata ✓ - - ✓ ✓ ✓ ✓ Yes1 No/Yes2
Manage metadata programmatically ✓✓
(limited)
✓
(improved)✓ ✓ ✓ ✓ ✓ ✓
Manage metadata with geoprocessing tools
✓ - - - - - ✓
(arcpy)- -
Publish map with full metadata ✓ - - ✓ ✓ ✓ ✓ ✓ N/A
Customize the metadata editor ✓ - - ✓ ✓ ✓ ✓ TBD 3 N/A
ISO 19115-1/-2/-3 - - - - - - ✓
(limited)TBD 4 TBD4
Metadata Support in ArcGIS Pro
• Metadata styles for geospatial metadata standards
• Display and edit full metadata
• Metadata for map layers
• Import and export metadata
• Save and filter content to files
• Copy ArcGIS metadata
• Upgrade CSDGM metadata
• Integration via Pro SDK
• arcpy.metadata!!
Metadata Support in ArcGIS Data Reviewer
• Schema Validation
- Item Metadata
- Stand-alone Metadata File
• Metadata Element Content Validation
- Pre-defined Expressions
- Custom XPath
Metadata Support in ArcGIS Server
• ArcGIS REST Services
• OGC Services
• INSPIRE Services (via ArcGIS for INSPIRE)
• Iteminfo & Metadata
Metadata Support in ArcGIS Online and ArcGIS Enterprise
• Metadata styles for geospatial metadata standards
• Display and edit full metadata
• Scores quality of content
• Metadata for item layers
• Export metadata
• Copy ArcGIS metadata
• Save content to file
Metadata Support in ArcGIS for Hub Sites
• Leverage existing metadata for public items
• Display and edit full metadata
• Scores quality of metadata content
• Search metadata
• Export metadata
• Leverage metadata for additionalaccess options
Desktop Web Device
Server Online Content and Services
ArcGIS Online
Respects Item Permissions –
Robust Access Model
WFS, WMS, KML, SHP, GeoServices
Hub Sites: From a single catalog, many portals into the data
Metadata Export to Third Party Catalogs
https://enterprisecontentnew-usfs.hub.arcgis.com/data.json
• All Online Hub Sites have /data.json
• Default DCAT format
• Configurable to support US GSA Project Open Data (data.gov)
Marten Hogeweg
Need more tools?
Metadata Support in Esri Geoportal ServerEnabling discovery and use of geospatial resources in heterogeneous environments
• Search and view geospatial resources
• Manage, publish, and store metadata
• View live map services
• Download data
• Born 2003
• Open Source since 2010
Metadata in the JavaScript APIhttps://github.com/Esri/arcgis-js-api/tree/master/dijit/metadata
• Part of JS API
• Used in ArcGIS Online/Portal for ArcGIS
• One Page JS/HTML App
• Load/Save to Local File
• Transform/Validate Metadata
• Support any XML Specification
- GeoPlatform Profile of ISO 19115-1
- SDSFIE-M
- BLM Profile of FGDC CSDGM
- DataCite
Marten Hogeweg
Metadata Profiles and Styles
Metadata Standards ( Then → Now / Future )
Ted Haberman : hdfcgroup.org
ISO 19115 (-1) Profiles
• A metadata profile for precision agriculture based on ISO 19115 standard.
• WMO Core Profile of the ISO 19115 Metadata Standard
• Transformation of HDF-EOS metadata from the ECS model to ISO 19115-based XML
• Draft African metadata profile of ISO19115• Samoan profile• NOKIS – an ISO 19115 Based Metadata System• The UDK and ISO 19115 Standard• Latin American Profile• Núcleo Español de Metadatos (NEM)• UK GEMINI• Biological profile• Coastal zone profile• S100 Metadata profile (IHO)
• Australia-New Zealand Profile
• North American Profile of ISO 19115:2003 (NAP)
• CAN/CGSB-171.100-2009
• INCITS 453-2009
• INSPIRE Metadata Directive• ISO19115/ISO19119 Application Profile for
CSW 2.0• C3Grid ISO 19115 Metadata Profile• Marine Community Metadata Profile of ISO
19115• Energy Industry Profile of ISO 19115-1• USGIN ISO 19139 Profile• DISDI Geospatial Metadata Profile (DGMP)• National System for Geospatial Intelligence
Metadata Foundation (NMF)
Metadata Styles
• FGDC CSDGM Metadata
• Content Standard for Digital Geospatial Metadata (CSDGM)
• ISO 19139 Metadata Implementation Specification
• ISO 19115:2003 / ISO 19139:2007
• ISO 19119:2005 / OGC CSW 2008
• ISO 19110:2005 / ISO 19110:2005 Amd.1 2011 Annex E
• North American Profile of ISO 19115:2003 (NAP)
• INSPIRE Metadata Directive
• ISO 19115-1/19115-3!!
Creating Your Own Metadata Style – ArcGIS Pro Metadata Toolkit
• Document Metadata XML for the Metadata Editor
• Change Pages in the Metadata Editor1
• Change Values in Drop-down Lists in the Metadata Editor
• Change Validation Rules
• Change Elements in a Metadata Page2
• Add Custom Metadata Element to a Page and to the ArcGIS Metadata Format
• Add Custom Pages to the Metadata Editor
• Change How Metadata is Displayed
• Change the Translators Used to Import and Export Metadata3
https://github.com/Esri/arcgis-pro-metadata-toolkit
There will be code!
Marten Hogeweg
Roadmap Feedback
Problem Oriented Features
Personalized Discovery
Smarter Facets
Advanced Search
Federated SearchBrowse Connected Content
Auditing page w/adv user/content search
Unified Metadata Editor (Across Platform)
INSPIRE Compliant Catalog and Item MD Export Options
ISO Compliant Catalog and Item MD Export Options
Improved Spatial Search
Automated Metadata Population
Default Metadata Options
Batch MD Editing Features
Easier MD Importing
Improved Imagery Discovery
Extendable Search
Improved Catalog Export Options
Deeper Search (MD, Semantic Relations)
GP – GeoPortal Server
DR – Data Reviewer Extensions
[Don’t] SEO My Public Data
Custom Profile Support (inc.
INSPIRE)
Print Your Certificate of Attendance
Print Stations Located in 150 Concourse Lobby
Tuesday12:30 pm – 6:30 pmExpoHall B
5:15 pm – 6:30 pmExpo SocialHall B
Wednesday10:45 am – 5:15 pmExpoHall B
6:30 pm – 9:30 pmNetworking ReceptionSmithsonian National Museumof Natural History
Download the EsriEvents app and find
your event
Select the session you attended
Scroll down to “Survey”Log in to access the
surveyComplete the survey and select “Submit”
Please Share Your Feedback in the App