Safeguarding GIS Data through Metadata Christopher Cialek & Nancy Rader Minnesota Land Management Information Center UW River Falls Department of Geography

Safeguarding GIS Data through Metadata

Christopher Cialek & Nancy Rader

Minnesota Land Management Information Center

UW River Falls Department of Geography

December 15, 2003

What’s Metadata?

If you had two cans without labels, which would you eat?

Without a label, how would you know which was tuna and which was cat food?

Cat Food?

Tuna?

By the end of the Workshop . . .

Understand what metadata is; appreciate its value

Become familiar with metadata standards

Create your own metadata records

Use a search engine to find data

Know where to go for help

You will:

From: "Tim Ogg" To: <[email protected]> Date: 8/3/00 11:53AM Subject: Re: soils documentation & status chart Hi Nancy - I have a Washington Co cover that I think came from the county or the SWCD. It was residing on the computer of a former employee so I don't know exactly where it came from. The attributes on it are all wrong. Polygons are severely mislabeled all over the place. I intend to get a "good" copy from the county or SWCD. Where’s the best place to look? Thanks Tim

From: stacey stark <stark-stacey@ To: MNPLAN.MNPLANPO(rader-n) Date: 9/21/98 12:09PM Subject: Quaternary Geology layer Nancy, I am trying to learn as much as I can about the sources of the Quaternary geology layer of the MGC100 data. I do not know who or where the data was transformed / processed from the EPPL7 format – if you have any information, that would be very helpful. (log indicates user name whpa?). I am also wondering if the coverage I have might be the scanned polygon coverage before it was converted to EPPL6? The coverage name used to be qgeo - but I don't recognize the user. Would you have information? I hope I am not too much of a pain, I talked to MGS. They have never processed a polygon coverage of Quaternary geology in this area. Thank you very much. Stacey Stark GIS Specialist EPA Mid-Continent Ecology Division

What’s Metadata?

“the information that makes data sets understandable, usable and

sharable.”

International Standards Organization

FDA Food LabelWe often use metadata without knowing it -- even a food label is an example of metadata!

What’s Metadata?

Structured format

Specific content

Necessary information

What’s Metadata?

Search metadata to find resources

in the library

Library community has developed metadata

systems to describe books

– Dublin Core

Allows you to

search by title,

author, subject…

What are Metadata Used For?

MANAGING DATABASES

COMPARING DATA SETS

FACILITATING DATA SHARING

PROVIDING TECHNICAL SPECS

FINDING DATA

Standards

Content Standard for Digital Geospatial Metadata (CSDGM)

– Established by the FGDC

– v1.0 in 1994, v2.0 approved 1998

– Structured, thorough, complex

– Foundation standard for the NSDI

– “Mandatory” for federal agencies

– Entire GIS data user community encouraged to use

– www.fgdc.gov/metadata/contstan.html

Standards

Minnesota Geographic Metadata Guidelines (MGMG)

– Derived from the federal standard in 1998

– Simplified, but retains all required fields

– Became state standard in 1999

– www.gis.state.mn.us/stds/metadata.htm

Standards

ISO 19115

– International geospatial metadata standard

– Conceptual design approved May 2003: ISO 19115

– Implementation schema draft under review: ISO 19139

– Quite complex, but only 30 core elements

– Promises more guidance than CSDGM

– Crosswalk to CSDGM: http://ogcengine.nima.mil/metadata/index.htm

Metadata Structure Examples

mandatory

mandatory, if applicable

optional

LINEAGE

Source Information

Source Citation

Source Time Periodof Content

Citation Information

Source Scale

Type of Source Media

Time Period Info

Source Currentness

Source Contribution

Source Citation Abbrev

Process Step

Process Date & Time

Source Used Citation

Process Description

Source Produced

Process ContactContact Information

CSDGM

LINEAGE

MGMG

LINEAGE

Statement

Process Step

Description

Rationale

Processor

Date & Time

Source

Description

Scale

Spatial Reference Sys

Source Citation

Source Extent

Source Step

ISO

TheMinnesotaGeographic Metadata Guidelines

The MGMG

SUBSET OF A FEDERAL STANDARD

MADE UP OF SEVEN SECTIONS

DRIVES WEB SEARCH TOOLS

USED BY OVER 75 ORGS IN MN

MN STANDARD; FGDC RECOGNIZED

Seven Sections of the Minnesota Geographic Metadata Guidelines

Minnesota State Guideline 17-1.2http://www.gis.state.mn.us/stds/metadata.htm

1Identification

2Data

Quality

3Spatial DataOrganization

4Spatial

Reference

5Entity andAttribute

6Distribution

7MetadataReference

Originator Attribute Accuracy Native Data SetEnvironment

HorizontalCoordinate Scheme

Entity and AttributeOverview

Publisher Metadata Date

Title Logical Consistency GeographicReference (Tabular)

Ellipsoid Entity and AttributeDetailed Citation

Publication Date Metadata ContactInformation

Identifier (optional) Completeness Spatial Object Types Horizontal Datum &Units

DistributorInformation

Metadata StandardInformation

Abstract Positional Accuracy Tiling Scheme Resolution Distribution Liability

Purpose Lineage Altitude Datum &Units

Transfer Format

Content Date Source Scale Depth Datum & Units Transfer Size

Currentness If Raster Ordering Instructions

Progress If Geographic Online Linkage

Maintenance andUpdate Frequency

If UTM

Spatial ExtentDescription

If State Plane

BoundingCoordinates

If County Coordinate

Keywords If User SpecifiedProjection

Constraints If Other

Contact Information

Browse GraphicInformationAssociated DataSets

Abstract

Purpose

1Identification

Originator

Title

Identifier (optional)

Content Date

Currentness

Progress

Maintenance andUpdate FrequencySpatial ExtentDescription

BoundingCoordinates

Keywords

Constraints

Contact Information


A Walk Through the Guidelines

Originator: name of organization or individual that developed the data

Title: name by which the data set is known

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information


TITLE

– Too cryptic: niclcpy3

– Too general:Landuse

– Acronyms:Wisconsin DOPs

– Too detailed: Wetland Polygon Coverage Overlay for

St. Cloud (USGS Quad)

– Just right (includes theme, area, date): Minnesota Telecommunications Service

Area Boundaries, June 2002

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Abstract: summary of what’s in the data set

Purpose: why the data set was developed

Time Period of Content: single date that best describes when the data are current

Currentness: text describing what the Time Period date is referring to, e.g., range of dates of aerial photography

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Abstract: This land cover data set was derived from 30 meter resolution LANDSAT Thematic Mapper (TM) satellite imagery. Classification is divided into 15 classes with source imagery dates ranging from September 1991 to August 1996. Both a raster and a vector version are available.

Purpose: Land use planning, natural resource monitoring

Time Period of Content: August 9, 2000

Currentness: Date of source imagery (LANDSAT-5 TM, bands 3, 4, and 5) ranges from September 20, 1991 to August 30, 1996.

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Spatial Extent: description of the geographic area covered (Wisconsin, St. Croix County)

Bounding Coordinates: the extreme north, south, east and west limits of coverage expressed in latitude and longitude values

W -95.4 E -89.5 N 49.4 S 45.5

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Keywords: words or phrases that summarize the theme and location of the data set, together with the name of any formal list of keywords (thesaurus)

Too general: GIS, layer, surveyJust right: Feedlot, animal agriculture,hog

Constraints: any restrictions to the access or use of the data set

Access: Due to increased security measures taken after 9/11/01, this data set is no longer available online.

Use: . . . right to use these data for any internal purpose

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Contact Information: the person who can answer questions about the content or development of the data set

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Browse Graphic: a sample illustration of the data set

Abstract

Purpose

1Identification

Originator

Title


Content Date

Currentness

Progress


BoundingCoordinates

Keywords

Constraints

Contact Information



Associated Data Sets: information about other, related data sets that may be of interest

If you’re interested in this data set, here are others that may also interest you. NOT a list of source materials.

For information on other air photos available for Minnesota, see www.lmic.state.mn.us/chouse/airphoto


Attribute Accuracy: qualitative or quantitative explanation of how accurately features in the data set have been described, including procedures used to assess accuracy (examples: field-checking, checkplots, frequency counts to find invalid codes)

2Data

Quality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale


Completeness: information about selection criteria, omissions, generalization, etc.

2Data

Quality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale

EXAMPLE: Geographic exclusion

“Data was not available for Smith County.”


Completeness: information about selection criteria, omissions, generalization, etc.

2Data

Quality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale

EXAMPLE: Categorical Exclusion

“Municipalities with population

under 2500 not included.”

A Walk Through the Guidelines Positional Accuracy: an explanation of what’s

known about the horizontal and vertical accuracy of the data set (can be qualitative or quantitative)

Qualitative example:

2Data

Quality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale

Data was collected in the field and plotted on a variety of base maps. Archaeological properties visited in the past 30 years are located on USGS maps. Almost all site locations are accurate to the quarter section. Most site locations are accurate to within a quarter-quarter section. Site boundaries are poorly defined, as are site centroids.

Minnesota State Historic Preservation Office Archaeological Inventory

A Walk Through the Guidelines Positional Accuracy: an explanation of what’s

known about the horizontal and vertical accuracy of the data set

Quantitative example:

2Data

Quality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale

Using the National Standard for Spatial Data Accuracy, this data set tested 1 foot horizontal accuracy at 95% confidence level.

City of Minneapolis (from Positional Accuracy Handbook)


Lineage: information about the sources of data used to construct the data set & processing steps applied

2Data

Quality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale

LINEAGE RECIPE2

DataQuality

Attribute Accuracy

Logical Consistency

Completeness

Positional Accuracy

Lineage

Source Scale

1. Source Information

Data set reference

Scale

Media

Time period of content

Source contribution

Source Metadata reference

2. Processing Step

Process description

software used

organization doing the processing

Process date

3. Miscellaneous Notes

A Walk Through the Guidelines3

Spatial DataOrganization

Native Data SetEnvironmentGeographicReference (Tabular)

Spatial Object Types

Tiling Scheme

A Walk Through the Guidelines4

SpatialReference

HorizontalCoordinate SchemeEllipsoid

Horizontal Datum &Units

Resolution

Altitude Datum &UnitsDepth Datum & Units

If Raster

If Geographic

If UTM

If State Plane

If County Coordinate

If User SpecifiedProjectionIf Other


Entity & Attribute Overview: description of the information content of the data set: the features it represents (entities) and details about them (attributes). An entity might be road and the attributes that describe it might include interstate, 6 lanes, concrete.

Entity & Attribute Detailed Citation: reference to other sources of detailed information on the content of the data set; pointer to a data dictionary

5Entity andAttribute

Entity and AttributeOverviewEntity and AttributeDetailed Citation


Examples: Land Use Codes5

Entity andAttribute

Entity and AttributeOverviewEntity and AttributeDetailed Citation

Useless:

212223

Slightly Better:

AGRICULTURAL LAND

21 - Cultivated Land 22 - Pasture Land23 - Transitional Agricultural Land

A Walk Through the GuidelinesMuch Better:

AGRICULTURAL LAND

21 - Cultivated Land Cultivated land includes those areas under intensive cropping or rotation, including periods when a parcel may be fallow. It represents land planted to forage or cover crop. The units exhibit linear or other patterns associated with current or relatively recent tillage.

22 - Pasture LandLand in active pasture use. This class was discontinued and combined into 23.

23 - Transitional Agricultural LandThis category includes areas that show evidence of past tillage but do not now appear to be continuously cropped or in a crop rotation. Parcels in this unit include fields that are idle or abandoned and may or may not have been planted to a cover crop. In addition to displaying some evidence of past tillage, they usually are relatively uniform in vegetation.


Publisher: organization or individual that distributes the data set

Distributor Information: person who can answer questions about the distribution of the data set

Distribution Liability: statement of any liability assumed by the distributor

Limitations

Warranty

Liability

Redistribution Conditions

Data Delivery Terms

6Distribution

Publisher

Publication Date


Distribution Liability

Transfer Format

Transfer Size

Ordering Instructions

Online Linkage


Ordering Instructions: instructions for obtaining the data set. If applicable, instructions for acquiring data through Online Linkage element below

Online Linkage: (optional) when the data set is available online, this is the link to the Internet site where it can be downloaded

6Distribution

Publisher

Publication Date


Distribution Liability

Transfer Format

Transfer Size

Ordering Instructions

Online Linkage

Purpose of a Metadata Entry Tool

Organizes metadata content

Provides help

Formats results– printed reports– webpages– Clearinghouse searches

Can it write the whole record for you?

Can you send me a tool to generate

metadata quickly?

Tools

ArcCatalog– Part of ESRI’s ArcGIS software– FGDC version is built-in– MGMG version, download at:

www.lmic.state.mn.us/chouse/arccatalog.html

DataLogr– Software independent– Free!– FGDC version, download at:

www.imagin.org/datalogr.htm

– MGMG version, download at: www.lmic.state.mn.us/chouse/datalogr.html

ArcCatalog

Editor– create a new record– change an existing metadata record

Stylesheet– needed to view a metadata record

MGMG Editor

7 sections

Help

FGDC Editor7 sections

Help

many sub sections for each section

MGMG Editor

FGDC Editor

Repeating screens

Lineage

Converting formats

ArcCatalog: stores records in XML format

FGDC Editor

– Use Export button to create HTML

MGMG Editor

– Use MGMG Converter to create HTML Creates hotlinks for web addresses

DataLogr: stores records in LGR format

Comes with FGDC or MGMG Converter

Hands-on Session with ArcCatalog

Part 1: Get started– Become familiar with viewing and editing screens

Part 2: Edit a record– Change the content of an existing metadata record

Part 3: Display a record– Create an HTML version of your edited record

– Compare different views of a record

Part 4: Create a new record– Evaluate automatically created record

– Create your own starter template

How to Make this Easier…

Create metadata DURING your project

When you create new data

When you change existing data

If you write metadata as you go along,at the end, it will be done!

It’s getting to be part of my thought process – I’m

starting to write things down when I do them


Prioritize legacy data

What is most critical?

What are you asked about the most?

What may be lost soonest?– Staff turnover

– Stuff that only one person or organization knows

– Information that you quickly forget


Use existing resources

Guidelines and tools

Starter templates

Existing documents

Other peoples’ metadata


Share the task

Ask someone else to read your record

Psychology

What is preventing you from getting this done?

Think metadata is optional?

Lack of resources?– Short term investment – long term payoff

– Streamline process (templates, share the task, etc.)

Don’t like to write?– Draft (can modify); fill out in any order; time of day

– Use editor

– Use interviewsI don’t

have time!

Part science, part art

Some information is very defined

Content

– Fixed set of choices

e.g., “Progress” has only three choices:

Complete, In Work, Planned

Format

– e.g., “Source scale denominator”: no commas (e.g., 24000)

Part science, part art

Use your judgment

Defining data sets

– not too fine or too broad

Amount of detail

– Answer the questions that you

want answered

Remember your audience

– If online, anyone with an Internet connection could see it.

• Spell out acronyms, add “Minnesota” to your title, etc.

• Title and abstract are critical

How much do I need to write?

Psychology

Forgetting?

– Ways to jog your memory

• Supporting documents, emails, other peoples’ memories

• Add later if you think of it

• Write something wrong in first draft

Data or process not perfect?

– Shortcuts, records not kept, …

– Purpose field

– “I don’t know”

I forget what we did…

Psychology

Carrots

Sticks

Quack!


Something is better than nothing!Do Nothing

Do Something

Organize using a metadata standard

Use a metadata tool

How do I prepare for ISO? NOW!

Continue metadata creation using CSDGM v2

Use ISO-poised software– ESRI ArcCatalog

– Intergraph / SMMS

Add Topic Categories as theme keywords with ‘ISO 19115 Category’ as thesaurus

Monitor FGDC website for news, materials, and training opportunities

How do I prepare for ISO? SOON!

Participate in US Profile reviews

Educate and inform your organization that a migration, with assistance is coming

Develop a metadata conversion plan using crosswalks and conversion software

Adopt CSDGM v3.0 when finalized

More Help

LMIC

FGDC

ArcCatalog

www.lmic.state.mn.us/chouse/meta_help.html

www.fgdc.gov/metadata/metadata.html

Contacts

Wisconsin: State Cartographer’s OfficeDirector: Ted Koch [email protected]

Outreach: A.J. Wortley [email protected]

FGDCMetadata Coordinator: Sharon Shin [email protected]

Contracted Assistance: Lynda Wayne [email protected]

Minnesota: LMICData Management: Chris Cialek [email protected]

GIS Data Coordination: Nancy Rader

[email protected]

Documents

Safeguarding GIS Data through Metadata Christopher Cialek & Nancy Rader Minnesota Land Management Information Center UW River Falls Department of Geography