Preserving Geospatial Data: Challenges and Opportunities

Steve MorrisSteve MorrisNCSU LibrariesNCSU Libraries

Indo-US Workshop on Trends Indo-US Workshop on Trends in Digital Preservationin Digital PreservationMarch 24, 2009March 24, 2009

Geospatial Data Types – Raster

• Digital orthophotography• Satellite imagery

Static data

Geospatial Data Types – Vector Data

• Point, line, and polygon• Attached attribute data

Often updated

Geospatial Data Types – Vector Data

• Transportation• Utilities• Environmental data• Cultural resources• Economic data• Political boundaries• Census• More …

Local data more detailed, current, and accurate than national data

Downtown Raleigh Near State Capitol

2005 Wake County Ortho

Imagery = DurableStatic Simple structureMostly open formats

Vector data = VolatileFrequent updateComplex structureMostly proprietary formats

Downtown Raleigh, NC Near State Capitol

2005 Wake County Ortho

Imagery = DurableStatic Simple structureMostly open formats

Vector data = VolatileFrequent updateComplex structureMostly commercial formats

New Location-Based Content

Present-day value in location-based services and mobile applications

Street ViewsOblique Imagery

3D Images

Carrboro, NC : Population 17,797 (2005 est.)

24 downloadable GIS data layers

4 OGC WMS services (web services)

6 web mapping applications

9 downloadable PDF map layers

Dynamic content Constantly updated information Data versioning

Digital object complexity Spatially-enabled databases Complicated, multi-component formats Proprietary formats

Geospatial Data: Compelling Issues

Data capture Backups are common, but not long-term archives Producer focus on current data Shift to web services-based access

Inadequate or non-existent metadata Consistent NC survey statistics: Only 40% of data

producers create and maintain metadata Existing metadata often needs to be normalized,

synchronized with the data, and remediated

Geospatial Data Preservation Challenges

Loss of memory about the data is also a problem

One of eight initial collection building projects in the Library of Congress NDIIPP (National Digital Information Infrastructure and Preservation Program)

Lead organizations: North Carolina State University Libraries and North Carolina Center for Geographic Information & Analysis (NCCGIA)

Focus: State and local government geospatial data in NC Repository development as catalyst for discussion Goal: Engage spatial data infrastructure in data archiving

Initial 3 year project extended to Dec. 2009

NC Geospatial Data Archiving Project (NCGDAP)

Spatial Data Infrastructure Role in Archiving Metadata standards and outreach

metadata quality, best practices Inventories

Reduce “contact fatigue”, shareable information store Content exchange networks

Leverage more compelling business reasons to put data in motion

Automate process, add technical & administrative metadata

Framework data communities Snapshot frequency, schemas, format strategies

Technical solutions: How do we preserve acquired content over the long term?

Cultural/Organizational solutions: How do we make the data more preservable—and more prone to be preserved—from point of production?

Current use and data sharing requirements – not archiving needs – are most likely to drive improved preservability of content and improvement of metadata

Different Ways to Address Preservation

Preservation Approaches: Temporal Data Snapshots

Issue: How frequently should county and municipal vector data layers be captured in archives?

Parcels, centerlines, jurisdictions, zoning, …

Parcel Boundary Changes 2001-2004, North Raleigh, NC

Preservation Approaches: Complex Spatial Databases

Manage database forward over time

Extract data layers to preservable form

Set aside archival snapshot of database

Preservation Approaches: Original Data vs. “Desiccated” Data

Complex data representations can be made more preservable (and less useful) through simplification

Preserving data representations (as well as data)

Counterpart to analog map = datasets plus data models, symbolization, classification, annotation, etc.

More data intelligence survives in PDF documents than survives in most other “desiccated” formats

Project Learning Outcomes Preservation of GIS “projects” is needed to support re-

creation of past work Preservation of data representations is needed to

document decision-making processes Validation, remediation, and conversion of data and

metadata is expensive: push for improvements upstream Some repositories handle “items”: can result in

“atomization” of data For vendors, frame data preservation as a “customer

problem” Must build the business case for preserving older data

Value in Older Data: Solving Business Problems

Suburban Development 1993/2002Near Mecklenburg County-Cabarrus County NC border

Land use change analysis

Real estate trends analysis

Site location analysis

Disaster response

Resolution of legal challenges Impervious surface maps

Project Action Outcomes State geospatial coordinating council recommendation:

“Establish archive and long term data access strategies” Archival and Long-Term Access Working Group formed

by state coordinating council Multi-State NDIIPP geospatial project started in 2007 Open Geospatial Consortium (OGC) Data Preservation

Working Group started Engagement with software vendors (notably ESRI)

Lead organizations: North Carolina Center for Geographic Information & Analysis (NCCGIA), State Archives of NC, with Library of Congress

Partners: State geospatial organizations of Kentucky and Utah State Archives of Kentucky and Utah NCSU Libraries in catalytic/advisory role

State-to-state and geo-to-Archives collaboration 2 year project: Nov. 2007-Dec. 2009 Archives as part of Spatial Data Infrastructure

GeoMAPP: Geospatial Multistate Archival and Preservation Partnership

Introduce GIS organizations and State Archives to each other

Archival selection and appraisal processes Retention schedule development Data transfer to archives Development of enhanced business case

GeoMAPP: Project Components

Thanks to the Library of Congress for financial support and collaboration as part of the National Digital Information Infrastructure and Preservation Program (NDIIPP)

Acknowledgement

Thank You!Contact:Steve Morris

Head, Digital Library Initiatives

North Carolina State University Libraries

Steven_Morris @ncsu.edu

NCGDAP:

http://www.lib.ncsu.edu/ncgdap

GeoMAPP

http://www.geomapp.com