Embed Size (px)
Citation preview
NHD Update Process Improvement Project
U.S. Department of the Interior
U.S. Geological Survey
Kevin McNinch, Dave Hughes, Paul Kimsey
Current NHD Tools
• NHD Geo Edit v3.3.2– Official release 04/18/2010– ArcGIS 9.3 sp1 & ArcGIS 9.3.1– NHD Model 1.06
Current NHD Tools
• NHD Geo Edit v3.3.3– Official release planned 06/07/2010– ArcGIS 9.3 sp1 & ArcGIS 9.3.1– Compatible with NHD Model 1.07
• Permanent Identifiers & ComID (retired in 1 year)• External Crosswalk table
– Backward compatibility to Model 1.06– Compatible with NHD Model 2.0
• WBD replaces old 1:250k hydrologic units
NHD Update ProcessImprovements
WHY WHAT HOW BENEFITS MID TERM LONG TERM
Short Term 0-1 Years
• WHY do we need to do this in the short term?
NHD Update ProcessImprovements
What do the Stewards Say:
1. Maintenance is too difficult.2. Inconsistent results after editing.3. Need “direct” editing of the NHD data.4. Quick and simple edits need to be easy.5. Software installation and performance is an
issue.
0
Status of Stewardship Today(Sub basins)
The Net Result
After five years of stewardship, the program is not as advanced as it should be.
• WHAT do we need to do to streamline the current process?
NHD Update ProcessImprovements
NHD Update Process Improvement Project
• Charter– 02/01/2010
• Survey– Distributed to all partners– Compiled results from 23 respondents
• Requirements Document– Draft April, 2010
Process Enhancements, Requirements Specifications:
• Ranked in order of High to Low:
• Refactor code for NHD tools and utilities
• Single Installation Package for all NHD Update Process Tools and Utilities
• Merge all current NHD toolbars into one toolbar with logical progression of tasks
Requirements Specifications:
• Utilize ArcSDE Versioning to manage NHD Update Process data transactions
• Allow editing with NHD GeoEdit Tool in geographic coordinate system
• Create queue for QC checks
• Create queued edit tool
Requirements Specifications:
• Make all NHD Update Process tools compatible with ESRI Personal Geodatabases (PGDB) and ESRI File Geodatabases (FGDB)
• Refactor NHD Update Process tools so that they are not reliant on ESRI Task Assistant
Requirements Specifications:
• Incorporate NHD Plus build and refresh QC checks to support NHD Plus creation
• Update process workflow to use ESRI ArcGIS Job Tracking Extension (JTX)
• Improve Metadata Process
Requirements Specifications:
• Reach Code Allocation utility process updates
• Bug Fixes and enhancements from NHD development group list
• Update tools and installation package to work with ArcGIS 10.0 when released
NHD Update Process
• Current process (29 steps)
NHD Stewardship Web site
Extract from database NHD75464.zip
UnZip
1804_orig1804_orig.mdbSave zip file in thisfolder
1804_alb1804_alb.mdbAll edits performed on this mdb
1804_qc1804_qc1.mdbCopy of NHD75464.mdb with transactions applied
1804_geo1804_geo.mdb1804_alb.mdb reprojected to geographic 1804_geo_load
1804_geo_load.mdbUsed for build flow, releveler and final XML extract
Reproject
Reproject
XML Extract
XML2pgdb
Yes
No
Make copy of final 1804_geo.mdb and name it1804_geo_load.mdb
Flow Diagram for Current NHD Maintenance Process
Corrections ?
NHD Stewardship Web site
Extract from database NHD75464.zip
UnZip
1804_orig1804_orig.mdbSave zip file in thisfolder
1804_alb1804_alb.mdbAll edits performed on this mdb
1804_qc1804_qc1.mdbCopy of NHD75464.mdb with transactions applied
1804_geo1804_geo.mdb1804_alb.mdb reprojected to geographic 1804_geo_load
1804_geo_load.mdbUsed for build flow, releveler and final XML extract
Reproject
Reproject
XML Extract
XML2pgdb
Yes
No
Make copy of final 1804_geo.mdb and name it1804_geo_load.mdb
Steps that can be eliminated from the current NHD Maintenance Process
Corrections ?
National Map
Replica Checkout
Flow Diagram for NHD Maintenance Process after Improvements
Replica Checkin
Run QC suite against entire
dataset
Resolve QC errors(Queued Edit tool)
Update using NHDGeoEdit tool
Run QC suite against entire dataset
Resolve QC errors(Queued Edit tool)
• HOW are we going to accomplish this?
NHD Update ProcessImprovements
• Agile Project Management - SCRUM• Requirements are captured as items in a list of
“product backlog”• Iterative development in a series of four-week
“sprints”• Product is designed, coded, and tested during
the sprint• Responsive to changing needs
NHD Update ProcessImprovements
Product Noise Level
Simple
ComplexAnarchy
Complicated
Technology
Re
qu
ire
me
nts
Far fromAgreement
Close toAgreement
Clo
se t
oC
erta
inty
Far
fro
mC
erta
inty
Source: Strategic Management and Organizational Dynamics by Ralph Stacey in Agile Software Development with Scrum by Ken Schwaber and Mike Beedle.
Putting it all together
Image available at www.mountaingoatsoftware.com/scrum
No changes during a sprint
• Plan sprint durations around how long you can commit to keeping change out of the sprint
Change
Scrum framework
•Product owner•ScrumMaster•Team
Roles
•Sprint planning•Sprint review•Daily scrum meeting
Ceremonies
•Product backlog•Sprint backlog•Burndown charts
Artifacts
• Benefits
• Agile development• Frequent delivery• Improved Communication
• Pro-active project management
NHD Update ProcessImprovements
• BENEFITS to the stewardship community
NHD Update ProcessImprovements
- Level of complexity allows for a much broader audience from “expert” to “knowledgeable users”
- Reduces training time - Reduces need for constant technical support
- Eliminates re-training
Where Does NHD Stewardship Go From Here?
• Looking beyond lifecycle of current NHD GeoEdit Tools and NHD Stewardship Processes– Current development will be as reusable as possible
• Begin planning and requirements gathering—create project charter to reflect this work– Completed NHD Architecture Planning Meeting May 13,
2010• Analysis of current process (On-going)• Create NHD Update Process that:
– Addresses a broader spectrum of NHD users– Simplifies
NHD User Base
Huge number of Casual Users
ManyKnowledgeable Users
SomeExperts
NHD Stewardship
Addressed Very Wellwith Edit Tool Complete Edit Capability
Poorly AddressedNo Intermediate ToolBasic Editing
Not Yet AddressedWeb Edit ToolIdentify and Markup EditsIn development
Experts
Casual Users
Knowledgeable Users
• MID TERM 2-3 Years
NHD Update Process
• Web-Based Markup/Editing Tools– Alabama Web Edit Tool(WET)– Requirements gathering group ongoing for
national web markup/editing tool• Detailed requirements gathering and
analysis• Improve web services
• LONG TERM 4+ Years
NHD Update Process
• Evaluating other technologies, trends, and tools for future of Vector Data Stewardship– Open source or COTS tools– Open data standards– Geosyncronization– Volunteered Geographic Information (VGI)
