Upload
muriel
View
40
Download
0
Embed Size (px)
DESCRIPTION
Lessons Learned from Getting Data into STORET: A State’s Perspective. Mike Beauchene, State of Connecticut Department of Environmental Protection; Mellony Hoskinson, Dwane Young, RTI; Dave Wilcox, Gold Systems. STORET Success. Overview. Connecticut’s Data Management System - PowerPoint PPT Presentation
Citation preview
Lessons Learned from Getting Lessons Learned from Getting Data into STORET:Data into STORET:
A State’s PerspectiveA State’s Perspective
Mike Beauchene, State of Connecticut Department of Environmental Protection;Mellony Hoskinson, Dwane Young, RTI; Dave Wilcox, Gold Systems
STORET SuccessSTORET Success
OverviewOverview
Connecticut’s Data Management SystemConnecticut’s Data Management System Re-formatting data to STORET Re-formatting data to STORET
specificationsspecifications– Column MappingColumn Mapping– Crosswalk TablesCrosswalk Tables– QueriesQueries
Challenges/BenefitsChallenges/Benefits
Cro
ssw
alk
The ProcessThe ProcessCHAOS
Database
Queries$$$
Step 1. Create CT DEP Database
Step 2. Populate the database
Step 3. Data AnalysisUpdate database and create crosswalks
Step 4. Build Queries
Step 5. Load to STORET
Now that the process is developed, data can be loaded to the CT DEP database, and easily migrated to STORET.
SIM
Connecticut - 2001Connecticut - 2001
Move data system from hard copy to electronic systemMove data system from hard copy to electronic system Use STORET data structure as guideline for setting up Use STORET data structure as guideline for setting up
Connecticut’s databaseConnecticut’s database– ProjectsProjects– TripsTrips– StationsStations
Key to this process is implementing required STORET data Key to this process is implementing required STORET data elementselements– Analytical methodsAnalytical methods– Gear TypeGear Type– Sample Collection Procedures, etc.Sample Collection Procedures, etc.
Step 1 – CT DEP creates a database to store monitoring data that meets DEP’s needs
Database ElementsDatabase Elements
Projects
Trips
Stations
Samples
Connecticut’s Connecticut’s Data Management SystemData Management System
Assigned Project IDs and appropriately Assigned Project IDs and appropriately tagged all related data under projecttagged all related data under project
Project managers selected from STORET Project managers selected from STORET lists which methods, gear, collection lists which methods, gear, collection procedures, etc. were used on projectprocedures, etc. were used on project
Projects are maintained separately, but all Projects are maintained separately, but all contain same data elements and can be contain same data elements and can be imported into STORETimported into STORET
Step 2 – Populate CT DEP Database
From State to STORET
AnalysisAnalysis
Identify and map CT DEP data elements to the Identify and map CT DEP data elements to the corresponding STORET data elementscorresponding STORET data elements– ProjectsProjects– StationsStations– TripsTrips– ActivitiesActivities
Identify missing data elements that would be Identify missing data elements that would be required by STORETrequired by STORET
Step 3 – STORET Comparison
Match CT DEP values with STORET valuesMatch CT DEP values with STORET values– Accounts for capitalization, abbreviations, etc.Accounts for capitalization, abbreviations, etc.– Provides a means for accounting for data Provides a means for accounting for data
elements that are required by STORET, but is elements that are required by STORET, but is not in DEP’s databasenot in DEP’s database
Can be easily modified to reflect changes Can be easily modified to reflect changes made in current systemmade in current system
Crosswalk Tables are the keys to creating Crosswalk Tables are the keys to creating new STORET formatted queriesnew STORET formatted queries
Crosswalk TablesCrosswalk Tables
Step 3 cont. – Build Crosswalk Tables
Example of how a characteristic in CT DEP’s Database could be broken down in a crosswalk table to provide for the necessary STORET data elements.
Data Select QueriesData Select Queries
Query designQuery design– State data tablesState data tables– Crosswalk tablesCrosswalk tables
Queries take care of conditional situations (i.e. Queries take care of conditional situations (i.e. adding *Non-Detect when necessary)adding *Non-Detect when necessary)
Since the queries run directly on CT DEP Since the queries run directly on CT DEP database, when DEP data changes, so do the database, when DEP data changes, so do the queriesqueries
Provide a means for flagging data in CT DEP Provide a means for flagging data in CT DEP database that has been loaded into STORETdatabase that has been loaded into STORET
Step 4 – Build Queries that use DEP database and crosswalk tables to create SIM ready files for loading into STORET
Load the Data to STORETLoad the Data to STORET
Using SIM and the data queries, DEP can now Using SIM and the data queries, DEP can now load the data into STORETload the data into STORET
Using these same queries, DEP then can flag all Using these same queries, DEP then can flag all of the data (Stations, Activities) that have been of the data (Stations, Activities) that have been loaded to STORET so that they would be excluded loaded to STORET so that they would be excluded from future data loadsfrom future data loads
Step 5 – Load the data into STORET
Challenges to OvercomeChallenges to Overcome
Some data elements discovered during the Some data elements discovered during the data analysis that are required by STORET, data analysis that are required by STORET, and therefore required changes to CT DEP’s and therefore required changes to CT DEP’s database.database.
Crosswalks have to be kept up to date. If Crosswalks have to be kept up to date. If CT DEP adds a new characteristic, then that CT DEP adds a new characteristic, then that characteristic needs to be added to the characteristic needs to be added to the crosswalk.crosswalk.
Benefits to StateBenefits to State Maintain original data management system Maintain original data management system If there is a need to change the format for the If there is a need to change the format for the
SIM-ready files, then it’s relatively easy to SIM-ready files, then it’s relatively easy to change the queries.change the queries.
DEP doesn’t need to adopt all of the STORET DEP doesn’t need to adopt all of the STORET codes, but rather just needs to keep the codes, but rather just needs to keep the crosswalks up to date.crosswalks up to date.
Now that the data analysis is done, and the Now that the data analysis is done, and the process is established, future data loads to process is established, future data loads to STORET are relatively painless.STORET are relatively painless.