Embed Size (px)
Citation preview
Monterey Bay 2006 Data Monterey Bay 2006 Data System OverviewSystem Overview
October 6, 2005October 6, 2005
Michael GodinMichael GodinMonterey Bay Aquarium Research InstituteMonterey Bay Aquarium Research Institute
(MBARI)(MBARI)
AgendaAgenda
The ChallengeThe Challenge
User & System RequirementsUser & System Requirements
Usage of the SystemUsage of the System
System ArchitectureSystem Architecture
Development MilestonesDevelopment Milestones
What do we need before the VE?What do we need before the VE?
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Data Management ChallengeData Management Challenge
Large number of data sourcesLarge number of data sources
Large variety of data sourcesLarge variety of data sources
Dynamic systemsDynamic systems
–Data sources may appear during Data sources may appear during the campaignthe campaign
No standard data formatNo standard data format
–Streams or files, automated or Streams or files, automated or manualmanual
Large number of data sourcesLarge number of data sources
Large variety of data sourcesLarge variety of data sources
Dynamic systemsDynamic systems
–Data sources may appear during Data sources may appear during the campaignthe campaign
No standard data formatNo standard data format
–Streams or files, automated or Streams or files, automated or manualmanual
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
User & System RequirementsUser & System Requirements
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
User RequirementsUser RequirementsUser Requirements
Participants
Other Users
Transfer Data From Server
Transfer Data To Server
Transfer Data From Server
Decision Making Tools
Image Access to Embargoed Data
Full Access to non-embargoed data
State of system information
Observation state
Adaptive sampling tools
Model-data comparison
Common Format Data
Original Format Data
Add new Data Streams to Server
Add new Data to Server
Recognize any Data Format
Quality/Version Control
Secure Archive
Model Outputs
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
System RequirementsSystem RequirementsMeeting User NeedsMeeting User Needs
System Requirements
Transfer Data To Server
Decision Making Tools
Add new Data Streams to Server
State of system information
Observation state
Adaptive sampling tools
Model-data comparison
Common Format Data
Original Format Data
Add new Data to Server
Recognize any Data Format
Quality/Version Control
Secure Archive
Transfer Data From Server
Data Access Security
Model Outputs
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Describing Incoming Data File StreamsDescribing Incoming Data File Streams
Interfaces for Adding DataInterfaces for Adding Data
Interfaces for Getting Data & ImagesInterfaces for Getting Data & Images
Supported Interfaces for Using DataSupported Interfaces for Using Data
Interfaces for System StateInterfaces for System State
Usage of the SystemUsage of the System
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Describing Incoming Data StreamsDescribing Incoming Data Streams
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for Adding DataInterfaces for Adding Data
FTPFTP
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for Adding DataInterfaces for Adding Data
FTPFTP
WebWeb
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for Adding DataInterfaces for Adding Data
FTPFTP
WebWeb
E-mailE-mail
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Dynamic Data Ingest
Your Server
FTP or HTTP
Interfaces for Adding DataInterfaces for Adding Data
FTPFTP
WebWeb
E-mailE-mail
ServerServerPullPull
Your DatabaseCentral Server
Storage
Request
DataResponse
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
•Live Access Server (LAS)Live Access Server (LAS)
Interfaces for Getting Data & ImagesInterfaces for Getting Data & Images
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for Getting Data & ImagesInterfaces for Getting Data & Images•Live Access Server (LAS)Live Access Server (LAS)•Metadata Oriented Query Assistant (MOQuA)Metadata Oriented Query Assistant (MOQuA)
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
•Ocean Data ViewOcean Data View
Supported Interfaces for Using DataSupported Interfaces for Using Data
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
•Ocean Data ViewOcean Data View•Space/Time Toolkit?Space/Time Toolkit?•AMES CIP?AMES CIP?
Supported Interfaces for Using DataSupported Interfaces for Using Data
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for System StateInterfaces for System State• Network Status (Nagios)Network Status (Nagios)
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for System StateInterfaces for System State• Network Status (Nagios)Network Status (Nagios)• Last Known Position Of AssetsLast Known Position Of Assets
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Interfaces for System StateInterfaces for System State• Network Status (Nagios)Network Status (Nagios)• Last Known Position Of AssetsLast Known Position Of Assets• Processing Stoplight ChartProcessing Stoplight Chart
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Where We Were: 2003 Data SystemWhere We Were: 2003 Data System
Where We Got to: Post-2003 Data SystemWhere We Got to: Post-2003 Data System
Where We Want to be: Where We Want to be: MB2006 Data System ArchitectureMB2006 Data System Architecture
Where We Will be Soon:Where We Will be Soon:Virtual Experiment Data System Virtual Experiment Data System
Data System ArchitectureData System Architecture
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
OriginalArchive
110234999
Where We Were: Where We Were: 2003 Data System2003 Data SystemArchitectureArchitecture
Devices
Models & Derived Outputs
PrincipalInvestigators’Processing, QC, & Archives
CommonArchive
101110110011
PrincipalInvestigators’Downloads
Derived Data
Primary Data
Polarbear
MBARI Firewall
CustomProcessing
PublicImageRequests
mbari.org(static images)
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
OriginalArchive
110234999
Where We Got To:Where We Got To:Post-2003 Data Post-2003 Data System ArchitectureSystem Architecture
Devices
Models & Derived Outputs
PrincipalInvestigators’Processing, QC, & Archives
CommonArchive
101110110011
PrincipalInvestigators’Downloads
Derived Data
Primary Data
Polarbear
MBARI Firewall
CustomProcessing
aosn.mbari.org
Data QueryData line 1more datalast data
PublicImageRequests
PrincipalInvestigators’Data Requests
Reanalyses
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
OriginalArchive
110234999
Data StreamDescriptions
MB2006 Data System ArchitectureMB2006 Data System Architecture
Devices
Models & Derived Outputs
DynamicData Ingest
PrincipalInvestigators’Processing, QC, & Archives
CommonArchive
101110110011
Data QueryData line 1more datalast data
Static Metadata
Dynamic Metadata
System State
Central Server
PrincipalInvestigators’Data Requests
PublicImageRequests
DecisionMakerQueries
Derived Data
Primary Data
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
OriginalArchive
110234999
Data StreamDescriptions
Virtual Experiment Data System ArchitectureVirtual Experiment Data System Architecture
Devices
Models & Derived Outputs
DynamicData Ingest
PrincipalInvestigators’Processing, QC, & Archives
CommonArchive
101110110011
Data QueryData line 1more datalast data
Static Metadata
Dynamic Metadata
System State
Central Server
PrincipalInvestigators’Data Requests
PublicImageRequests
DecisionMakerQueries
Derived Data
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
Development MilestonesDevelopment MilestonesBasic Data Ingest, Processing, Storage, and Basic Data Ingest, Processing, Storage, and Serving: October 28, 2005Serving: October 28, 2005– Mods to MBARI’s Shore Side Data SystemMods to MBARI’s Shore Side Data System– Mods to NOAA/PMEL’s Live Access ServerMods to NOAA/PMEL’s Live Access Server
Data Version Control, Asset Map: 11/14/2005Data Version Control, Asset Map: 11/14/2005Waypoint–Path comparison: before VE?Waypoint–Path comparison: before VE?Post-Virtual Experiment Milestones:Post-Virtual Experiment Milestones:– User Access Controls & Public ImagesUser Access Controls & Public Images– Stop-Light Chart, Network MapStop-Light Chart, Network Map– Custom Output Data Formats & GraphicsCustom Output Data Formats & Graphics– Saved Queries and Routine OutputsSaved Queries and Routine Outputs
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
What Do We Need for the VE?What Do We Need for the VE?
Example Data Sets, by 10/21/2005Example Data Sets, by 10/21/2005
Desired Data Outputs, by 10/28/2005Desired Data Outputs, by 10/28/2005
Metadata by 11/14/2005Metadata by 11/14/2005(entry form will be forthcoming…)(entry form will be forthcoming…)
– Data Import Specifications (how to get data)Data Import Specifications (how to get data)– Data Reference InformationData Reference Information
Nice to have by 11/14/2005:Nice to have by 11/14/2005:– Waypoint filesWaypoint files
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
The EndThe End
M. Godin, MBARI, 10/6/2005M. Godin, MBARI, 10/6/2005
![Searching for CO 2 in marine sediment pore waters: Methods ... · et al, 2012]. However, through recent advances in engineering at the Monterey Bay Aquarium Research Institute (MBARI),](https://img.dokumen.tips/doc/110x75/5f9e44cf13e36f770173372b/searching-for-co-2-in-marine-sediment-pore-waters-methods-et-al-2012-however.jpg)
