Upload
ashley-whitehead
View
222
Download
1
Tags:
Embed Size (px)
Citation preview
28 Mar 2006
ORION CI
ORION CI Conceptual Architecture Team
Progress Report
AgendaContext
Presentation (Options)Discussion
28 Mar 2006
ORION CI
Expanding Role of Environmental Observing
• From individual expedition to collaborative observation. – Long-term persistent presence, continuous 24/7 time series– Increasing numbers and diversity of sensors– Multiple, complex data streams– Many more types of users and applications– Ability to control instruments, observing systems, and large-scale
experiments.
• Ubiquitous data requiring greater differentiated expertise.– From kilobits to terabytes in a matter of hours.– Collaborative discovery by groups of researchers with different
vocabularies and expertise looking at complex data streams.
• A need for cooperating networks of data/processing/analysis systems that are different than in the past.
28 Mar 2006
ORION CI
Team Members
Matthew Arrott (UCSD, Chair)Alan Chave (WHOI)John Graybeal (MBARI)Eric Guillemot (NEPTUNE Canada)Ingolf Krüger (UCSD)Benoît Pirenne (NEPTUNE Canada)
28 Mar 2006
ORION CI
Team Mandate
• Develop a set of information system capabilities.• Develop conceptual level design options to
support the Ocean Observatories Initiative (OOI) systems operations and RFP development.
• Develop a CI Work Breakdown Structure (WBS).• Create resource estimates for each line in the
WBS to prepare a Rough Order of Magnitude (ROM) cost and timeline for the project.
28 Mar 2006
ORION CI
Presentation Options
• How will using ORION be different from the way science is done today?
• What will CI do for us as individuals and as a community?
• How has CI been implemented in other environments?
• What is the current architectural concept?• How can we be sure this is a viable,
implementable infrastructure?• How did you go about putting this together?
28 Mar 2006
ORION CI
Getting A Book: The Way It Used To Be
QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture. QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
I need to special order a book.What book do you want?What book do you want?
It’s by Rachel Carson, Silent Spring.Yes, we can order that. It’s 23.95.Yes, we can order that. It’s 23.95.
When will you have it?Maybe Tuesday, but if it’s not in stock, not for a month.Maybe Tuesday, but if it’s not in stock, not for a month.
Can you send it to me?No, you’ll need to pick it up.No, you’ll need to pick it up.
OK, can I pay by phone?Yes, we’ll need your credit card and address.Yes, we’ll need your credit card and address.
OK, here’s my card info and address…OK, we’ll call you when it’s in.OK, we’ll call you when it’s in.
28 Mar 2006
ORION CI
Getting A Book: The Way It Is Now
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
ORION CI
Doing Science: The Way It Has Been
Hey Francisco, you still running that mooring in Monterey?Yeah, John, what do you need?Yeah, John, what do you need?
Is it collecting current data?Yeah, you looking for profiles?Yeah, you looking for profiles?
Yup, are you doing that with an ADCP?Yes, we’ll probably start profiling soon, too.Yes, we’ll probably start profiling soon, too.
No, don’t need that. How do I get the data?You want the raw data or QCd? Real-time?.You want the raw data or QCd? Real-time?.
Oh, the QC’d, delayed mode data, definitely, last 3 years worth.That’d be off our dods site, here’s the address..That’d be off our dods site, here’s the address..
Documentation come with that?We got some, I’ll have to get one of my techs to send it to you…We got some, I’ll have to get one of my techs to send it to you…
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
ORION CI
Doing Science: The Way It Is and Will Be
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
28 Mar 2006
ORION CI
Doing Science: Another Option
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
ORION CI
Interfacing with ORION
Working with the system will be at least as easy as it is now.
• Building an instrument?– Instrument interfaces are defined– Libraries and adapters are provided to help you.– Core instruments will be taken care of.
• Discovering data?– Lots of options, most of which actually work.– Just “get the data” (as you want it) once you find it.– Not just ORION data, data from lots of places.
• Running an observatory?– Access to system status.– Ability to coordinate the entire system as needed.
28 Mar 2006
ORION CI
A Science Scenario
• What we have here– An example for illustration– Only one of many applicable science scenarios
• The example– “Developing a complex instrument for ORION”– Actually a platform, but we’ll call it an instrument here– Covers most of the lifecycle of the instrument
• And furthermore…– Accessing and using data from the instrument
28 Mar 2006
ORION CI
Developing a Complex Instrument for ORION
• Platform carrying multiple sensors• Commandable• Not always connected• Multiple non-trivial data sets• Data transport varies
– may be stored and copied out of the instrument later– may be streamed in (near) real time
28 Mar 2006
ORION CI
Steps Along The Way
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
28 Mar 2006
ORION CI
Develop: Create Instrument
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
What does my instrument have
to do?
In any case, I have to get money (may involve research).
Finally, I’m building the instrument.
ORION just sent me instructions on testing
my instrument.
What does ORION say my instrument
must do?
Can I really justify building this? (May call for research.)
ORION IO may give me funds (e.g., for
core systems)!
I guess I’d better test it.
Darn, need to fix that./
Yay, it’s ready!
28 Mar 2006
ORION CI
Commission: Test and Validate Instrument
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Does system know about my
instrument?
Does my instrument talk to system correctly?
Put instrument in the water and
hook it up.
Make sure the instrument is “behaving.”
Make sure my instrument is
working correctly.Tell system about
my instrument.Record the transaction.
28 Mar 2006
ORION CI
Publish: Announce Availability of Instrument
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Declare what is the final version of my instrument
Group this with the other instruments
of similar products.
Tell everyone this instrument
is available.
Describe my instrument (using ORION forms).
Make sure this description is valid
and complete.
Enter my instrument into the
approved list.
28 Mar 2006
ORION CI
Discover: Find the Instrument
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
What kind of an instrument am I
looking for?
Son of a gun, I didn’t know they have a forbitz!
Let’s look over here, this list
looks relevant.
Describe the details I’m looking for.
What’s the best way to find it?
(How & where?)
OK, I’ll browse/ search using this
interface.
Start the search already.
Darn, need to try again./
Yay, found it!
28 Mar 2006
ORION CI
Acquire: Get OK to Command Instrument
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Please let me control the frobnitz
(private) instrument.
“Authorizing access,
please wait.”
“Instrument documentation is
in the mail.”
“Can user control this
thing?”
Never mind, I didn’t want to use that
instrument anyway.
Your instrument access list includes:
frobnitz ID#2365
Hi, Pat? Jan
here.
Of course.
Do you agree to call the provider now and
give credit later?
“Sure, if user calls me and credits me.”
28 Mar 2006
ORION CI
Use: Command Instrument
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
This is an instrument I can
command.
I’d better cite that scientist
like I agreed to.
Thank you for giving me credit on your paper.
“set sample rate on ‘instrument 23493’ =10
Hz for 3600 secs”
Changing sample rate to
10 Hz for 1 hour.
I’m getting the data back, I can start my
analysis now.
Do I need to do more?
OK, don’t need to command it
any more.
28 Mar 2006
ORION CI
Govern: Restrict Instrument Control
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Is it still OK to use the instrument
(so much)?
Define access rights and policies
for instrument
“Is it, or is it not, OK to use instrument
right now?
Is it OK for this user to use the
instrument?
28 Mar 2006
ORION CI
Publish: Announce Data Available
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Decide on the final format for data stream(s).
Group this data with the other
similar products.
Tell everyone this data product
is available.
Describe data stream (using
ORION forms).
Make sure this description is valid
and complete.
Enter my stream into the list of
available data.
28 Mar 2006
ORION CI
Discover: Find the Data
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
What kind of an data am I looking
for?
Son of a gun, I didn’t know they
have carbon data!
Let’s look over here, this data set relevant.
Describe the data sets I want
in detail.
What’s the best way to find it?
(How & where?)
OK, I’ll browse/ search using this
interface.
OK, it knows me and is letting me see the data set.
Start the search already.
Darn, need to try again./
Yay, found it!
28 Mar 2006
ORION CI
Acquire: Obtain the Data
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Please give me access to the
frobnitz data stream.Yes.“Authorizing
access, please wait.”
Oh look, the download button’s
enabled.
Can user have
data?
Do you accept this data is not quality
controlled?
Oh, I didn’t want the data if it isn’t
Quality Controlled.
Click on link, watch data stream in.
(done)Sure, if user
agrees to disclaimer
28 Mar 2006
ORION CI
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Use: Modify Data, Present ResultsThis is just data.
On we go.
I’d better cite that scientist
like I agreed to.
Thank you for giving me credit on your paper.
I’m getting data, my software’s running, I detected upwelling! I write my paper.
Can I turn this stream
off?
Yup, don’t need to access it any
more.
28 Mar 2006
ORION CI
For the individual
• Provide workflow and resource management tools to automate experimental design and execution
• Provide a secure work environment that automatically mediates conflicts
• Generalize and automate query and publish/subscribe processes for retrospective and real-time data
28 Mar 2006
ORION CI
• Dynamic Data Driven Assimilation System (DDDAS) use case
• Workflow binds diverse fixed and mobile instruments to assimilation model and data repository on shore
• Resource conflict mediation is automated
28 Mar 2006
ORION CI
• Distributed remote multi-mission laboratories distributed on an RCO
• Resource intensive, shared use operations require automated brokering
• Quality of service policy constraints are important
28 Mar 2006
ORION CI
For the community
• Facilitates collaborative experimentation and communication
• Automated integration of ORION policies and external obligations
• Ability to archive data with reliable discovery services for future use
28 Mar 2006
ORION CI
• Automated tracking and coordination of the state of observatory resources
• External resources can be integrated with observatory assets and operations
• Science use case becomes the observatory use case
28 Mar 2006
ORION CI
CI: Evolved from Grid Computing
• Grid Computing is a term for loosely coupled distributed computing across a diverse community of resources, owners
• Many Grid Computing efforts and products– Computational Grid (TeraGrid, Open Science Grid)
• Decomposition of large problems into many small atomic tasks– Data Grid (Storage Resource Broker, OGSA-DAI)
• Federation & cataloguing of distributed data repositories– Service Grid (On-Demand and Location-based service models—IBM,
Microsoft)• Generalization of computational and data grids as “Service” patterns• Emergence of Service Oriented Architectures
– Autonomic Grid (DMTF, WSDM from HP, IBM, CA, Oracle, Opsware)• Autonomous resource management, load balancing and fault
detection isolation and recovery applied to networks of coupled resources
• Grid computing contributes to overall cyberinfrastructure
28 Mar 2006
ORION CI
ORION CI Building on Other CI Projects
• GriPhyN, Atlas, Ligo, CMS – Data distribution and shared computational grid
• NVO (National Virtual Observatory) – Community data model, and shared data repositories and
applications
• BIRN (Biomedical Information Resource Network)– Federated data repositories of disparate data models into a
common meta-catalog– Resolution of disparate data models through mediation
• GEON (GEOsciences Network)– Extends data mediation model with ontologies
• TeleScience and NEES– Developed real-time control of remote instrumentation and the
coupling of remote physical and simulated systems
28 Mar 2006
ORION CI
Environmental Cyberinfrastructures like ORION
• What other projects are most like this?– LEAD (Linked Environments for Atmospheric Discovery)– NEON (National Ecological Observatory Network)
• What makes them similar?– Significant increase in the number and diversity of
instruments and data products – Real-time stream processing and inquiry– Interaction with the sensing environment– Semantic heterogeneity (mixed vocabularies)
28 Mar 2006
ORION CI
Viable Architecture
How can we be sure this is a viable, implementable architecture?
28 Mar 2006
ORION CI
Reasons for Confidence
• Open and transparent design process• Ongoing review by external and internal experts and
stakeholders • Iterative functional deployments reflecting use priorities
and system risks • A rapidly maturing field
– Major subsystems have multiple implementations in existence – Science efforts on which to build (BIRN, GEON)– Many sophisticated commercial collaboration architectures,
particularly in web domain (Flikr, eBay, MySpace)– Pioneering efforts in ocean cyberinfrastructure (LOOKING on
research side, NEPTUNE Canada in operation)
• Innovation will be focused on increased usability, effectiveness and reliability
28 Mar 2006
ORION CI
Concerns
• Some technical areas deserve extra attention– Incorporation of streaming data, a new data type, into
scientific process– Specification and adoption of instrument services– Effective operation of systems of systems that will
scale with the anticipated growth of ORION– Interfacing with external organizations and resources
• Implementing Organizations take this on– Evaluation and response to the most challenging tasks– Collaborative development encouraged to maximize
success
28 Mar 2006
ORION CI
Methodology & Timeline
• DoDAF model
• Development Process– Requirements– Operational activities– Organizational and operational entities
• Dictionary– Domain model– System components– WBS– Use domain experts for final costing
• Timeline– May 15, 2006
28 Mar 2006
ORION CI
Characterize the System
• Identify the main areas of Research Activity, Internal/External lines of Operations, and/or Support Functions.
• Describe the operations that are normally conducted in the course of achieving an experiment or research goal. – Description includes: Capabilities, operational activities,
input/output flows between activities, input/output flows from/to activities outside the scope of the architecture
• Characterize the command, governance, and business relationships among the human, organizations, and organizational type roles that are the key participants/stakeholders in the architecture.
• Depict the operational nodes, including organizations, with need lines between those nodes that indicate a need to exchange information.