Upload
cornelius-gordon
View
220
Download
1
Tags:
Embed Size (px)
Citation preview
Building a KEPLER Extension using Ptolemy II
KEPLER Collaboration Staff
Presented by: Ilkay Altintas and Efrat Jeager
@ San Diego Supercomputer Center, UCSD
2/20/2004 Building a KEPLER Extension Using Ptolemy II 2
Outline Overview of Ptolemy II and Kepler collaboration Ptolemy II
Ptolemy II Architecture Download, Make and Install The Vergil user interface and application Directors and Actors Ports and Parameters Manager
Kepler Overview of pre-built Kepler actors and demos Ingredients to build a Kepler extension Building you first actor
HelloWorld revisited…
2/20/2004 Building a KEPLER Extension Using Ptolemy II 3
The KEPLER Systemfor Scientific Workflows …
A framework for design, execution and deployment of scientific workflows
Caters specifically to the domain scientist Builds on Ptolemy II (next slide... :-) Application pull from various projects
http://kepler.ecoinformatics.org
2/20/2004 Building a KEPLER Extension Using Ptolemy II 4
… based on Ptolemy II
A set of Java packages for heterogeneous, concurrent modeling, design and execution.
Strengths include: Precisely defined models of computation and
component interaction e.g. Process Networks (PN) – data-flow oriented
An intuitive GUI that lets rapid workflow composition
A modular, reusable and extendable object-oriented environment
An XML based workflow definition – MoML Workflows defined in Ptolemy II MoML XML schema Easily exchangeable
2/20/2004 Building a KEPLER Extension Using Ptolemy II 5
Actor-Oriented Design
Reference: PtolemyII Design Document-1, Chapter 1
2/20/2004 Building a KEPLER Extension Using Ptolemy II 6
Focus on Actor-Oriented Design
Adapted from the *.ppt slides by Edward A. Lee (See References)
Object orientation: class name
data
methods
call return
What flows through an object is
sequential control
Actor orientation:actor name
data (state)
portsInput data
parameters
Output data
What flows through an object is
streams of data
2/20/2004 Building a KEPLER Extension Using Ptolemy II 7
Layered Software Architecture
Ptolemy II packages have carefully constructed dependencies and interfaces
PN
CSP
CT
DE FSM
SD
F
Kernel
Data
Actor Math
Graph
Adapted from the *.ppt slides by Edward A. Lee (See References)
2/20/2004 Building a KEPLER Extension Using Ptolemy II 8
Ptolemy II Architecture
Core packages (actor, data, kernel, math, util) The data model(--abstract syntax) of the models Abstract semantics
User Interface (UI) packages MoML and visual interface support
Library packages Domain polymorphic actors
Domain packages Implementations of different models of
computation
2/20/2004 Building a KEPLER Extension Using Ptolemy II 9
Overview of the Key Classes
2/20/2004 Building a KEPLER Extension Using Ptolemy II 10
Installing Ptolemy II
http://ptolemy.eecs.berkeley.edu/ptolemyII/
User version Version Ptolemy II 3.0.2 – Web Start Installer
Source download Precompiled (…downloaded before…) Source-only code – 4 steps to make
cd “PTII” rm –f config.* ./configure make fast install
2/20/2004 Building a KEPLER Extension Using Ptolemy II 11
Abstract Syntax of PTII Models
P ortP ort
A ctor A ctor
L inkR elation
A ctor
P ort
connection
connection conn
ectio
n
L inkLi
nk
A ttribu tes A ttribu tes
A ttribu tes
Abstract syntax choices:
• Hierarchy is tree structured (like XML).
• A relation mediates connections.
• Ports can link multiple relations and relations can link multiple ports.
• Ports mediate connections across levels of the hierarchy (no statecharts-style level-crossing links)
• …Abstract syntax defines the structure of a model, but says little about what it means.
•Hierarchical Entities, Ports, Connections and Attributes
Adapted from the *.ppt slides by Edward A. Lee (See References)
2/20/2004 Building a KEPLER Extension Using Ptolemy II 12
Models of Computation
Semantic interpretations of the abstract syntaxDifferent models <=> Different semantics <=> Different
execution
Are actors active? passive? reactive? Are communications timed? synchronized? buffered?
Models of Computation:
• continuous-time• dataflow• rendezvous• discrete events• synchronous• time-driven• publish/subscribe•…
process { … read(); …}
process { … write(); …}
channel
port port
receiver
One Class of Semantic Models: Producer / Consumer
Adapted from the *.ppt slides by Edward A. Lee (See References)
2/20/2004 Building a KEPLER Extension Using Ptolemy II 13
Director Governs the execution of a composite entity.
Scheduling, dispatching threads, generate code, etc. A composite entity is called opaque if it doesn’t have a
local director. An opaque composite entity inherits the director of its
container as its executive director.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 14
Vergil User Interface The graph editor Actor library Director library Utilities Ports Parameters Quick tour
2/20/2004 Building a KEPLER Extension Using Ptolemy II 15
Using Vergil
Executing a pre-built model from the graph editor
Building a model Choosing actors Making connections Choosing a director Building composite actors Saving and running the model
Start Exercise-0
2/20/2004 Building a KEPLER Extension Using Ptolemy II 16
MoML
Modeling Markup Language A primary persistent XML file format
for Ptolemy II.
ptolemy filename.xml ptexecute filename.xml vergil filename.xml moml configuration.xml filename.xml
2/20/2004 Building a KEPLER Extension Using Ptolemy II 17
An Example MoML File<?xml version="1.0" standalone="no"?><!DOCTYPE entity PUBLIC "-//UC Berkeley//DTD MoML
1//EN""http://ptolemy.eecs.berkeley.edu/xml/dtd/
MoML_1.dtd"><entity name="test"
class="ptolemy.actor.TypedCompositeActor"><property name="director"class="ptolemy.domains.sdf.kernel.SDFDirector"/><entity name="ramp"
class="ptolemy.actor.lib.Ramp"/><entity name="plot"
class="ptolemy.actor.lib.gui.SequencePlotter"/><relation name="r"
class="ptolemy.actor.TypedIORelation"/><link port="ramp.output" relation="r"/><link port="plot.input" relation="r"/></entity>
SDF domain
Top-level entity
Reference: PtolemyII Design Document-1, Chapter 6
2/20/2004 Building a KEPLER Extension Using Ptolemy II 18
Type System
2/20/2004 Building a KEPLER Extension Using Ptolemy II 19
Building Kepler Extensions
FOCUS: How to build actors? Exercises are under:
http://www.sdsc.edu/~altintas/KeplerTutorial/KeplerHandsOn.txt
Please complete the preparation for
the programming exercises…
10 minute break…
2/20/2004 Building a KEPLER Extension Using Ptolemy II 20
Ingredients We have: Java 1.4.2 Cygwin for Developers (for Windows users) Ant 1.5 or higherDownloaded: Compiled Ptolemy II source code
http://www.sdsc.edu/~altintas/KeplerTutorial/ptII3.0.2.tar.gz Kepler extensions (a version from CVS)
http://www.sdsc.edu/~altintas/KeplerTutorial/kepler.zip
Completed the preparation for the hands-on programming exercises.
Coding-style guidelines are at: http://users.sdsc.edu/~altintas/codingStyle.pdf Please use them!
2/20/2004 Building a KEPLER Extension Using Ptolemy II 21
Adding Actors Domain and/or data polymorphic actors Use object-oriented inheritance to avoid
code duplication 3 base classes: Source, Sink, Transformer
To use the actors in Vergil Add them to one of the actor libraries Most libraries are under $PTII/ptolemy/actor/lib Libraries are XML files
The basic structure of an actor: See
http://www.sdsc.edu/~altintas/KeplerTutorial/ActorStructure.txt
2/20/2004 Building a KEPLER Extension Using Ptolemy II 22
Actor Interfaces: Ports & Parameters
input portsoutput port
p1
p2
p3
parameters:
a1 = valuea2 = value
input/outputport
port
Example:
2/20/2004 Building a KEPLER Extension Using Ptolemy II 23
Anatomy of an Actor: Ports
Used for “message transport”, can be an input, an output, or both.
Key class: IOPortIOPort (Can be connected to other IOPort instances via IORelations.)
Use TypedIOPort TypedIOPort in order to benefit from the type system! (Domain specific: DEIOPortDEIOPort)
Receiver and Sender interfaces depending on the usage of the port.
Public members of the actors!!!
2/20/2004 Building a KEPLER Extension Using Ptolemy II 24
Ports: Introduction to the API
public TypedIOPort portName; //Definition//Create the portportName = new TypedIOPort(this, “portName”, true, false);portName.setMultiport(true); //Can support more than one linkint width = portName.getWidth(); //0 or 1 if single port//Reading and WritingportName.send(channelNumber, token);Token token = portName.get(channelnumber);//Setting the type of the portportName.setTypeEquals(BaseType…a type in the type
system…);portName.setTypeAtLeast(…must be a port or parameter…);
2/20/2004 Building a KEPLER Extension Using Ptolemy II 25
Anatomy of an Actor: Parameters
Public members of the actors! Similar API with ports…
public Parameter parameterName; //Definition//Creation and setting the initial value: 2-waysparameterName = new Parameter (this, “parameterName”,
new Token(…token value…));ORparameterName = new Parameter (this,
“parameterName”);parameterName.setExpression(…tokenValue…);
2/20/2004 Building a KEPLER Extension Using Ptolemy II 26
Anatomy of an Actor: Constructors
The major task: To create and configure ports and
parameters super(container, name);
Carries the NameDuplication and IllegalAction exceptions from the super class.
The icon for the actor can be set here.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 27
Exercise-1
Create an actor with An output port A parameter
Introduction to the fire method. Adding your actors into the actor
library Using your actors in a workflowBreak for 10 minutes when finished…
2/20/2004 Building a KEPLER Extension Using Ptolemy II 28
Execution of an actor
initialize()
Iterations
wrapUp()
prefire
fire()
postfire()
2/20/2004 Building a KEPLER Extension Using Ptolemy II 29
Action Methods
initialize()initialize(): Initialize the state variables of an actor.
prefireprefire(): Returns a boolean which indicates if the actor wants to fire. Can also be used to perform an operation
that will happen exactly once per iteration. firefire(): The main point of execution.
For reading inputs, producing outputs, read the current parameter values.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 30
Action Methods (cont.)
postfirepostfire(): Has two tasks: Updating persistent state Determining whether the execution of
an actor is complete. wrapUpwrapUp(): For displaying final results.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 31
Things to remember when implementing a fire() method
Use the methods of the Token class for arithmetic whenever possible (to get data polymorphism)
If data-polymorphism is not necessary, set the type to a base type then cast the token to that type.
Cannot assume that there is be data available at all the input ports (for domain-polymorphism)
Do not update the persistent state in fire() (use postfire())
2/20/2004 Building a KEPLER Extension Using Ptolemy II 32
Implementing Polymorphism
Class: PortParameterFunction A PortParameterFunction object will be returned as a
function of two objects. Set the type of the output equal to the type of this
object. Type system will compute the type of the
PortParameterFunction object and use it as the type of the output when necessary.
We’ll see this in the example 2!
2/20/2004 Building a KEPLER Extension Using Ptolemy II 33
Exercise-2
Getting data through an input port. Understanding the action methods. Designing a polymorphic actor. Using inner classes
Break for 10 minutes when finished…
2/20/2004 Building a KEPLER Extension Using Ptolemy II 34
The manager
Controls the overall execution of a model. Interacts only with the “top-level composite actor” startRun() -> run() -> execute() ExecutionListener interface provides the manager
with info on the events generated during execution.
Hand-outs... show how Ptolemy II handles mixing models of computations hierarchically.
--Using opaque composite actors!
2/20/2004 Building a KEPLER Extension Using Ptolemy II 35
Exceptions A uniform mechanism for reporting errors Base class: KernelException Exception chaining re-implemented since Java
versions < 1.4 doesn’t support it. The detail message includes the detail message from the
cause argument. A protected _setCause() method is implemented, but not
the public initCause() method that JDK1.4 has. Non-severe exceptions: IllegalActionException,
NameDuplicationException, NameDuplicationException. Severe-exceptions: KernelRuntimeException,
InvalidStateException, InternalErrorException.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 36
Exercises-3&4
Stopping a model using an actor Utilizing the manager functions Handling exceptions Using the postfire method Creating a customized icon for an
actor Using an existing code/application in
Kepler.Break for 10 minutes when finished…
2/20/2004 Building a KEPLER Extension Using Ptolemy II 37
Existing Kepler Actors Actor prototyping tool Generic Web and Grid Service RDBMS Connection and Querying Generic User Interface and
Transformation Biological Service and Data Access Rock Classification EML Data Ingestion GARP Native Species Pipeline (Using JNI
to utilize C++ code)
2/20/2004 Building a KEPLER Extension Using Ptolemy II 38
Actor Prototyping Tool
Allows “place-holder” actors to be defined on the fly by non-programmers during workflow creation
Scientists can thereby create workflows without programming knowledge
Workflows created with these actors can be executed once their functionality is implemented by a programmer
2/20/2004 Building a KEPLER Extension Using Ptolemy II 39
Distributed SWFs in KEPLER
Web and Grid Service plug-ins WSDL, GWSDL ProxyInit, GlobusGridJob, GridFTP, DataAccessWizard
WS Harvester Imports all the operations of a specific WS (or of all the WSs in a UDDI repository) as Kepler actors
WS-deployment interface (…ongoing work…) XSLT and XQuery transformers to link non-fitting
services together
2/20/2004 Building a KEPLER Extension Using Ptolemy II 40
A Generic Web Service Actor
Given a WSDL and the name of an operation of a web service, dynamically customizes itself to implement and execute that method.
Configure - select service operation
2/20/2004 Building a KEPLER Extension Using Ptolemy II 41
Set Parameters and Commit
Set parameters and commit
2/20/2004 Building a KEPLER Extension Using Ptolemy II 42
WS Actor after Instantiation
2/20/2004 Building a KEPLER Extension Using Ptolemy II 43
Web Service Harvester
• Imports the web services in a repository into the actor library.• Has the capability to search for web services based on a keyword.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 44
Composing 3rd-Party WSs
Output of previousweb service
User interaction &Transformations
Input of next web service
2/20/2004 Building a KEPLER Extension Using Ptolemy II 45
Current Grid Actors
2/20/2004 Building a KEPLER Extension Using Ptolemy II 46
Providing RDBMS Access Database connection.
Opening a database connection and passing it to all actors accessing this database.
Database Querying. A generic actor that queries a database and
provides its result. DBConnection type and
DBConnectionToken. A new IOPort type and a token to distinguish a
database connection from any general type.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 47
Database Connection OpenDBConnection actor:
Input: database connection information Output: A DBConnectionToken, a reference to a
database connection instance, through a DBConnection output port.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 48
Querying a Database DatabaseQuery actor:
Input: A query string (SQL) and a database connection reference.
Parameters: output type – XML, Record or String. Process: Execute query. Produce results according to
parameters.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 49
Using Bio-Services Actors: PIW
2/20/2004 Building a KEPLER Extension Using Ptolemy II 50
Sub-workflow: Gene Sequence Processing
Data transformations between the tasks
2/20/2004 Building a KEPLER Extension Using Ptolemy II 51
Sub-workflow: Transfac Detail
2/20/2004 Building a KEPLER Extension Using Ptolemy II 52
Classifying Igneous Rocks
2/20/2004 Building a KEPLER Extension Using Ptolemy II 53
Actors for Modeling The Classifier
Diagrams information and transitions between them.
Extracted from the mineral composition and this level’s diagram coordinates.
SVG to polygons.
Classifier: Locates the point’s region.
Finer granularity
Displays the point in the diagram for this level.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 54
EML Data Ingestion Actor
Ingests any data format described by EML metadata
Converts raw data to Ptolemy format
Data can then be operated on with other actors
2/20/2004 Building a KEPLER Extension Using Ptolemy II 55
Using EML ingestion actor
2/20/2004 Building a KEPLER Extension Using Ptolemy II 56
Third-Party Local Applications
2/20/2004 Building a KEPLER Extension Using Ptolemy II 57
What did we learn? Basic concepts of Ptolemy II Creating actors within the Ptolemy II framework The existing Kepler extensionsWhat we didn’t learn: Extending the type system
Need to define a class for the new type and add it to the type lattice
Extending the user interface Effigy, Tableaux (Similar to Model-View-Controller architecture)
Creating our own director Developing a director is harder!
Need to inherit the Director class, and implement the semantics of how the actors will behave. (Semantics==Model of computation)
2/20/2004 Building a KEPLER Extension Using Ptolemy II 58
Ptolemy II Design Document
Volume 1:User-Oriented
Volume 2:Developer-Oriented
Volume 3:Researcher-OrientedAdapted from the *.ppt slides by Edward A. Lee (See References)
2/20/2004 Building a KEPLER Extension Using Ptolemy II 59
Becoming a Kepler member?
http://kepler.ecoinformatics.org/members.html
Email me:
[email protected] get a read-only account.
Will upgrade to become a contributing member depending in time.
2/20/2004 Building a KEPLER Extension Using Ptolemy II 60
References Ptolemy II Design Documentation and Source Code http://ptolemy.eecs.berkeley.edu/papers/03/ptIIDesignIntro/ Kepler website: http://kepler.ecoinformatics.org Overview of Ptolemy II (Powerpoint presentation), Edward A.
Lee, October 9, 2003, presented in course EE249 - Design of Embedded Systems: Models, Validation and Synthesis.
An Overview of the Ptolemy Project a nd Actor-Oriented Design (powerpoint presentation), Edward A. Lee, February 4, 2004, OMG Technical Meeting, Anaheim, CA.
Presentations from the Ptolemy website: http://ptolemy.eecs.berkeley.edu/ptolemyII/
Kepler documentation, javadocs and this tutorial will be made available at the website soon.