Embed Size (px)
Citation preview
Michael SobolewskiTexas Tech [email protected]
Grid and Service-Oriented Computing:The Intergrid Perspective
Part II
Presentation Agenda – Part II• Grid Federated S2S Environment • GISO Programming• Monitoring Execution of SO Programs• ServiceUI• Provisioning and Autonomic Computing• Design Issues – UML Class Diagrams• Code Mobility in SORCER• File Store Service• SGrid and Intergrid• Surrogate Architecture for Mobility• Web/Grid Services in SORCER• DB Context Providers• A Grid Killer Application• SORCER Research Projects• Summary
Applying OO Techniques
• Service activity is a special object of type: Exertion
• Exertions are executed by network objects/service providers of type: Servicer
• Service providers form P2P environment• Service is requested by calling the method: service(Exertion)
• Service providers are identified by a type with methods:
public ServiceContext selector(ServiceContext)
public interface Exertion {// Apply this exertion method to the specified contextpublic Exertion exert() throws RemoteException, ExertionException;
…}
Exertion Interface
• All service activities implement this interface:
public interface Servicer {// Put into action the specified exertion
public Exertion service(Exertion exertion) throws RemoteException, ExertionException;
// Monitoring methods…
}
Service Peer Interface: Servicer
• All services implement this interface:
Contexts and Context Methods
Job Task Service Context Service Method
…CC
TC
TC – Task Context, CC – Control Context
ServiceMethodScriptMethodXYZMethod
DAS
GEOMMAT
Shank
MAT_RENE5
Mat_Rene5.dat
PP
STRESS
Post_Stress.dat
LC
Gas_Loads
ExtGLoads.dat
BC
Cyclic
BCCyclic.dat
DiskAirfoil
UIF
Airfoil.uif
Mesh
Disk_PRT
AS_PRT
AS_Shank.dat
AS_Shank.prt
AS_PRT
Disk.prt
Mesh
Stress_Tet
Stress_Tet
AS_Shank.datAS_Shank.dat
AS_Shank.datModal_Hex
CC
ContextMethod attributes: service type, selector, group, provider name, method typeMethod type: preprocess, process, postprocess, append
Service BindingJob as a Megaapplication
Job
Task
Context
Method
Method type: preprocess, process, postprocess, append
Federation Member
Federation of Services
Job
Task
Context
Method
Method type: preprocess, process, postprocess, append
Job as Runtime Environment
GISO Programming
Domain ServicesDomain Services
Service-Oriented,Service-Oriented,Grid-ProgrammingGrid-Programming
andandDevelopment ToolsDevelopment Tools
Service-Oriented ProgramsService-Oriented Programs
Generic End-User ClientGeneric End-User Client
Service-Oriented Infrastructure Services Service-Oriented Infrastructure Services (SORCER.core)(SORCER.core)
Grid Interactive Service-Oriented Programming
Job Structure
Airfoil
Platform
Shank
Dovetail
FiperJob
FiperContext
FiperTask
FiperMethod
FiperContext
Exertion
FiperMethod
FiperContext
FiperTask
FiperMethod
FiperContext
Exertion
FiperMethod
.
.
…
ControlContext
FiperMethod
Turbine Blade Solid Geometry
Mechnical Analysis of TB
Turbine Blade FEM
DiscretizeSolid Geometry Into FEM(ICEM, Mesh)
in
out
DescretizationStrategy
Turbine Blade FEM w/ BCs
Apply Boundary Conditions to FEM(SIESTA, ApplyBCs)
in
out
Boundary Condition (BC) Model
Turbine Blade FEM w/ BCs & Materials
Apply Materials to FEM(SIESTA, ApplyMaterails)
in
out
Materials Model
Turbine Blade FEM Element &Nodal Stresses
Solve FEM for Mechanical Stresses(ANSYS, Mechanical Analysis)
in
out
Mechanical Solution Strategy
AutoShank Solid Geometry Part
Generate Solid Shank (AutoShank, Autoshank)
in
out
AutoShank Seed Geometry Part,AutoShank Parameters
in
in
in
in
Turbine Blade FEM
DiscretizeSolid Geometry Into FEM(ICEM, Mesh)
in
out
DescretizationStrategy
Turbine Blade FEM
DiscretizeSolid Geometry Into FEM(ICEM, Mesh)
in
out
DescretizationStrategy
Turbine Blade FEM w/ BCs
Apply Boundary Conditions to FEM(SIESTA, ApplyBCs)
in
out
Boundary Condition (BC) Model
Turbine Blade FEM w/ BCs
Apply Boundary Conditions to FEM(SIESTA, ApplyBCs)
in
out
Boundary Condition (BC) Model
Turbine Blade FEM w/ BCs & Materials
Apply Materials to FEM(SIESTA, ApplyMaterails)
in
out
Materials Model
Turbine Blade FEM w/ BCs & Materials
Apply Materials to FEM(SIESTA, ApplyMaterails)
in
out
Materials Model
Turbine Blade FEM Element &Nodal Stresses
Solve FEM for Mechanical Stresses(ANSYS, Mechanical Analysis)
in
out
Mechanical Solution Strategy
Turbine Blade FEM Element &Nodal Stresses
Solve FEM for Mechanical Stresses(ANSYS, Mechanical Analysis)
in
out
Mechanical Solution Strategy
AutoShank Solid Geometry Part
Generate Solid Shank (AutoShank, Autoshank)
in
out
AutoShank Seed Geometry Part,AutoShank Parameters
AutoShank Solid Geometry Part
Generate Solid Shank (AutoShank, Autoshank)
in
out
AutoShank Seed Geometry Part,AutoShank Parameters
inin
inin
inin
inin
Actions Data
Process Turbine Analysis
Web Based User Agent
}FiperContextFIPERTaskFIPERJob
Context EditorContext NameDomain Subdomain
Context Node
Data Node
Task Editor
Task Editor
FiperMethod Definition
Context forTask
Task Name
Created Task
FiperMethod
FiperContext
Task Name
Job Browser
Job Editor OperationsDomain Subdomain Job Name
JobTasks
Job EditorFeatures
Create a new Job
and add to Job
Create a new Task and add to Job
{
Task EditorFeatures
Browse & Add Job to Job Browse & Add Task to Job
Control Context
Job Context
Job Task 0
Context0
Task1
Context 1
Drag and Drop shows that thisInput will come from task0, AutoShank Output Node
Adding Context to Task
Selecting a Task for a Job
Monitoring Conceptual View
Monitorable Interface
All ServiceProviders are Monitorable
to monitor, control execution and debug SO programs
Execution States of Exertions
DeletedInitial
Error
Running
Done FailedSuspended
run stop
suspend
resumestep
resumestep
• ServiceProvider – Exertion Status Map (ESM)
• Jobber – Dispatcher Thread Map (DTM)
• Exception/Errors returned in ServiceContexts
Monitoring Framework
•State of running SO program updated via DB•Clients can monitor the SO program via web portal•Clients can stop/resume/suspend•Monitoring requests are handled by Service Broker
WebPortal
Client1
Client2 Persister
Jobber
p1 p1 p1
mon
itor(
rex)
service(ex)service(job)
service(job)
monitor(rJob)
DB
monitor(r
Job)
Job Browser
Control Panel
Launcher
Job List
Exertion List
Control Context
Review = Suspend
Job Monitor
Job Monitor
Full Job Edit Capability
Job ExecutionControls
MO - Complete Framework
Monitorable ServiceBrokerMonitorable
Monitorable
register(RemoteEventListenerduration,timeout SOP) : RuntimeExertion
1
execute ( sop-x)
update(..)stop(..)error(..)
3
4For (every sop-x) x=0,1,2,… Init (cookie, monitorable, duration, timeout)
2
session sessionsession
Monitor ServiceUI
ServiceRequestor
SOPFederation
session Persistent session
session Transient session
SOP = { sop1, sop2, …..}Or = sop-x , x=1,2,……
MonitorManager
SessionManager
session
UIManager
notify (Monitor Event)
5
Result or Failure Notification
Monitorable ServiceBrokerMonitorable
Monitorableexecute ( sop-x)
update(..)stop(..)
notify (Monitor Event)
7
8
Service Provider
9
MonitorManager
session sessionsessionsession
Suspend/stop/resume a Job
MonitorableServicer
suspend/stop/resume job(……)
MonitorableServicer
MonitorableServicerMonitorable
Servicer
suspend/stop/resume job(……)
Service Broker
notify (Monitor Event)
1
2
3Monitorable
Servicer
MonitorableServicer
MonitorManager
•register (leaseDuration, job,
listener)
MonitorSession•init (monitorable, duration,
timeout)•getState()•update(context)•done(context)•failed(context)•getLease()
invokes
MonitorableServicer
•stop(Ueid)•suspend(Ueid)•resume(Ueid)•step(Ueid)
Service Provider
Jobber
MonitorableDispatcher
Servicer.
Registers and gets backRuntime Job
xxxServiceProvider
DispatchesExertion withSession Objects
IssuesMonitorablecalls
Remote EventListener
MonitorableProvider
SessionManager
•init(cookie, monitorable, duration, timeout)•update(cookie, context)•done(cookie, context)•failed(cookie, context)
MonitorUIManager
•getSOPInfo(Principal)•getSOP(referenceID,
Principal)
ServiceJob
ServiceTask
1
0..*
Dispatches exertions in job
Providers use sessionto update informationabout runtime exertions
1
1
11
Monitoring Framework
Service UI
A "direct-use" client talks to a service through its object interface
Service UI
A user interacts with a service via a UI object
EntryServiceTask
Context/Method/Task/Job
RemoteServiceMethod
ServiceContext
ServiceMethod
ArithmeticMethod
ServiceTask
0..*
Hasdata
Entry
EntryServiceJob
ServiceJob
ClientSite
ServerSite
1..*
Defines behavior
Defines remotebehavior
Exertion
Serializable
EntryExertion
Remote1..*
EntryServiceTask
RemoteServiceMethod
Task Execution
SorcerJoiner
ServiceContext
ArithmeticMethod
ServiceProvider
Servicer
ArithmeticProviderImpl Publishes
Proxy
ServiceTask
Executes
UnicastRemoteObject
0..* Hasdata
ArithmeticRequestor
RequestorRunner
Provider ArithmeticInterface
ProviderWorkerJavaSpace
Uses
Runs
Invokes
ArithmeticRemote
Remote
RequestorDrops task
Submits task
Submits task
Job Execution
Uses
RequestorRunner
<<HTTP>>
SorcerLauncher
Cataloger
ServiceCatalogImpl
ProxyProtocol
JobberImpl DisapatcherFactory
TaskDispatcherFactory
Creates
Invokes
ServiceTaskDispatcher
CatalogTaskDispatcher
TaskDispatcher
Uses
SpaceTaskDispatcher
ServiceServlet Jobber
Provider Requestor
ServletProtocol
ArithmeticRequestor
Service access: Direct, Catalog, Space
ProviderAccessor
Uses
JobBrowser JavaSpace
Submitsjob
Drops job
ServiceProvider
SORCER Smart ProxyBL on Client and Server
UnicastRemoteObject
ProviderProviderWorker
JavaSpace
Uses
Uses
ServiceBean
ServiceProvider
ServiceBean
Adapter
ProviderDelegate
Delegates to
ServiceProxy
Remote
ArithmeticProxy
ServiceJoiner
ArithmeticBean
Exports Provider
ArithmeticService
Service
ArithmeticProvider
Provisioner
Deploys by QoS
ArithmeticRemote
Remote
ArithmeticInterface
Servicer
Publishes
provider’s proxy
Uses
Mobility
• Mobility stems from desire to move either toward resources or away from scarcity
• Mobility of computing entities– Physical mobility
• Computers– Logical mobility
• Running user application (active process)– Migrate within a local cluster of computers– Enable load distribution and fault resilience
• A mobile agent– Migrate anywhere on the Internet– Act on the user behalf to pursue specific goals
• Mobile code– Applets– MIDlets– RMI/Jini/Rio/SORCER
Mobile Agents
• A mobile agent – a network application that migrates in a network (grid) – Active– Autonomous– Goal-driven– Acting on behalf of a user
• Stationary agents– Intelligent agents– Multi-agent systems
Agents and Mobile Code
• Mobile code solves problems– Heterogeneity (platform independence)– Dynamic class loading– Security and safety– Multi threading support– Object serialization– Reflection– Universal availability of JVM– A large number of docking stations
Mobile Code
1. Proxies
2. Exertions
3. Task Methods
4. Agents
5. SORCER Beans (JSBs)
6. Service UIs
SORCER Code Mobility has many forms
Agents in SORCER
SelfAgent
ServiceTask ServiceJob
ServiceContext
Uses
EntryTask EntryJob
RemoteAgent
Remote
ObjectLoggerStores
Restores
1..*
1..*
EntryExertion
AgentBootstrapBootstraps
JDK1.1 surrogateLimited objects
Agents
Dropper
Provider
doTaskstopTaskdoJobstopJobsuspendJobresumeJobstepJob
Servicer
service
Uses
Uses
ServiceProvider
Jobber
Entry
Requestor
getTaskgetJob
RequestorRunner
Uses
Uses
Uses
ServiceMethod
EntryMethod
Uses
Uses
1..*1..*
Uses
Agent
act
Exertionpreprocessprocesspostprocess
Self
think
SORCER ProvidersJDK 1.4 objects
TaskAgent JobAgent1..*
Self-Aware Grid
• A grid federation knows what is doing
• Learn from experience and adapt to surprises
• Is aware of its behavior and explain itself
• Is able to anticipate different scenarios and predict and plan for novel futures
• It would learn, not crash, when faced with a new situation
• Self-testing, self-debugging, and self-explaining within a federation
What is Rio?
• A Dynamic Service Delivery architecture based on the capability to provision services through the network using Qualitative and Quantitative QoS attributes
• Addresses essential issues for the development of a dynamic self healing services environment
RIO & Autonomous Computing
Dynamic, Distributed systems- Capability to adapt to unforeseen changes on the network- Driven by increasing componentization, distribution &
heterogeneity- Scale to orders of magnitude
Dynamic Policy driven systems- Injecting rules & policies into the service fabric allowing
greater automation, scalability and controlled behavior
Telemetry- Being able to monitor, meter, gauge and observe stimulus
through the system- Not just machine based, but network-wide
Rio Architecture Overview
Development & AssemblyDevelopment & Assembly
Dynamic ProvisioningDynamic Provisioning
SubstratesSubstrates
Jini Service BeansJini Service Beans
ToolsTools
FederationFederation
ResourcesResources
JavaTM 2, JiniTMJavaTM 2, JiniTM
DynamicContainer
AssimilationIntegration &
Interoperability
AssimilationIntegration &
Interoperability
Jini Service Bean Basics
• Jini Service Beans (JSBs) are the fundamental domain specific computational entities on the network
• JSBs are Java objects
• JSBs can encapsulate access to NDI (legacy) components
• Provides an easy to use programming model while maintaining access to low-level APIs
• Are provisionable based on their QoS attribute
Cybernode
• Jini Service Beans are instantiated by Cybernodes– Cybernodes run on computational resources– Cybernodes can contain multiple service beans
• Address lifecycle issues, including development, deployment, and runtime
• Provides the basic infrastructure to load, instantiate and destroy Jini Service Beans
CybernodeCybernode
Cybernode
Jini Jini Lookup Lookup ServiceService
Code Code ServerServer
Service Bean Attributes
Service Bean Attributes
Download JSB resources
Instantiate JSBs
JSBs register with LUS
Platform Attributes
Platform Attributes
Operational String
• Aggregated collection of application and/or infrastructure software assets that when put together provide a specific service on the network
• An Operational String is an object graph composed of objects that provide context on how to provision and instantiate services
• Can be created by referencing an LDAP repository, serialized objects, or from structuredXML documents (there can be many other sources as well)
Operational String
ServiceElement
ServiceProvisionManagement
ServiceBeanAttributes1
1
0..*
Quality of Service
• Approach– Software components need to run on most
appropriate compute resource based on definable criteria
– Compute resources have capabilities• CPU, Disk, Connectivity, Bandwidth, …
• Rio provides an extensible model allowing the declarative association of QoS
Provision Manager
• Provide a provisioning model to dynamically deploy, monitor & manage service components as described in an Operational String
• Implement a Fault Direction and Recovery strategy for service components
• Provide pluggablepluggable load distribution and Resource Cost analysis mechanisms to effectively take use of resources on the network
Dynamic Provisioning
Jini Lookup Service
discover
Start a monitor for each service in an Operational String
notify
ProvisionManager
Discover/join
Register/update for provision notifications with QoS attribute
Load an Operational String from disk or via remote method invocation
Notify ServiceInstantiation resources with a ServiceProvisionEvent based on the assignability of the JSBs QoS to the ServiceInstantiation resources QoS
Code Server
As needed download JSB class files
Instantiate & initialize JSB
ServiceBean
Instantiator
Service Instantiator
Fire notification events to event registrants for added, removed, updated Operational String actions as well as failed Provision attempts
Lincoln
• Provides the capability to enable dynamic discovery of Jini Lookup Services across networks that are out of multicast range, or do not forward multicast packets
JLS JLSLincoln Lincoln
Multicast Announcement
Group BGroup A
svca svcb
Multicast Announcement
RIO Provisioner
Lincoln Example
• Coast to coast discovery
• No extra service configuration
• Key to collaboration
Fullerton, CA
Minneapolis, MNTewksbury, MA
Portsmouth, RI
Developer Facilities
• Events– Built-in registration
and lease handling of user defined events
– Simple semantic for specifying and discovering events
• Resource Pools– Threads– Objects– Database
Connections
• Watches– Every component is
‘watchable’– Add data points to a
collectible set and graph accumulated results
Service Beans Service Beans
• Jini™ technology-based Service Beans (“JSBs”) are the fundamental domain specific computational entities on the network
• Are provisionable based on their QoS attribute
• Jini technology-based Service Beans are instantiated by Cybernodes– Cybernodes run on computational resources– Cybernodes can contain multiple service
beans
Summary
Provider Bootstrapping
Bootstrapping Type Server Type NDS Technology
java SorcerJoiner
-sProviderClass
RMI server
(JRMP/IIOP)
JNDI/RMI Reg
JNDI/LDAP
RMI/CORBA
java SorcerJoiner
-pProviderClass
Service provider
(Jini)
LUS Jini
java SorcerJoiner
-pProviderClass:ProxyClass
Service provider
with smart proxy
(Jini)
LUS Jini
Provisioning (Rio) JSB LUS Rio/Jini
Command Design Pattern
File System
File SystemRDBMS
API Classes
Utility Classes
AC
LM
anag
er
JDB
C
Cmd 1
Cmd 2
Cmd 3
Cmd 4
Cmd 5
Cm
d M
anag
er
File
Sto
reP
rovi
der
RemoteInputStreamServer
RemoteOutputStreamServer
1
2
34
5
6
2
1 34
56
1,1 - Requestor requests FSS provider for file upload/download based on DocumentDescriptor
2,2 - FSS forwards it through the command execution engine
3,3 - Database operations are done
4,4 - FSS spawns a RemoteOutputStream/RemoteInputStream server based on-if client wants to upload/download and returns back the InputStram/OutputStream adapter inside DocumentDescriptor
5,6 - Client communicates to server via OutputStreamProxy and uploads file.
5,6 - Client communicates to server via InputStreamAdapter and downloads file.
Standalone
JVMClients
SORCERDistributed File System
File System (UNIX)
Servlet EngineFileStorer
SORCER Service
FileStorerSORCER Service
File System (Linux)
FileStorerSORCER Service
File System (Win 2003)
Browser
JVM
Program
JVM
FileStore Clients Interactions
Client request is passed on from one server to another until one of the File Store service has the file actually present in its File System and the handle of file is passed onto client.
DBMS
ORACLE
DBMS
mySQL DBMS
MS SQL
Document Descriptor
DocumentDescriptordocumentIDdocumentNamefolderPathoutputStreaminputStream
InputStreamAdapter
InputStream
Remote
RemoteInputStreamServer RemoteOutputStreamServerFileStorer
getInputDescriptor(DocumentDescriptor)getOutputDescriptor(DocumentDescriptor)
OutputStreamProxy
OutputStream
contains
containscontains
contains
Document Descriptor: Details
DocumentDescriptor
setDocumentID(String)setFolderPath(String)setDocumentName(Name)setPrincipal(GAppPricipal)
FileStorer
getInputDescriptor()getOutputDescriptor()
InputStreamAdapter
read( ): intread( byte[] ): intread( byte[], int, int): int
InputStream
read( ): intread( byte[] ): intread( byte[], int, int): int
write( int )write( byte[] )write( byte[], int, int)
RemoteOutputStream
Remote
write( int )write( byte[] )write( byte[], int, int)
OutputStreamProxy
OutputStream
write( int )write( byte[] )write( byte[], int, int)
read( int ): byte [ ]
RemoteInputStream
containscontains
containscontains
ApplicationServlet
UploadServlet
RemoteInputStreamServer
RemoteOutputStreamServer
GAppUses
Uses
Uses
Servlet Engine
Servlet Engine FS Provider
Upload
Download
HTTP
HTTP
JRMP
JRMP
vs. FileStore Provider
...
CRD Internal Network
GRC HTTP Proxy Server
Web Browser
FIPER Intraportal
1 2
CRD Perimeter Network Controlled by Firewall
3 4 GRC SSL FIPER
ExtraPortal
Web Browser
FIPER Intraportal
7 8
Parker Perimeter Network controlled by Firewall
Parker HTTPProxy Server5 6 Parker SSL
FIPER
Extraportal
GRC Firewall
Parker Internal Network
Parker Firewall
B2B Setup
Nozzle Combustor CAD/IO B2B
IKS
4. Perform modal analysis
2. Request for nozzle validation
1. Update combustor PCS
3. Check for nozzle insertion
5. Perform CFD blow analysis
(UG) (ProE)
(Blow Analysis)
Vertical iGrid Grids
Cybernodes – iGrid.meshCybernodes – iGrid.mesh
Service Providers – iGrid.gridService Providers – iGrid.grid
Exertions – iGrid.spaceExertions – iGrid.space
Computing Devices – iGrid.netComputing Devices – iGrid.net
SORCER.coreSORCER.core
SORCER.gridSORCER.grid
iGrid.grid – service providers including services from technology (horizontal) gridsSORCER.core – SORCER infrastructure service providersSORCER.grid – SORCER domain specific service providers
SS Beans – iGrid.fieldSS Beans – iGrid.field
iGrid Layered Architecture
Domain iGrid ServicesDomain iGrid Services
Service-Oriented,Service-Oriented,Grid-ProgrammingGrid-Programming
andandDevelopment ToolsDevelopment Tools
Service-Oriented ProgramsService-Oriented ProgramsiGridApplication
Generic End-User Client/ServiceUIGeneric End-User Client/ServiceUI
Technology-Oriented (Horizontal) GridsTechnology-Oriented (Horizontal) Grids
iGridMiddleware
Service-OrientedService-OrientedComputing Computing
EnviRonment (SORCER.core)EnviRonment (SORCER.core)
Horizontal iGrid GridsWrapper + Native + SORCER
JXTAOGSA
SORCER.ogsa
SORCER.grid
Native ProviderTechnology SORCER wrapper-grids: Jini, JXTA, CORBA, Web Services, Grid Services, .NETSORCER.gridTechnology native grids
Service RequestorService BrokerSORCER Service ProviderSORCER Wrapper Provider
SORCER.jxta
SORCER.grid
SORCER.core
SGrid Dispatcher UI
• Choose the Application to run (For example Proth)
• Specify the Job Size for the jobs
• Set the Arguments, Attributes and Executables for the application
Arguments
• Specify Arguments, Input Files, Output Files for the Application
• Can be added above or below the selected option
• Can be reordered according to user’s requirement (Up, Down, Delete buttons)
• Specify which caller to use for execution, depending upon its
- Location (HPCC, SORCER)- Host Name (129.118.56.61)- Operating System- Node Name (Amber)
• ‘*’ is the default option which indicates that the application can be run by any caller in the Federation
Executable Attributes
Executables - Windows
• Specify Windows Executables and Library Files
• The files can be dynamically downloaded from File Store
Grid Provider
• A Grid Provider is responsible for SGrid Publishing ServiceUI Accepting user inputs and creating a Job Submitting the Job to Jobber Receiving results, notifying client, getting
inputs and storing results in the File Store
• Advantages of such a design• User friendly with respect to the Job Editor UI• Zero install with a Jini service browser and a
web-based File Store user agent
Caller Provider
• A Caller receives a caller service context that abstracts any system call across different platforms (Solaris/Linux/Windows)
• Callers are capable of Downloading platform specific executable
binaries and libraries Downloading source and compile it on-the-fly Make system calls with arguments specified in
the service context
• Callers download/upload files from/to the SORCER File Store Provider
Caller Service Context
String
src/bin/bytecode/callx
array attribute<attribute name>[ ]
URI
caller
call
cmd dir
envp[ ]in[ ] out[ ]
program
compiler
win/linux/unix
src[ ] bin[ ]
exec type
x
lib[ ]
parameters
arg[ ]load lib[ ]
isOveritable
SORCER URIs• A generic way to represent any resource in SORCER.grid
URLssorcer://<authority>/<prv interface>/<prv name>?<query>
- authority is a network host- query is a list of additional provider’s published attributes
URNssorcer:<prv interface>/<prv name>?<query>sorcer:serviceID?<query>
SORCER File Store URNs are in the format:sorcer:sorcer.core.FileStorer/<FS name>?folder=<folder name>&file=<file-name>orsorcer:FileStorer?folder=<folder name>&file=<file-name>orsorcer:FileStorer/HPCC?folder=<folder name>&file=<file-name>
- with no interface qualification used it is assumed that the interface is from sorcer.core.*
Proth’s Theorem
Proth's Theorem (1878):
Let N = h.2k+1 with 2k > h.
If there is an integer a such that
a (N-1)/2 = -1 (mod N),
then N is prime.
GridDispatcher
Jobber
SORCER Proth Interactions
Download ServiceUI
Upload inputs
Submit Proth Job
SORCER.space
File StoreProvider
Caller
Caller
File StorePortal
Caller
Caller
Caller
Caller
SORCER.grid
Download outputs
Upload inputs
Get results
Jini Service Browser – download Proth ServiceUIMail Reader – get notifications
Web Browser – download/upload filesMail Reader – get notifications
Submit JobGet results
Get inputsStore outputs
Running Proth
FSS Web User Agent
Distributed BLAST in SORCERIntroduction
• BLAST (Basic Local Alignment Search Tool) is a sophisticated software package used in Bioinformatics and Sequence Analysis for rapid searching of nucleotide and protein databases
• BLAST is powerful and has been optimized for performance; however the public sequence databases are big and growing rapidly . Thus, it takes a huge amount of computer resources to search sequence queries in a large database
• As a solution to this problem we provide an implementation of BLAST in a distributed environment using SORCER
Goals
• To increase performance by making the BLAST Process Distributed and sharing idle CPU and other resources
• Focus on high throughput of large numbers of submissions instead of high performance on any single job
• Provide a working system that is easy to Install and Use
• Heterogeneous network- provide a portable system that can be installed on different Compute devices
• The Business Logic is separate from the whole framework. All Libraries and Business Logic Code is Downloaded.
• Self-healing environment, can overcome crashes i.e. fault detection and recovery
• Enable large batch BLAST processes distributed over regular WAN or LAN
BlastProvider
Jobber
S-BLAST Interactions
1 - Download S-BLAST service UI2 - Submit S-BLAST data3 - Read Input files4 - Submit job5 - Write tasks6 - Read task7 - Write task8 - Read tasks9 - Return job10 - Store results11 - Download outputs
Upload inputs
SORCER.space
File StoreProvider
Tasker
Tasker
File StorePortal
Tasker
TaskerTasker
SORCER.grid
Download outputs
Upload inputs
Jini Service Browser – download S-BLAST service UIMail Reader – get notifications
Web Browser – download/upload filesMail Reader – get notifications
Jobber
Jobber
BlastProvider
4
9
12
8
5 7’
6’
6’’
11
7’’’
7’’
6’’’6’’’’7’’’’
6’’’’’
7’’’’’
10
3
Accessing SORCER.grid
SORCER.grid
LUS
Registers services using service proxy
Gateway
SORCER service
Service proxy
Service oriented program
Jobber
Mobileclient 1
Request service
Downloads service proxy
3
Transfers results
6
Discover service
2
4Gets service
5Renders service
Mobile Devices Support
HotSpot CVM KVM Card VM
Java 2Enterprise
EditionJava 2
StandardEdition
J2ME CDC
Handheld
ProfileMID
Profile
SmartCard
ProfileJ2ME CLDC
TVProfile
FoundationProfile
RMIProfile
TVProfile
PersonalProfile
TVProfile
CarProfile
Auto ProfileAuto
Profile
Vertical Grids of iGrid
Cybernodes – iGrid.meshCybernodes – iGrid.mesh
Service Providers – iGrid.gridService Providers – iGrid.grid
Exertions – iGrid.spaceExertions – iGrid.space
Computing Devices – iGrid.netComputing Devices – iGrid.net
SORCER.core
SORCER.grid
iGrid.grid – service providers including services from technology (horizontal) grids
SORCER.core – SORCER infrastructure service providers
SORCER.grid – SORCER domain specific service providers
xmlExertions–iGrid.xmlxmlExertions–iGrid.xml
Application Servers – iGrid.srvApplication Servers – iGrid.srv
Web Services – iGrid.wsWeb Services – iGrid.ws
Surrogate Services
Jini Capable Machine
Surrogate Host
HTTP/HTTPS
Private Protocol
Exported ServiceOther
Exported Service
Inter-connect Specific Code
SORCER.grid
SORCERservice
Jobber
Service Oriented Program
Deploy Calendar Service
Request
Request
Dep
loy
Cal
enda
r Ser
vice
Deployed SORCER SUROGATE Service
Mr. X
Get me C
alendar of M
r. X
Service UI
Request
SORCER Calendar Service Created
SORCER.grid
Surrogate Client
Provide Service
Mr X service
Interaction Using Private Protocol
Web Service to SORCER
UDDIUDDI
SORCER-Web Service GatewaySORCER-Web Service Gateway
JobberExertion Space
Exertion Space
Service CatalogerService Cataloger
Application ServersiGrid.srv
1
2
3’
3’’’
3’’ SORCER.grid
SORCER to Web Service
SORCER – Web Service GatewaySORCER – Web Service Gateway
JobberExertion Space
Exertion Space
Service CatalogerService Cataloger
Application ServersiGrid.srv
UDDIUDDI
2
1
3
5
SORCER.grid
4
Web Service
SO Context Providers
Miner
EDSEDS
EDS
CDS
SORCER.grid FSS
EDS
EDS
CDS
ReconciledContext
ReconciledContext
MAPM
AP
EDS – Elementary Data Store Provider, CDS – Compound Data Store Provider
Durability of Competitive Advantage
Improve engineeringproductivity 10%-20%
Reduce product development
cycle time by 30%
Create a“Category Killer”
Product
1
2
3
4
5
Years
Innovate or Die
Electric Power
Technology Evolution
Mass Adoption
Lab
Public Recognition
Early Adopters
Grid Killer Application
• Generator - electric bulb
• Power grid – lighting and electric appliances
• Microprocessor - PC (spreadsheets, editors)
• Client/server – web browser (documents)
• Grid – service browser (services)
Next Medium?
Time Technology Medium
1950 TV Broadcast
1980 Time sharing E-mail
1990 Client/Server WWW
2000 Grid Computing ???
Service Browsers
• Continuous discover of services
• List of actually available services
• Service properties
• Service UI
• Admin Service UI
• Available for mobile devices
• Integrated with Web Browser?
Inca X Service Browser
MIDP Service Browser
Blitz Dashboard
SORCER Research Domains• Service-Oriented ProgrammingMission: To address research issues in utilizing network objects to execute a network-centric and reliable control strategy of a grid-based program.
• Service-Oriented Computing EnvironmentMission: To develop architectural innovations by distributing service and context providers with grid infrastructure providers to enable execution of service-oriented programs.
• Service-Oriented Programming Development ToolsMission: To improve service-oriented programming and software deployment by improving the methods used to create, test, debug and monitor execution of such programs.
• Service-Federated Assurance and SecurityMission: To study the cryptography and information security to secure service-oriented grid environments.
• Self-Aware Service FederationsMission: To develop technology that facilitates self-awareness of intelligent service federations. To build and investigate high-performance, practical self-healing software systems for service-federated environments.
• Autonomic Service FederationsMission: To conduct research that combines computer science and biology and facilitates self-management of very complex service-federated environments.
• Service Federated GridsMission: To lead the object-oriented federated services to its full potential by developing common protocols integrated with grids that promote its evolution and ensure its operability.
Theses and 4-blockers
1. Autonomic Management and Monitoring of SO Programs Sekhar Soorianarayanan (Dr. Mike Sobolewski (chair), Dr. Phil Smith, Dr. Noe Lopez Benitez)
2. Mobile Computing Environment with SORCER Surrogate ServicesRavi Malladi (Dr. Mike Sobolewski (chair), Dr. Phil Smith, Dr. Per Anderson)
3. A Surrogate Framework for Personal Profile Devices Mukundan Desikan (Dr. Mike Sobolewski (chair), Dr. Hector Hernandez, Dr. Jean Strahlendorf)
4. A Framework for Integrating SORCER with Web Services Pathangi Rama Krishna Rao (Dr. Mike Sobolewski (chair), Dr. Phil Smith, Dr. Hector Hernandez)
5. Integrating SORCER with RPC Style Web ServicesKiran Masapari (Dr. Mike Sobolewski (chair), Dr. Hector Hernandez, Dr. Yu Zhuang)
6. Service-Oriented Data Mining in SORCERTimmayya Kalappa Ame (Dr. Mike Sobolewski (chair), Dr. Susan Mengel, Dr. Hector Hernandez)
7. Agent-based MetamodelingSandhya Madireddy
8. SILENUS - SORCER Integrated Local New User's StorageMax Berger
Michael [email protected]
