Component Frameworks and Programming Approaches for Application Web Services

Component based Applications Programming within a Service-Oriented Grid Environment

RainerSchmidt, Austria Research Centers, DME

EchoGRID, Third Strategic Workshop, June 2008

Component Frameworks and Programming Approaches for Application Web Services

Rainer Schmidt,Austrian Research Centers [email protected]
VGE-CCA



Outline
The VGE-CCA Component FrameworkVGE and VGE-CCA extensions CCA libraries, Infrastructure Elements, Client EnvironmentApplication Component Libs (clibs)Plugged into the Web Service ContainerDistributed Application CoordinationProgramming Model Component Creation and Composition, Handling StateData Staging, Coupling, Infrastructure ServicesThe Planets Interoperability FrameworkA Service-oriented Architecture for Digital PreservationComponent and Programming Model Configuration and Execution Environment



Common Component Architecture
CCA is designed for high-performance applicationFacilitates import of scientific code into frameworks.Provides a plug-and-play environment3 party composition capabilities, based on InteractionsCCA Fundamental ConceptsComponents: Units of software that can be composedFrameworks: Environment for components to run and interactCCA Specification defines behavior for component and framework interactionPlatform and language neutral.does not prescribe implementation detailsApplicable to parallel and distributed/Grid environments.



VGE - On-Demand Supercomputing
HPC Service provision frameworkEncapsulates native HPC applications on parallel hardwareOffers common set of services (conversational)job execution, monitoring, file transfer, QoS, error recoveryNegotiable QoS support for time-critical applicationsAdvance Reservation, Perf. Models Dynamic Configuration, WSLAStandard Web service technologyAXIS, SOAP attachments, WS-AddressingSecurity: TLS, WS-Security, X.509 certificates, End-to-EndApplication services, Registries, High-level client APIEU Projects: GEMSS, Aneurist



VGE-CCA
Distributed CCA framework atop a Web service based Grid environmentModels VGE application services as distributed CCA components.Adds port-based composition capabilities to Web services.Plugin mechanism for application-specific libraries Model for Distributed Grid ProgrammingActively interacting Grid application componentssequentially, concurrentlyCo-scheduling concurrently running Grid components.Client composition and computational steering.



The VGECCA System
Service-side CCA LibrariesNo modification of VGE core functionality (still a valid VGE service)Adds Builder service (creation and connection)Provide CCA framework to the application service.Transparently utilize remote framework services (registry, proxy)Component Libraries (clibs) Plugin MechanismExtend VGE comp. with app. specific. logic, ports, dependenciesMethod for distributed application coordination.Framework Web ServicesComponent Registry, Proxy Registry, Negotiation ServiceComponent-based Client APIDistributed programming by interlinking parallel app. components.



Anatomy of a VGE-CCA Component
VGE Ports(App, QoS,Recovery)
Creation +ConnectionPort
App. SpecificProvides Ports
CCA lib
Port+Conn.Table
CCATypes
CCAServices
VGE libs
App. Handler
OS-level interaction
service specificcomp. code
Fw. Web Service(component, proxy repository)
connectedServices

binaries, files



Creation and Composition
Component Creation by ContextualizationConversational Identifier, WS - AddressingDifferent Types of Connectionsstateful + stateless interactionssequential + concurrent running applicationsPort Connections to Infrastructure ServicesConnect Grid component to Infrastructure Web serviceE.g. CRL-Service, Logging ServiceConnect using port mechanismvisible dependenciesConfigured descriptively or dynamically found



Application Coupling
Coupling Application PartsCo-scheduled application components.VGE Advance ReservationSelection: QoS Negotiation ServicePort-based Interactions of concurrently runningComponents interact as neededApplication Specific Component Plugins
Ant Colony Optimization:

- Component signals current best solution



Example: Ant Colony Optimization
ACO algorithm for solving a VRP.MPI-based application with mixed parallelization strategySeveral colonies of ants cooperate in finding good solutions.Cooperating colonies periodically exchange single solutions.
Basic Decomposition Approach:Partition each colony of ants into n (# processors) sub-colonies (fine grained)Distributed coarsely grained families.Provide Ant Colony as service.Exchange information between colonies



Distributed Coordination
Ant Application Components exchange solutions as required.Client Application remains unaware of application specific interaction logicClient is not involved into computation, may go offline.Only required for application steering (start, stop, )
Idea: Aggregate Distributed Resources



Application Specific Plugins
Pluggable Component Libraries (clibs)hooks needed to extend an application serviceintercept port invocationsadd custom ports and dependenciesapplication specific component interaction using the service-side CCA Framework"Signal" Interface added for Ant componentsProvides and uses PortImplemented as one-to-many connectionDaemon keeps track of local optimum and signals if required.



Programming Model
VGE-CCA Programming ModelAbstract Specification (ports, apps, quality)registry and negotiationSpatial Compositionport-based connections, dynamic proxy Control-Flow (Computational Steering, Monitoring, Data Staging)Client workflow orchestrationSeparates applications composition from runtime control flow steeringVGE-CCA provides an extensible API for basic CCA constructs and specialized components.



ACO Client Application
//component descriptioncmpDsc.addProperty(CConst.APP, "ACO");qosDsc.setBeginTime(beginTime);qosDsc.setWallTime(wallTime);cmpDsc.addProperty(CConst.qosDsc, qosDsc);//creation and application assemblingVgeComponent ant1 = builder.createInstance(cmpDsc); ...AntComponentGroup antComposite = new AntComponentGroup(ant1, ant2,ant3);//client workflowantComposite.upload(vrp_infile);antComposite.start(); while(antComposite.getStatus != ...) {



Planets Project
Integrated Services for Digital PreservationIST FP6 Project - IP 14mConsortium of Libraries, Archives, Universities, Tech. Companieshttp://www.planets-project.euOGF Preservation Environments RGFocused on Data GridsSub-Project: Interoperability FrameworkDefines a SOA for Digital Preservation builds atop Application Web Service ModelInteroperability, Virtualization



Distributed Digital Preservation System
Integration of heterogeneous environmentExisting Applications, 3rd Party Services, RepositoriesMassive Data Amounts, Complex Object TypesResults in Complex Grid Applications and WorkflowsStrong focus on data provenanceDevelop a Common Data ModelEnsure repository consistencyProvide Transaction modelComponent-based Programming ModelAbstract Complexity Configuration and Execution Plug-and-Play Environment



Planets Interoperability Framework
Generic, distributed portal framework Ties together the involved componentsMigration, Emulation, Characterization ServicesPreservation Planning, Format Registries, RepositoriesCommon Services: User Management, Service Registry, Workflow Enactment, SecurityProvide Programming and Execution EnvironmentCurrently based on EJB/Web Service ModelAdd Independent DP Component ModelProgramming and Workflow features



IF Architecture

PLANETS Interoperability Framework

Service Bus

Core Elements

Security/Authentication Authorization

Monitoring/Logging/Auditing

Workflow Execution Engine

Error/Exception Handling

Transaction Manager

PLANETS Applications

Testbed

Preservation Planner

Characterisation Tools

.

WorkflowDesigner

Administration Tool

Database Layer

WorkSpace

Person 1



App. Service - Abstraction Levels
Level 0: Application
Level 1 Web Services
A
B
C
Level 2 Planets Components
Workflow - Level
Planets IF - Instance
Planets IF - Instance



Components Model
Abstraction based on layered approachLevel 1: Any network accessible applications (services)Level 2: Compostable HL Abstractions (components)API/Type System for digital preservation componentsHigh-level with explicit context dependenciesDefined interfaces and compositionExchange/manipulate a common data model.Life cycle, exception handling, Abstracts data model on component levelImplicitly maintain information about provenance of dataAutomate Repository InteractionConsistency, Transactions, Abstraction



Programming and Execution
Levels of application abstraction and constructionDifferent user groups with different expertise: PA -> Component developers; PP -> Workflow developers; Archivists, Libarians -> User.Workflow TemplatesAssembled from high-level preservation componentsIndependent from underlying services/middlewareRely on provider conceptWorkflow Configuration and InstantiationAutomatically by Preservation Planning Services ORBased on Graphical Interface (Portal)Process archive consists of template and associated view.



Conclusions
Light-weight App. Service ContainersNo changes to underlying application (legacy apps.)Advanced Grid Services (SLA, interactions)Provide High-Level Programming ModelReq. Extensibility: Based on Common Independent Concepts Abstract and Independent from underlying MiddlewareAllow Encapsulate Application ComplexityDistributed CoordinationImplicitly interact with Infrastructure ServicesGreat Potential for Cloud ComputingBridge SOA and Component TechnologiesComposition, Dynamic Resource Selection



I will introduce VGE-CCA a distriubuted CCA frameworkBasic Idea- I will give a general introduction CCA and component based Grid application composition- VGE-CCA is our approach to apply port-based composition to Grid application Web services
distributed and Grid based frameworks: XCAT, LeginCCA, ...Component technology provides a powerful way for constructing complex software systems by decoupling software implementation from application assembly.
on-demand or Utility Compouting Model
clibs in order to insert the component logicruntime computational steering
Libraries implement the CCA framework: dont change just add methods e.g for compostionprovide frameworks services like port registration, and a connection tableVGE services are generic -> clibs are libraries that are specifically tailored to an underlying application.
VGE-CCA Component and Component ContainerService Specific Code: Does Grid interaction, Developer needs to write the logic and plug it into VGE
Another idea to use CCA for connectiong/configuring infrastructure (non-computational) services (e.g. CRL Service, Logging Service). You can keep a lot of the inter-service communication out of the service program based on interfaces and allow to configure/dynamically find a service to connect tofixed choreograhy for b2b
concurrently running components communicate autonomously and hidden to the userthe interaction logic is provided by an component library
ACO alg. try to solve combinatorial optimization problems by simulating the behavior of natural ant colonies that communicate via pheromene trails.Application uses aco for finding round-trip routes that start and end at a single depotVRPs are np-hard problems not solvable in polynomial time.Family iteratively finds and improves solution paths (fine-grained). Parallel running colonies exchange solutions or parts of pheromone information (coarse grained).

the same decomposition approach can be used for a number of combinatorical and numerical problems, also to use imagine different solver implementations using different heuristics (hearistic)
the advantage is...
there is not one uses port for every connected component, they can dynamically be added or removed
Write application once run anywheredescribe components you want to use, based on application and qos meta-data..at runtime! the client perfoms the control flow
now you would use introspection to discover uses/provedes portsbut we created a componentgroup abstraction -> client composite

Documents

Component Frameworks and Programming Approaches for Application Web Services