© Donald F. Ferguson, 2014. All rights reserved. Topics in Computer Science: Modern Internet Service Oriented Application Development Lecture 5: Composite

© Donald F. Ferguson, 2014. All rights reserved.

Topics in Computer Science:Modern Internet Service Oriented Application DevelopmentLecture 5: Composite Applications

Dr. Donald F. [email protected] (Admin: [email protected])

2 © Donald F. Ferguson, 2015. All rights reserved.

Modern Internet Service Oriented Application Development –Lecture 5: Composite Applications

Contents



Contents• Components and composite applications

– Introduction– Setting the context

– Composite applications

– Types of composition; Styles of integration

– Modeling, visual notation and executable processes

– Structural composition: A conceptual model – Service Component Architecture

– Data flow composition: Message/event/service bus oriented composition (iPaaS)– Basic building blocks

– Message/data flows and processes

– Control flow composition: Workflow and orchestration oriented composition– Basic building blocks

– Processes and workflows

• Next assignment

• Discussion



ComponentsandCompositeApplications



Introduction



APIs Galore



Setting the Context – “The World is not Enough.”

The world gives you … You ned to make …



Composite Applications –Some Definitions• “In computing, a composite application is a software application built by combining multiple existing functions

into a new application. … A composite application consists of functionality drawn from several different sources.

The components may be individual selected functions from within other applications, or entire systems whose

outputs have been packaged as business functions, modules, or web services.” (http://

en.wikipedia.org/wiki/Composite_application)

• “Composition refers to a way of delivering enterprise solutions by assembling them from prebuilt components,

instead of building them from scratch. It also includes personalization and customization abilities, so that users

can easily and quickly modify specific functionality in the solution.” (http://

msdn.microsoft.com/en-us/library/bb220803.aspx)

• “A composite application orchestrates independently developed programs, data and devices to deliver a new

solution that none of the previously available applications could deliver on its own. Each resource accessed by a

composite application uses a different data model. In most cases, the composite application supports user

interactions beyond those provided by the leveraged applications. In other cases, a composite application may

act as a service. Some composite applications are built by leveraging other composite applications.” (http://

www.gartner.com/it-glossary/composite-application-2)

http://en.wikipedia.org/wiki/Composite_application

http://en.wikipedia.org/wiki/Composite_application

http://msdn.microsoft.com/en-us/library/bb220803.aspx

http://msdn.microsoft.com/en-us/library/bb220803.aspx

http://www.gartner.com/it-glossary/composite-application-2

http://www.gartner.com/it-glossary/composite-application-2



Composite Applications –Some Pictures



Setting the Context• You have learned

– How to build a reusable “component”

– That surfaces a well-defined, reusable API

– And how to monitor, trace, manage and secure the API.

• Any moderately sized company or organization, there are– Thousands of internal APIs.

– 100s of published APIs.

– 100s of consumed, public, external APIs

– Tens (or hundreds) of thousands of available, external, public APIs.

• By and large,– Application development involves

– Controlling the ordering, flow and execution (if … then, while … do, etc) using “code”

– Of API calls (JDBC, class libraries, etc.)

– Modern, Internet SOA Application Development involves– New techniques for controlling the ordering, flow and execution

– Of “web” callable APIs.



Composition Diagrams

• The diagrams are always “nodes” and “arcs.”

• Structure: node A <x> node B, where x is Requires/depends/calls/synchronizes with/…

• Control:– Execute A and then execute B, where A and B are tasks.

– There is a shared set of “data objects” the tasks manipulate.

• Data flow: Do A to the message/document/… and then do B to A’s output

Control Flow Structure Data Flow



Simplistic Taxonomy of Integration

Integration

UIInformationIntegration

ETL, MapReduce

Federated DB

Message/Event

CEP/CPEMessage

Flow

RT events

WorkflowOrchestratio

n

Human Centric

Automation



Modeling, Visual Notation, Executable Models

• Development projects tend to be systematic or opportunistic.

• Opportunistic projects and some systematics project team will tell you that they do not use modeling because– We tried it and it did not work.

– Too slow and too much time. We are agile and want to get to the code.

– That’s the way old guys do it.

• I ask the following questions– Do you have meetings and scrums?

– Do you use a whiteboard? Do you draw diagrams?

– Does someone copy down the information and convert to PPT or diagrams?

– Do you take pictures of the whiteboard?

– Do you email the pictures and PPT?

– Do you realize you are doing modeling, just poorly?

• My view– A model and visual notation is like PPT, but there is precise and consistent interpretation of shapes, colors, symbols, …

– Like PPT, there is an underlying file format/schema for the diagram.

– An executable model has a file format that is complete enough (with possible editing) that some engine can interpret the file the

same way that a JVM interprets bytecodes or a CPU interprets instructions.



Three Core Concepts

Editor

Visual Notation Diagram

“Program”

“Engine”



Of Course, maybe that’s just me …



StructureCompositionDiagrams



Service Component Architecture



Flexible Implementation – Some of which we will Cover



Eclipse SOA SCA Tool



OSGi Blueprint are Similar



OSGi Blueprints



Comments

• Structural composition and models is a recurring concept– OSGi– SCA– J2EE dependency references– Camel– UML Module and Component Diagrams– etc.

• The concept waxes and wanes in popularity, and seems to be waning.– The explosion of micro-services and ad hoc applications.– Structure integrated into behavior composition– More dynamic, query based binding

• We will not spend more time on the topic.



DataFlowComposition



Point-to-Point Connections are BadDoing this … Leads to this … …



Q&ADiscussion



SettingtheContext



An“Old Cranky Guy”ExampleSCA



Three Core Composition Models• Structure

– What are the APIs, and possibly components, that I need? For example, a commerce application needs– Content management system for images, manuals, how-to videos, …– Catalog systems for products, sub-products, prices, …– Inventory management– Credit card processing– Customer account and relationship management

– And what gets connected to what, and how.

• Behavior– Data flow

– Updating thing T in component C needs to– Send an update(Q) thin in component B and D– But I need to split the data because B and D only need some of T– And I need to remap the data because B and D have different schema for “T” stuff.

– Control flow: Opening a new credit card requires things to happen in order– Validate the application is correct– Perform a credit check– Call a system to create a risk score– Have an analyst approve the application– Send an email to the customer for approval– Send a message to the place that is going to make the card– … …

• People like diagrams.

• But tend to use similar diagramstypes for all composition.

• And be inconsistent about whatsymbols means



You Have Already Seen the Pattern

• This is the same pattern as API Mgmt

• But …

• What happens in the “proxy”is a lot more complicated.

• The relationship betweenthe proxy and the APIs

• Is many-to-many

• And has data flowor sequencing.



Integration is PurpleBoomi Example



Enterprise Integration Patterns with WSO2 ESBhttps://docs.wso2.com/display/IntegrationPatterns/Enterprise+Integration+Patterns+with+WSO2+ESB



More Examples



More Examples



How do you Build the Composite

• Take off a palette

• Set properties

• Put on canvass

• Set control values

• Connect flow



Composition• Each “clipart widget” has

– A unique, documented shape (visual notation).– A little implementation piece of code (think POJO) that does the function.– Rules about what other widgets can connect to it/from it.– A set of properties that tailor this instance’s behavior

• For example, – When I drag a routing widget onto the canvass to make a routing instance– I specify

– The valid, input JSON format.– A the routing rule table

– {“Person.iq” <= 50; http://www.Columbia.edu/api/professors}– {“Person.color” == “Yellow”; http://www.pbs.org/api/SessameStreet/birds}

– And connect to other things on the palette

http://www.columbia.edu/api/professors

http://www.pbs.org/api/SessameStreet/birds



Dell Boomi



Test Drive



Workflow/BPMN



BPMN

BPMN 2.0 is an international standard for business process modeling.The BPMN specification describes

• how the elements of a process diagram have to look like (notation),

• how these can be combined with each other (meta model / syntax),

• what a diagram means (semantics) and

• how diagrams can be transferred from one tool to another (XML interchange format).

Process models describe sequences of business activities from start to finish, e.g. Order-to-Cash, Account Open



BPMN and Activiti• Some terms

– BPMN 2.0 is a standard for defining/documenting workflows/orchestrations.

– Activiti (and many others) are implementations of the standard– Runtime engine that can interpret and execute a BPMN process

– A WYSIWYG editor– With prebuilt, configurable shapes palette that extend base BPMN shapes. Huh?

• BPMN has the concepts of a “Task,” which is one “step” in a process.

• But in a process model, this is just “prose” like “Order a Pizza”

• Implementations come with “common” types of tasks, e.g.• Make an SQL call on a database.

• Call a REST API

• Put a document on a worklist

• etc.

– A composition canvass for assembling shapes into processes.

– Debugger, monitoring, reporting, versioning, etc.

• This is– The same concept as enterprise integration patterns and Boomi

– But, – The clipart is different

– And the graph mostly represents “control flow,” not data flow.



Simple BPMN Diagram



BPMN Symbols



BPMN Symbols



BPMN Symbols



BPMN Symbols



BPMN Symbols



BPMN Symbols

Lots more go here.

There are references, e.g. http://www.bpmn-tool.com/en/tutorial/



BPMN Symbols



BPMN Symbols

Seriously dude?

There are references, e.g. http://www.bpmn-tool.com/en/tutorial/



BPMN Symbols



And You can Define Processes



How do you Implement?• Notify a person/role.

• Have them do something.

• Have the use anapp that calls theengine to tell youthey are done.

• Send an email.

• POST a document

• etc.

• Decision table

• Call Drools

• etc.

• Insert into a DB

• Call a web service

• Run a Java app.

• etc.

• Run some JavaScript or Java or …

• Right at this point in the workflow/process

• And manipulate the data/documents/control.



User Task



Script Task



Service Task



















CommentsThree Core Aspects of BPMN and Actviti

• Human Centric Workflow– Define an organization model

– Users, Groups– Properties for users and groups

– Define workflow processes that have Human Tasks. For each Human Task– Define the form and data that enables a person to perform the task.– Define “rules” for assigning a task to people or groups.

• Automation/Orchestration– Implement a set of POJOs implementing the Delegate interface. Each POJO implements an automation

action by …– Call a REST service– Sending a message on JMS or SQS– Etc.

– Define a workflow with one or more Service Tasks. For each Service Task– Specify/bind to a Delegate.– Map process data variables to inputs and outputs of the Delegate.



AWS Simple Workflow



AmazonSimple Workflow



Application Structure• Role of the Activities Worker: The activities worker performs the

various tasks that the workflow must accomplish. It consists of:– The activities implementation, which includes of a set of activity methods

that perform particular tasks for the workflow.

– An activity worker, which uses HTTP long poll requests to poll Amazon SWF for activity tasks. When a task is available, Amazon SWF responds to the request by sending the information required to perform the task. The activity worker then calls the appropriate activity method, and returns the results to Amazon SWF.

• Role of the Workflow Worker: The workflow worker orchestrates the execution of the various activities, manages data flow, and handles failed activities. It consists of:

– The workflow implementation, which includes the activity orchestration logic, handles failed activities, and so on.

– An activities client, which serves as a proxy for the activities worker and enables the workflow worker to schedule activities to be executed asynchronously.

– A workflow worker object, which uses HTTP long poll requests to poll Amazon SWF for decision tasks. If there are tasks on the workflow task list, Amazon SWF responds to the request by returning the information that is required to perform the task. The framework then executes the workflow to perform the task and returns the results to Amazon SWF.



Assignment 3



Backup

Documents

© Donald F. Ferguson, 2014. All rights reserved. Topics in Computer Science: Modern Internet Service Oriented Application Development Lecture 5: Composite