Paradigms for Process Interaction ECEN5053 Software Engineering of Distributed Systems University of Colorado, Boulder

Paradigms for Paradigms for Process InteractionProcess Interaction

ECEN5053 Software Engineering of Distributed Systems

University of Colorado, Boulder

10/6/2004 ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder

Paradigms

Ways to combine the three basic process interaction paradigms

All were developed as graph theory problems Now that distributed computing exists,

– we can use them– recognize problems as variants of these paradigms– apply combinations to obtain solutions– some have grown up to be patterns


A limitation

There are entire books on fault tolerant systems Necessary where high availability and reliability

are essential Outside the scope of this course Therefore, the algorithms will assume the

processors do not fail The Patterns take this into account to some extent The more you know, the more you know you don’t

know


Overview Manager/Workers, aka “Distributed Bag of Tasks”

– Master-Slave Architectural Design Pattern Heartbeat algorithms Pipeline algorithms

– Pipes and Filters Architectural Design Pattern Probe/Echo algorithms Broadcast algorithms Token-passing algorithms

– Asynchronous Completion Token Pattern Replicated servers Microkernel Architectural Design Pattern Broker Pattern


Overview(continued) Communication patterns

– Forwarder-Receiver– Client-Dispatcher-Server– Publisher-Subscriber


Manager/Workers

Several worker processes share a bag containing independent tasks

Each worker repeatedly removes a task from the “bag”, executes it, possibly generates one or more new tasks that it puts into the “bag”.

This is a variant of which primary paradigm? Useful for:

– Problems with a fixed number of independent tasks

– Problems resulting from divide-and-conquer approach


Manager/Worker (cont. 1) Benefits

– Easy to vary the number of workers– Easy to ensure that each does about the same

amount of work ImplementationManager Workerimplement the bag get tasksdole tasks deposit results bycollect results comm with mgrdetect termination


Master-Slave Pattern (POSA 1)

A way to organize work Supports fault tolerance, parallel computation, and

computational accuracy. A master component distributes work to identical

slave components and computes a final result from the results these slaves return

Applies the divide and conquer principle Partition work into several subtasks processed

independently A special case is called “triple modular

redundancy” – result is considered valid if at least 2 provide the same result.


Master-Slave Pattern3 application areas Fault tolerance:

– Execution of a service is delegated to several replicated implementations. Failure of service executions can be detected and handled.

Parallel computing – most common:– A complex task is divided into a fixed number of

identical sub-tasks executed in parallel. Computational accuracy:

– Execution of a service is delegated to several different implementations. Inaccurate results can be detected and handled.


Heartbeat

Reason for the name – worker actions– expand– contract– process– repeat

Useful– Grid computations – image processing– Cellular automata – simulations; e.g. forest fires,

biological growth– What do these have in common?


Heartbeat Implementation Recall meaning of a “barrier”

– Ensures every worker finishes one iteration before starting the next

– Heartbeat implements a “fuzzy” barrierprocess Worker [ = 1 to numberOfWorkers] { declarations of local variables; initializations of local variables; while (not done) { send values to neighbors; receive values from neighbors; update local values – process information; }}


Pipelines

A filter process – receives data from an input port, – processes it, and – sends results to an output port

What primary paradigm category is this? A pipeline is a linear collection of filter processes You see this concept in Unix pipes in a single CPU

environment.


Pipelines (cont.) Useful

– A way to circulate values among processes– A sorting network– Parallel computations

Three basic structures for a pipeline– Open: Input source and output destination are

not specified. Can “drop down” anywhere.– Closed: Open pipeline connected to a

coordinator as input source and output consumer– Circular: Pipeline whose ends connect


Pipeline example Already looked at 2 distributed implementations

of matrix multiplication a * b = c where n x n matrices are dense.

1. n workers; 1 worker per row of a but each worker stored all of b

2. n workers but each stored only one column of b n workers, each produces 1 row of c workers have none of a or b to start connected in a closed pipeline thru which they

acquire the data items and pass the results


Pipeline example (cont) Coordinator

– sends every row of a and every column of b down the pipeline to the first worker

– eventually receives every row of c from the last worker Each worker

– receives rows of a, – keeps the first it receives– passes the subsequent rows down– receives columns of b, passes them and computing one

inner product– sends its row of c to next, then receives and passes on rows

of c from previous workers in the pipeline– last worker sends its row of c and others it receives to

coordinator


Pipes and Filters Pattern POSA 1 – under Architectural Patterns Provides a structure for systems that process a

stream of data. Each processing step is encapsulated in a filter

component. Data is passed through pipes between adjacent

filters. Recombining filters allows you to build families of

related systems.


Pipes and Filters -- forces Be able to enhance system by exchanging

processing steps, recombining steps Small processing steps are easier to reuse in

different contexts Non-adjacent processing steps do not share

information Different sources of input data exist Possible to present or store final results in various

ways Explicit storage of intermediate results in files

clutters directories and is error-prone May not want to rule out multi-processing steps


Pipes and Filters - solution Divides the task into several sequential processing

steps– Connected by a data flow; output of a step is

input to the next– Each step is implemented by a filter component– A filter consumes and delivers data

incrementally to achieve low latency and enable real parallel processing

– Input is a data source; Output is a data sink.– Input, filters, output connected by pipes which

implement the data flow between the steps


Pipes and Filters - Scenarios


Pipes and Filters – Scenarios


Pipes and Filters – Benefits & Liabilities Benefits

– No intermediate files necessary but possible– Flexibility by filter exchange– Flexibility by recombination– Reuse of filter components– Rapid prototyping of pipelines– Efficiency by parallel processing

Liabilities– Sharing state information is expensive or inflexible– Efficiency gain by parallel processing may be illusion– Data transformation overhead– Error handling – Achilles heel

Documents

Paradigms for Process Interaction ECEN5053 Software Engineering of Distributed Systems University of Colorado, Boulder