SS ZG653Second Semester,2012-13 1 Topic Architectural Patterns Pipe and Filter

SS ZG653Second Semester,2012-13 1

Topic

Architectural Patterns Pipe and Filter


PIPES & FILTERSMud to Structure


Pipes and FiltersThe pipes and filters architectural pattern provides a structure for systems that process a stream of data. Each processing step is encapsulated in a filter component. Data is passed through pipes between adjacent filters. Recombining filters allows you to build families of related systems.


Pipes and Filters• Type of Data Flow Architecture• Filter is a component and pipe is a connector• Filter has interfaces from which a set of inputs can flow

in and a set of outputs can flow out.• Incremental transformation of data by successive

components.• All data does not need to be processed for next filter to

start working.• Any set of filters may be combined in any order,

although reasonable semantics are not guaranteed by this style.


Pipes and FiltersFilter • Independent entities• Does not share state with other filters.• Do not know the identity to upstream and

downstream filters.Pipes• Stateless data stream• Source end feeds filter input and sink receives output.


Pipes and Filters

Lexical Analyzer

Parser

Source File

ASCII Text

Token Stream

Semantic Analysis

Abstract Syntax Tree

Code Generator

Augmented Abstract Syntax Tree

Optimizer

Object Code

Object File

Optimized Object Code


Pipes and Filters• Compilers• Various “translation” systems


Pipes and Filters – 3 part schemaPattern

Context Processing Data StreamsProblem System that must process or transform a stream of input

data.Expect flexibility by exchanging or reordering the processing steps. Possible to build family of such systems.• Future enhancements – exchange processing steps or

recombination• Small processing steps – aid reuse• Non adjacent processing steps do not share information• Different sources of data exist• Store final result in various ways• Explicit storage of intermediate results should be

transparent to users – is error prone if done by users• Multiprocessing the steps

Solution Pipes and filters – data source to data sink


Pipes and Filters• Filter component is a

processing units of the pipeline

• Enriches, refines or transforms its input data

• Enriches – computing and adding information

• Refine – concentrating or extracting information

• Transforms – delivering data in some other representation

ClassFilter

Responsibility• Gets inputs data• Performs a

function on its input data

• Supplies output data

Collaborators• Pipe


Pipes and Filters• Pipe denotes connection

between filters• Data source and first filter• Last filter and Data sink• Synchronises two active

component

ClassPipe

Responsibility• Transfers data• Buffers data• Synchronizes

active neighbors

Collaborators• Data Source• Data Sink• Filter


Pipes and Filters• Data source represents input

to the system• Sequence of data of the

same structure or type

ClassData Source

Responsibility• Delivers input to

processing pipeline



Pipes and Filters• Data sink collects results

from end of the pipeline• Active: pulls results from

preceding processing stage• Passive: allows preceding

filter to push or write the results into it

ClassData Sink

Responsibility• Consumes output



Dynamics• Scenario I

– Push pipeline [Activity starts with the Data source]– Filter activity started by writing data to the filters– Passive Filter [Use direct calls to the adjacent pipeline]Data Source Filter 1 Filter 2 Data Sink

Write(data)Transform(data)

Write(data)Transform(data)

Write(data)


Dynamics• Scenario II

– Pull pipeline– Control flow is started by the data sink calling for

data

Data Source Filter 1 Filter 2 Data Sink

Read

Read

Read

dataTransform(data)

data

data

Transform(data)


Dynamics• Scenario III

– Push-pull mixed pipeline

Data Source Filter 1 Filter 2 Data Sink

Read

Read

data

Transform(data)

data

Write(data)

Transform(data)


Dynamics• Scenario IV

– All filters actively pull, compute and push data in a loop– Each filter runs its own thread of control– Filters are synchronised by buffering pipe between them

Buffering PipeData Source Filter 1 Filter 2 Data Sink

Read

data

Transform(data)

Read

Write(data)

data

Write(data)

Read

dataTransform(data)

Write(data)

Read

dataTransform(data)

Write(data)

Transform(data)


Implementation# Steps1 Divide the system’s task into a sequence of processing stages

2 Define the data format to be passed along each pipe

3 Decide how to implement each pipe connection

4 Design and implement the filters

5 Design the error handling

6 Set up the processing pipeline


1: Divide the systems tasks into sequence of processing stages

• Each stage must depend on the output of the predecessor

• All stages conceptually connected by data flow


2: Define data format to be passed along each pipe

• Define a uniform format results in the highest flexibility because it makes recombination of filters easy

• Define the end of input marking


3: Decide how to implement each pipe connection

• Decision determines active or passive filter• Using a separate pipe mechanism that

synchronises adjacent active filters provide a more flexible solution


4: Design and implement the filters• Depends on

– Task it must perform– Adjacent pipe

• Active or Passive filters• Performance and tradeoffs• Filter reuse


5: Design the error handling• Never neglect error handling• No global state shared; error handling hard to

address• Strategies in case of error – depend on

domain


6: Setup the processing pipeline• Use of standardised main program• Use of user inputs or choice


Variants

• Tee and Join pipeline– Filters with more then one input and/or more

than one output


Benefits

• No intermediate files necessary, but possible• Flexibility by filter exchange• Flexibility by recombination• Reuse of filter components• Rapid prototyping of pipelines• Efficiency by parallel processing


Liabilities• Sharing state information is expensive or inflexible• Data transformation overhead• Error handling


Known Uses• Find (at least 2) more popular uses and document

them

Documents

SS ZG653Second Semester,2012-13 1 Topic Architectural Patterns Pipe and Filter