Markov Chain Analysis – Example - Startseite TU Ilmenau · Integrated HW/SW-Systems Andreas Mitschele-Thiel 2-Feb-11 1 Markov Chain Analysis – Example Example: single server with

Integrated HW/SW-Systems 1Andreas Mitschele-Thiel 2-Feb-11

Markov Chain Analysis – Example

Example: single server with a queue holding up to 2 requests

3 2 1 0

denotes the arrival rate denotes the service rate

Steady-state analysis: mapping of Markov chain on a set of linear equations

state 3: p2 = p3state 2: p1 + p3 = p2 + p2state 1: p0 + p2 = p1 + p1 state 0: p1 = p0normalization equation: p0 + p1 + p2 + p3 = 1

resolution of system of equations to derive state probabilities

p0 = 1 / (1 + / + (/)2 + (/)3 )p1 = (/) / (1 + / + (/)2 + (/)3 ); p2 = ...

derivation of mean values and distribution functions from state probabilitiesutilization: U = 1 - p0; mean number of jobs in system: N = p1 + 2p2 + 3p3 ;blocking probability B; distribution functions for waiting time, response time, etc.

steady state implies thatarrivals = completions


Queuing Network Analysis - Example

Example: communication system (unbounded queues)

prozessor120 MIps

prozessor100 MIps

phy network30 Mbps

= 100 MIps / 2 MI = 50 1/s

= 30 1/s

= 43 1/s

= 25 1/s

medium1 Mb

2 MI

2,8 MI

arrival rate: 25 packets/s

prot. stack

prot. stack

Results (assuming exponential service times and arrivals): utilization = (e.g. physical network util. = 83 %) population per station ni = ( mean total population N =

ini =7,39

mean response time T = N/ = 296 msec (Little’s law)


Discrete-event Simulation

P1P2

12

5 34

6

0 2 4 6 8 10

Idea: model the relevant events of the system process the events according to their temporal order (similar to the real

execution with the exception that simple processing blocks are modeled only)

Example: Execution on a 2-processor system (non-preemptive)

Discussion no assumptions danger of including too many details evaluation is time consuming problems with rare events large set of tools and models available

Legend:green: execution timeblue: process priority (1=low)

1

2

34 5

6

2

43

1

4

1

/1

/4/3

/6

/5

/2


Comparision of Methods

Analysis Analysis of multiprocessor systems

Verification of Real Time requests

Modelling of parallel processing (prec. constraints)

best case

worst case

average

Process graph no

Task graph not really ?

Schedulability

Utiliz.-based no no

Resp.-based no no

Markov chains no

Queuing networks

no no

Operational analysis

no no

Simulation no

Measurements no


Heuristic Search

Most heuristics are based on an iterative search comprising the following elements: selection of an initial (intermediate) solution (e.g. a sequence) evaluation of the quality of the intermediate solution check of termination criteria

select initial solution

select next solution(based on previous solution)

evaluate quality

acceptance criteria satisfied accept solution as„best solution so far“

termination criteria satisfied

y

y

n

search strategy


Hill-Climbing – Discussion

simple local optimizations only: algorithm is not able to pass a valley to finally reach

a higher peak idea is only applicable to small parts of optimization algorithms but needs to

be complemented with other strategies to overcome local optimas


Random Search

also called Monte Carlo algorithm

Idea: random selection of the candidates for a change of intermediate solutions or random selection of the solutions (no use of neighborhood)

Discussion: simple (no neighborhood relation is needed) not time efficient, especially where the time to evaluate solutions is high sometimes used as a reference algorithm to evaluate and compare the

quality of heuristic optimization algorithms idea of randomization is applied to other techniques, e.g. genetic algorithms

and simulated annealing


Simulated Annealing

Idea: simulate the annealing process of material: the slow cooling of material leads to

a state with minimal energy, i.e. the global optimum

Classification: Search strategy

random local search

Acceptance criteria

unconditional acceptance of the selected solution if it represents an improvement over previous solutions

otherwise probabilistic acceptance

Termination criteria

static bound on the number of iterations (cooling process)


Simulated Annealing – Discussion and Variants

Discussion: parameter settings for cooling process is essential (but complicated)

slow decrease results in long run times fast decrease results in poor solutions discussion whether temperature decrease should be linear or logarithmic

straightforward to implement

Variants: deterministic acceptance nonlinear cooling (slow cooling in the middle of the process) adaptive cooling based on accepted solutions at a temperature reheating


Genetic Algorithms – Basic Operations

crossover

1 1 0 0 1 0 1 0 1 1 0 1 0 1 0 0 1 0 0 1

1 1 0 0 0 0 1 0 0 1

1 1 0 0 0 1 1 0 0 1

mutation


Genetic Algorithms – Basic Algorithm

General parameters: size of population mutation probability candidate selection strategy (mapping quality on probability) replacement strategy (replace own parents, replace weakest, influence of

probability)

Application-specific parameters: mapping of problem on appropriate coding handling of invalid solutions in codings

crossover

replacement

mutationselection

population

Replacement and selection rely on some cost function defining the quality of each solution

Different replacement strategies, e.g. “survival of the fittest”

Crossover selection is typically random


Genetische Algorithmen – Minimum Spanning Tree

small population results in inbreeding larger population works well with small mutation rate tradeoff between size of population and number of iterations


Genetic Algorithms –Basic Operations

Mutation => crating a new member of the population by changing one member


Genetic Algorithms –Basic Operations

Crossover => crating a new member of the population from two members


Genetische Algorithmen – Traveling Salesman Problem

minimal impact of mutation rate with small population negativ impact of high mutation rate with larger population (increased randomness) – impact not quite clear


Genetic Algorithms – Discussion

finding an appropriate coding for the binary vectors for the specific application at hand is not intuitive problems are redundant codings, codings that do not represent a valid solution, e.g. coding for a

sequencing problem tuning of genetic algorithms may be time consuming parameter settings highly depend on problem specifics suited for parallelization of optimization


Tabu Search

Idea: extension of hill-climbing to avoid being trapped in local optima allow intermediate solutions with lower quality maintain history to avoid running in cycles

Classification: Search strategy

deterministic local search

Acceptance criteria

acceptance of best solution in neighborhood which is not tabu

Termination criteria

static bound on number of iterations or dynamic, e.g. based on quality improvements of solutions


Tabu Search – Algorithm

The brain of the algorithm is thetabu list that stores and maintains information about the history of the search.In the most simple case a number of previous solutions are stored in the tabu list.More advanced techniques maintain attributes of the solutions rather than the solutions itself

select initial solution

select neighborhood set(based on current solution)

remove tabu solutionsfrom set

set is empty

increase neigborhood

termination criteria satisfied

y

y

n

nevaluate quality and

select best solution from set

update tabu list


Tabu Search – Organisation of the History

The history is maintained by the tabu listAttributes of solutions are a very flexible mean to control the search

Example of attributes of a HW/SW partitioning problem with 8 tasks assigned to 1 of 4 different HW entities:(A1) change of the value of a task assignment variable(A2) move to HW(A3) move to SW(A4) combined change of some attributes(A5) improvement of the quality of two subsequent solutions over or below

a threshold value

Aspiration criteria: Under certain conditions tabus may be ignored, e.g. if a tabu solution is the best solution found so far all solutions in a neighborhood are tabu a tabu solution is better than the solution that triggered the respective tabu conditions

Intensification checks whether good solutions share some common properties

Diversification searches for solution that do not share common propertiesUpdate of history information may be recency-based or frequency-based (i.e. depending on the frequency that the attribute has been activated)


Tabu Search – Discussion

easy to implement (at least the neighborhood search as such) non-trival tuning of parameters tuning is crucial to avoid cyclic search advantage of use of knowledge, i.e. feedback from the search (evaluation of

solutions) to control the search (e.g. for the controlled removal of bottlenecks)


Heuristic Search Methods – Classification

Search strategy search area

global search (potentially all solutions considered) local search (direct neighbors only – stepwise optimization)

selection strategy deterministic selection, i.e. according to some deterministic rules random selection from the set of possible solutions probabilistic selection, i.e. based on some probabilistic function

history dependence, i.e. the degree to which the selection of the new candidate solution depends on the history of the search no dependence one-step dependence multi-step dependence

Acceptance criteria deterministic acceptance, i.e. based on some deterministic function probabilistic acceptance, i.e. influenced by some random factor

Termination criteria static, i.e. independent of the actual solutions visited during the search dynamic, i.e. dependent on the search history


Heuristic Search Methods – Classification

Heuristic Search strategy Acceptance criterion

Termination criterion

Search area Selection strategy History dependence

local global det. prob. random none one-step

multi-step

det. prob. stat. dyn.

hill-climbing

x x x x x

tabu search

x x x x x x

simulated annealing

x x x x x

genetic algorithms

x x x x x x x

random search

x x x x x


Single Pass Approaches

The techniques covered so far search through a high number of solutions.

Idea underlying single pass approaches: intelligent construction of a single solution (instead of updating and

modification of a number of solutions) the solution is constructed by subsequently solving a number of subproblems

Discussion: single-pass algorithms are very quick quality of solutions is often small not applicable where lots of constraints are present (which require some kind

of backtracking)

Important applications of the idea: list scheduling: subsequent selection of a task to be scheduled until the

complete schedule has been computed clustering: subsequent merger of nodes/modules until a small number of

cluster remains such that each cluster can be assigned a single HW unit


Single Pass Approaches – Framework

The guidelines are crucial and represent the intelligence of the algorithm

derive guidelines forsolution construction

select subproblem

decide subproblembased on guidelines

possibly recompute oradapt guidelines

final solution constructed

y

n


List Scheduling

List scheduling: subsequent selection of a task to be scheduled on some processor (or HW

entity) operation is similar to a dynamic task scheduler of an operating system

assign priorities to the tasksaccording to some strategy

select executable taskwith highest priority

assign task to a processoraccording to some strategy

schedule complete?

y

n

priorisationstrategy

assignmentstrategy


List Scheduling – Example (1)

Problem: 2 processors 6 tasks with precedence constraints find schedule with minimal execution time

HLFET (highest level first with estimated times) length of the longest (critical) path to the sink

node (node 6)Assignment strategy

first fit

Resulting schedule:

Legend:green: estimated timesred: levels (priorities)

1

2

3

4 5

6

2

43

1

4

1

/8

/5/4

/1

/5

/6

P1P2

1 25

34

6

0 2 4 6 8 10



Problem (unchanged): 2 processors 6 tasks with precedence constraints find schedule with minimal execution time

SCFET (smallest co-level first with estimated times)

length of the longest (critical) path to the source node (node 1)

Assignment strategy first fit

Resulting schedule:

Legend:green: estimated timesblue: co-levels (priorities)

1

2

3

4 5

6

2

43

1

4

1

/2

/6/5

/8

/7

/3

P1P2

1 2 534

6

0 2 4 6 8 10



Problem: 2 processors 6 tasks with precedence constraints find schedule with minimal execution time

HLFET (highest level first with estimated times) length of the longest (critical) path to the sink

node (node 6)Assignment strategy

first fit

Resulting schedule:

Legend:green: estimated timesred: levels (priorities)

1

2

3

4 5

6

2

43

1

4

1

/8

/5/4

/1

/5

/6

P1P2

1 25

34

6

0 2 4 6 8 10


Clustering - Basics

Partitioning of a set of nodes in a given number of subsets

compute the „distance“between any pair of clusters

select the pair of clusters with the highest affinity

merge the clusters

termination criteria holds

y

n

assign each node to adifferent cluster

Application: processor assignment (load

balancing – minimize interprocess communication)

scheduling (minimize critical path)

HW/SW partitioning

Clustering may be employed as part of the optimization process, i.e. combined with other techniques


Clustering

probabilistic deterministic

hierarchical partitioning

A node belongs to exactly one cluster or not

Each node belongs with certain probabilities to different clusters

Starts with given number of K clusters (independent from nodes)

Starts with a distance matrix of each pair of nodes

Termination after a given number of iterations

Termination after all nodes belong to one cluster

Results depend on the chosen initial set of clusters

Exact method: always the same result


Hierarchical Clustering

Replace the selected pair in distance matrix by a cluster representative

Recompute distance matrix

All nodes in one cluster

y

n

Determine the distance between each pair of

nodes

Select the smallest distance

Dendrogramm

Stepwise reduction of the number of clusters

Algorithm is kind of subsequent merger of nearest neighbors (nodes/clusters)


Hierarchical Clustering

Dendrogramm

Algorithm is kind of subsequent merger of nearest neighbors (nodes/clusters)


Partitioning Clustering (k-means)

Recompute positions of the cluster representative

Based on the positions of the nodes in each cluster

Number of iterations reached

y

n

Choose positions of k initial cluster representative

assign each node to the nearest cluster representative


Clustering – Application to Load Balancing

compute the sum of the communication cost

between any pair of clusters

select the pair of clusters with the highest comm. cost that does not violatethe capacity constraints

merge the clusters

reduction of comm. cost without violation of constraints

possiblen

y

assign each node to adifferent cluster

Optimization goal: minimize inter-process (inter-

cluster) communication limit maximum load per processor

(cluster) to 20


Clustering – Application to Load Balancing

7 5

76

92

10

18

4

3

54

76

921

8

4

3

54

12

6

21

4

3

54

16

12

6

21

4

3

54

16

12

21

3

8

16

18

9

16

20

76

921

8

4

3

54

12

21

3

8

16

18


Clustering – Variants

Distance Metrics Euclidean Manhattan Minkowsky Mahalanobis Jaccard

Clustering methods

Partitioning methods Hierarchical methods

k-meansFuzzy-c-meansSOMCliqueOne PassGustafson-Kessel algorithm

Agglomeration(bottom up)

Division(top down)

Single linkageComplete linkageAverage groupCentroidMSTROCK

WardsTree Structural Vector QuantificationMacnaughton-Smith algorithm

Camberra Chebychev Correlation Chi-square Kendalls‘s Rank

Correlation


Clustering – Hierarchical Algorithms

Single linkage

Centroid-based

Complete Linkage

Algorithms implement different methods to compute the distance between two clusters


Clustering – Single Linkage

0123456789

10

0 2 4 6 8 10

P1

P2

P3P4

P5

P6

P7 Distance between groups is estimated as the smallest distance between entities Example:

1.4,min 4545255)4,2( dddd

Cluster # P1 P2 P3 P4 P5 P6 P7

P1 0 7.2 5 7.1 6.1 9.2 7

P2 - 0 3 1.4 5.4 3 6.7

P3 - - 0 2.2 2.8 4.3 4.3

P4 - - - 0 4.1 2.2 5.4

P5 - - - - 0 5.1 1.4

P6 - - - - - 0 6

P7 - - - - - - 0


Clustering – Single Linkage

0123456789

10

0 2 4 6 8 10

P1

P2

P3P4

P5

P6

P7


P1 0 7.2 5 7.1 6.1 9.2 7

P2 - 0 3 1.4 5.4 3 6.7

P3 - - 0 2.2 2.8 4.3 4.3

P4 - - - 0 4.1 2.2 5.4

P5 - - - - 0 5.1 1.4

P6 - - - - - 0 6

P7 - - - - - - 0

Cluster # P1 C24 P3 C57 P6

P1 0 7.1 5 6.1 9.2

C24 - 0 2.2 4.1 2.2

P3 - - 0 2.8 4.3

C57 - - - 0 5.1

P6 - - - - 0

Cluster # P1 C243 C57 P6

P1 0 5 6.1 9.2

C243 - 0 2.8 2.2

C57 - - 0 5.1

P6 - - - 0


Clustering – Group Average

0123456789

10

0 2 4 6 8 10

P1

P2

P3P4

P5

P6

P7


P1 0 7.2 5 7.1 6.1 9.2 7

P2 - 0 3 1.4 5.4 3 6.7

P3 - - 0 2.2 2.8 4.3 4.3

P4 - - - 0 4.1 2.2 5.4

P5 - - - - 0 5.1 1.4

P6 - - - - - 0 6

P7 - - - - - - 0

Distance between groups is definedas the average distance between all pairs of entitiesExample:

8.421

45255)4,2( ddd


Clustering – Group Average

0123456789

10

0 2 4 6 8 10

P1

P2

P3P4

P5

P6

P7


P1 0 7.2 5 7.1 6.1 9.2 7

P2 - 0 3 1.4 5.4 3 6.7

P3 - - 0 2.2 2.8 4.3 4.3

P4 - - - 0 4.1 2.2 5.4

P5 - - - - 0 5.1 1.4

P6 - - - - - 0 6

P7 - - - - - - 0

Cluster # P1 C24 P3 C57 P6

P1 0 7.2 5 6.6 9.2

C24 - 0 2.6 4.5 2.6

P3 - - 0 3.6 4.3

C57 - - - 0 5.6

P6 - - - - 0

Cluster # P1 C243 C57 P6

P1 0 6.4 6.1 9.2

C243 - 0 4.8 2.5

C57 - - 0 5.1

P6 - - - 0


Clustering – Centroid-based

0123456789

10

0 2 4 6 8 10

P1

P2

P3P4

P5

P6

P7 Determine distances between centroids (k,l)Merge centroids with the least distance

22),(lklk yyxx CCCClkd


P1 0 7.2 5 7.1 6.1 9.2 7

P2 - 0 3 1.4 5.4 3 6.7

P3 - - 0 2.2 2.8 4.3 4.3

P4 - - - 0 4.1 2.2 5.4

P5 - - - - 0 5.1 1.4

P6 - - - - - 0 6

P7 - - - - - - 0

xx

x

x


Clustering – Centroid-based

0123456789

10

0 2 4 6 8 10

P1

P2

P3P4

P5

P6

P7


P1 0 7.2 5 7.1 6.1 9.2 7

P2 - 0 3 1.4 5.4 3 6.7

P3 - - 0 2.2 2.8 4.3 4.3

P4 - - - 0 4.1 2.2 5.4

P5 - - - - 0 5.1 1.4

P6 - - - - - 0 6

P7 - - - - - - 0

xx

x

x

Cluster # C1 C24 C3 C57 C6

C1 0 7.1 5 6.5 9.2

C24 - 0 2.5 5.4 2.5

C3 - - 0 3.5 4.3

C57 - - - 0 5.5

C6 - - - - 0


Differences between Clustering Algorithms

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Single Linkage

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Complete Linkage

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Centroid Linkage

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

K-means

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Ward

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Single Linkage

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Complete Linkage

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Centroid Linkage

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

K-means

-3 -2 -1 0 1 2 3

x 104

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5x 104

X (m)

Y (m

)

Ward


Clustering – Discussion

Results Exact results (single linkage) Not-exact results often several iterations are necessary (K-means)

Metrics Strong impact to clustering results Not each metric is suitable for each clustering algorithm Decision for one- or multi-criteria metrics (separated or joint clustering)

Selection of Algorithm Depends strongly on the structure of the data set and the expected results

Some algorithms tend to separate outlayers in own clusters some large clusters and a lot of very small clusters (complete linkage)

Only few algorithms are able to detect also branched, curved or cyclic clusters (single linkage)

Some algorithms tend to return clusters with nearly equal size (K-means, Ward)

Quality of clustering results The mean variance of the elements in each cluster (affinity parameter) is often

used In general the homogeneity within clusters and the heterogeneity between clusters

can be measured However, the quality prediction can be only as good as the quality of the used

metric!


Branch and Bound with Underestimates

Application of the A* algorithm to the scheduling problem

Example: scheduling on a 2-processor system (processor A and B)Process graph communication || processing

Legend:green: processing timesblue: communication times

1

2 3

4

5

98

3

2 5

6 1

f(x) = g(x) + h(x)g(x) exact value of partial scheduleh(x) underestimate for remainder(min (altern.rem.proc, rem.comm. + rem.proc.)

Search is terminated when min {f(x)} is a terminal node (in the search tree)



Example: computation of f(3)

1

2 3

4

5

98

3

2 5

6 1

1 -> Af(1)=17

1 1 -> Bf(2)=17

2

2 -> Af(3)=22

3

44

case 1: path 1-2-4g(x) = 5 + 8 = 13h(x) = 3f(x) = 16

AB

1 2

0 4 8 12 16 20 24

3->4

1->33

3 44A

B1 2

0 4 8 12 16 20 24

case 2: path 1-3-4g(x) = 5h(x) = 5 + 9 + 3f(x) = 22

f(x) = g(x) + h(x)g(x) exact value of partial

scheduleh(x) underestimate for

remainder



Application of the A* algorithm to the scheduling problem

Example: scheduling on a 2-processor system (processor A and B)Process graph Search Tree

Legend:green: processing timesblue: comm. times

1

2 3

4

5

98

3

2 5

6 12 -> Af(3)=22

3 2 -> Bf(4)=18

4 3 -> Bf(6)=22

63 -> Af(5)=18

5

2 -> Af(7)=25

7 2 -> Bf(8)=18

8

4 -> Af(9)=24

9 4 -> Bf(10)=18

10

1 -> Af(1)=17

1 2 1 -> Bf(2)=17

f(x) = g(x) + h(x)g(x) exact value of partial

scheduleh(x) underestimate for

remainder

Search is terminated when min {f(x)} is a terminal node (in the search tree)


References

A. Mitschele-Thiel: Systems Engineering with SDL – Developing Performance-Critical Communication Systems. Wiley, 2001. (section 2.5)

C.R. Reeves (ed.): Modern Heuristic Techniques for Combinatorial Problems. Blackwell Scientific Publications, 1993.

H.U. Heiss: Prozessorzuteilung in Parallelrechnern. BI-Wissenschaftsverlag, Reihe Informatik, Band 98, 1994.

M. Garey, D. Johnson: Computer and Intractability. W.H. Freeman, New York, 1979.

Documents

Markov Chain Analysis – Example - Startseite TU Ilmenau · Integrated HW/SW-Systems Andreas Mitschele-Thiel 2-Feb-11 1 Markov Chain Analysis – Example Example: single server with