CS 584

Review

Systems of equations and finite element Systems of equations and finite element methods are related.methods are related.

Gauss-Seidel Iteration

Gauss-Seidel parallelism can be Gauss-Seidel parallelism can be increased by using graph coloring.increased by using graph coloring.

Partitioning

P0 P1

P2 P3

Communication

P0 P1

P2 P3

Finite Element Method

Used for deriving approximate Used for deriving approximate numerical solutions to partial differential numerical solutions to partial differential equations over a discretized domain.equations over a discretized domain.

Two sources of errorTwo sources of error– numerical (due to approximation)numerical (due to approximation)– discretizationdiscretization

Finite Element Method

Domain is boundedDomain is bounded Some grid is imposed on the domainSome grid is imposed on the domain Equations are generated to indicate the Equations are generated to indicate the

relationships between grid pointsrelationships between grid points Jacobi or Gauss-Seidel iteration is used Jacobi or Gauss-Seidel iteration is used

to to relaxrelax the system and converge to the the system and converge to the solutionsolution

The Heat Equation

The steady-state temperature u at any The steady-state temperature u at any point (x,y) on a metal sheet is governed point (x,y) on a metal sheet is governed by:by:

u02

2

2

2

yx

u

Discretization allows us to transform this Discretization allows us to transform this into a system of linear equations which into a system of linear equations which will approximate the partial derivatives.will approximate the partial derivatives.

The Finite Element Method

For most applications, the finite element For most applications, the finite element graph is not a regular structure.graph is not a regular structure.

The corresponding system of equations The corresponding system of equations is, most often, very large and very sparse.is, most often, very large and very sparse.

Efficient parallelization of the system Efficient parallelization of the system depends heavily on the way the domain is depends heavily on the way the domain is partitioned among processors.partitioned among processors.

Partitioning Techniques

Regular gridsRegular grids– One dimensional stripingOne dimensional striping– Two dimensional blockingTwo dimensional blocking

Generalized GraphsGeneralized Graphs– LevelizationLevelization– Scattered DecompositionScattered Decomposition– Recursive BisectionRecursive Bisection

Levelization

Begin with a boundaryBegin with a boundary– Number these nodes 1Number these nodes 1

All nodes connected to a level 1 node All nodes connected to a level 1 node are labeled 2, etc.are labeled 2, etc.

Partitioning is performedPartitioning is performed– determine the number of nodes per processordetermine the number of nodes per processor– count off the nodes of a level until exhaustedcount off the nodes of a level until exhausted– proceed to the next levelproceed to the next level

Levelization

Levelization

Want to insure nearest neighbor comm.Want to insure nearest neighbor comm. If If pp is # processors and is # processors and nn is # nodes. is # nodes. Let Let rrii be the sum of the number of be the sum of the number of

nodes in contiguous levels i and i + 1nodes in contiguous levels i and i + 1 Let Let r = r = maxmax{r{r11, r, r22, … , r, … , rnn}} Nearest neighbor communication is Nearest neighbor communication is

assured if assured if n/p > rn/p > r

Scattered Decomposition

Used for highly irregular gridsUsed for highly irregular grids Partition load into a large number Partition load into a large number rr of of

rectangular clusters such that rectangular clusters such that r >> pr >> p Each processor is given a disjoint set of Each processor is given a disjoint set of

r/pr/p clusters. clusters. Communication overhead can be a Communication overhead can be a

problem for highly irregular problems.problem for highly irregular problems.

Recursive Bisection

Recursively divide the domain in two Recursively divide the domain in two pieces at each step.pieces at each step.

3 Methods3 Methods– Recursive Coordinate BisectionRecursive Coordinate Bisection– Recursive Graph BisectionRecursive Graph Bisection– Recursive Spectral BisectionRecursive Spectral Bisection

Recursive Coordinate Bisection

Divide the domain based on the physical Divide the domain based on the physical coordinates of the nodes.coordinates of the nodes.

Pick a dimension and divide in half.Pick a dimension and divide in half. RCB uses no connectivity informationRCB uses no connectivity information

– lots of edges crossing boundarieslots of edges crossing boundaries– partitions may be disconnectedpartitions may be disconnected

Some new research based on Some new research based on graph graph separatorsseparators overcomes some problems. overcomes some problems.

Recursive Graph Bisection

Based on graph distance rather than Based on graph distance rather than coordinate distance.coordinate distance.

Determine the two furthest separated Determine the two furthest separated nodesnodes

Organize and partition nodes according to Organize and partition nodes according to their distance from extremities.their distance from extremities.

Computationally expensiveComputationally expensive– Can use approximation methods.Can use approximation methods.

Recursive Spectral Bisection

Uses the discrete Laplacian Uses the discrete Laplacian Let A be the adjacency matrixLet A be the adjacency matrix Let D be the diagonal matrix whereLet D be the diagonal matrix where

– D[i,i] is the degree of node ID[i,i] is the degree of node I

LLGG = A - D = A - D

Recursive Spectral Bisection

LG is negative semidefiniteLG is negative semidefinite Its largest eigenvalue is zero and the Its largest eigenvalue is zero and the

corresponding eigenvector is all ones.corresponding eigenvector is all ones. The magnitude of the second largest The magnitude of the second largest

eigenvalue gives a measure of the eigenvalue gives a measure of the connectivity of the graph.connectivity of the graph.

Its corresponding eigenvector gives a Its corresponding eigenvector gives a measure of distances between nodes.measure of distances between nodes.

Recursive Spectral Bisection

The eigenvector corresponding to the The eigenvector corresponding to the second largest eigenvalue is the Fiedler second largest eigenvalue is the Fiedler vector.vector.

Calculation of the Fiedler vector is Calculation of the Fiedler vector is computationally intensive.computationally intensive.

RSB yields connected partitions that are RSB yields connected partitions that are very well balanced.very well balanced.

RCB 529 edges cut RGB 618 edges cut

RSB299 edges cut