Genetic Algorithms for Graph Partitioning and Incremental Graph Partitioning

8/2/2019 Genetic Algorithms for Graph Partitioning and Incremental Graph Partitioning

1/9

Genetic Algorithms for Graph Partitioning and Incremental GraphPartitioning

Harpal Maini Kishan Mehrotra Chilukuri Mohan Sanjay R a wSchool of C om pute r and Information Science, 4-116 CSTSyracuse University, Syracuse, NY 13244-410 0email: hsmaini/kishan/mohan/[email protected]

AbstractPartitioning graphs into equally large groups of

nodes, minimizing the number of edges between dif-ferent groups, is an extremely important problem inparallel computing. This paper presents genetic al-gorithms for suboptimal graph partitioning, with newcrossouer operators ( K N U X , DKNUX) hat lead to or-ders of magnitude improvement over traditional ge -netic operators in solution quality and speed. Ourmethod can improve on good solutions previously ob-tained by using other algorithms or graph theoreticheuristics in minimizing the total communication costor the worst case cost of communication for a singleprocessor. We also extend our algorithm to Incremen-tal Graph Partitioning problems, in which the graphstructure or system properties changes with time.

1 IntroductionGraph partitioning is the task of dividing the nodesof a graph into groups called partitions, in such a waythat each partition has roughly the same number ofnodes, and minimizing the cut-size, .e., the number ofedges that connect nodes in different partit ions. Thisproblem has important applications in parallel com-puting. For instance, efficiently parallelizing manyscientific and engineering applications requires parti-tioning da ta or tasks among processors, such that thecomputational load on each node is roughly the same,while inter-processor communication is minimized.'Partially supported by NSF grant CCR-9110812 and

DARPA contract #DABY63-91-C-0028. The contents of thispaper do not necessarily reflect the position or policy of theUnited States government, and no official endorsement shouldbe inferred.

Obtaining exact solutions for graph partitioningis computationally intractable, and several subopti-mal methods have been suggested for finding goodsolutions to the graph partitioning problem. Im-portant heuristics include recursive coordinate bisec-tion, recursive graph bisection, recursive spectral bi-section, mincut based methods, clustering techniques,geometry-based mapping, block-based spatial decom-position, and scattered decomposition [3, 11, 12, 151.Genetic algorithms (GA's) are stochastic state-space search techniques modeled on natural evolution-ary mechanisms 141. Th e population, a set of indi-viduals (potential solutions to the optimization prob-lem) steadily changes with time due to the applica-,tion of operators such as crossouer and mutation. Aselection process determines which individuals (fromamong parents and offspring) remain in the next gen-eration. Genetic algorithms have been used in thepast t o find good suboptimal solutions to t he graph-partitioning problem [ l,8, 5, 61.We present genetic algorithms for graph partition-ing, using new crossover operators that utilize infor-mation available from the history of genetic search.Our work is characterized by the following features:

1. Use of prior information to improve solu tions.2 . Efficient partitioning of graphs to which incre-

mental updates are made.3. Paralleiizability, using a distributed genetic algo-4.Refinement of partitions obtained by other meth-

ods.5 . Optimization of the worst case communication

cost, a non-differentiable unction.We have obtained excellent results due to newlydeveloped genetic recombination operators (KNUX

rithm model.

4491063-9535194 4.000 1994 EE E

Authorized licensed use limited to: ULAKBIM UASL - Bahcesehir University. Downloaded on March 9, 2009 at 16:44 from IEEE Xplore. Restrictions apply.
mailto:hsmaini/kishan/mohan/[email protected]:hsmaini/kishan/mohan/[email protected]


2/9

and DKNUX) that exploit domain-specific knowledge.These give improved solutions and faster convergencerates when compared with the traditional crossoveroperator s. Exact comparisons of the different algo-rithms are not available due to the unavailability ofbenchmark problems and results. However, our exper-iments with the traditional crossover operators usedby some of these researchers gave resul ts of lower qual-ity than using the operators presented in this paper.The results achieved by our methods are better orcomparable to the best known methods for graph par-titioning, for graphs with a few hundred nodes. Thequality of solutions obtained using DKNUX is com-petitive with recursive spectral bisection as a graphpartitioning strategy, especially for incremental graphpartit ioning. However, genetic algorithms do requiremuch more execution time than greedy algorithms,and are recommended in appl ications where the qual-ity of solution is important enough to warrant the ex-tr a computational effort. Fortunately, GAs are read-ily parallelizable, with near-linear speedups. Applyinga prior graph contraction st ep should precede the par-titioning of very large graphs using GAs.The rest of this section introduces notation. Sec-tion 2 describes how genetic algorithms are applied tothe graph partitioning problem. Experimental resultsfollow.1.1 Notation

Consider a graph G = (V ,E ) ,where V represents aset of vertices, E represents a set of undirected edges,the number of vertices is given by n = [VI,nd thenumber of edges is given by m = /El . The graph-partit ioning problem consists of finding an assignmentscheme M :V- that maps vertices to partitions.We denote by B(q) he set of vertices assigned to a par-tition q, i . e . , B(q)= {U E V M(v )= q } . For graphsrepresenting the computational structure of a physicaldomain, each vertex U, E V , 1 5 i 5 m correspondsto a physical coordinate in a d-dimensional space( x , ~x S 2 , . ., E d ) ,and each edge is a pair (vZ1 w t 2 ) .For such graphs, these edges connect physically prox-imate vertices.The weight w, orresponds to the computation cost(or weight) of the vertex w,. The cost of an edge~ ~ ( ~ 1 , 2 1 2 )s given by the amount of interaction be-tween vertices 211 and 212. Thus the weight of everypartition can be defined as

w ( q ) = C u , E B ( q )wt.The cost of all the outgoing edges from a partitionrepresent the total amount of communication cost andis given by

We would like to make an assignment such tha t thetime (W(q)+ PC(q)),spent by every node is mini-mized, where P represents the cost of unit computa-tion/cost of unit communication on a machine. Thisobjective is achieved by minimizing either

W(q )+ P C(q) ,9 9

which focuses on the total communication cost, or byminimizingW(q )+ P m y ( 4 ,

9

which focuses on the communication cost for th e worstpart ition. For domain decomposition methods, opti-mizing the latter function is more desirable. The for-mer has been traditionally used because most methodsrequire that the function being optimized is differen-tiable. Our methods work with either formulation.

2 Genetic Algorithms for Graph Par-titioningThis section describes the representation used tosolve the graph partitioning problem, the functionsbeing optimized, the genetic operators used, and vari-ous methods of improving th e performance of the G A.

2.1 RepresentationFor graph partitioning, we select a vector repre-sentation for each individual (candidate solution), inwhich the i t h element of an individual is j iff the i t hnode of the graph is allocated to the partition la-belled j . For instance, the string 11100011 representsthe mapping that assigns nodes 1,2,3,7,8 to partition(processor) 1 and nodes 4,5,6 to partition (processor)

0.2. 2 Fitness function

Fitness is a numerical quantity evaluated usingthe implied load imbalance and communication costs.If the graph is one in which the i t h node is adjacentto the (i + l)st ode for each i, then 11100011 wouldbe less fit than 11100001 (which is a more balancedpartition), but more fit than 10101011 (which has 6inter-partit ion edges). We use the following two fitnessfunctions, approximating W(q)by (IB(q)I- k)2 nd

450



3/9

approximating C(q) by the number of edges leadingout of partition q.LnFitness Function 1: c ( l B ( q ) l- ) z + , b x C ( y )

important to obtain a good, fast heuristic estimate ofa solution. DKNUX utilizes information inherent inthe history of the genetic search, and continually up-dates the estimate I to be the current best solution,using this to build the bias vector.2. 5 Distributed Population ModelL

nFitness Function 2: c ( IB (y ) I- ) +,bmqwC(y)Q2.3 Crossover

One-point crossover 141 works by selecting a site inchromosomes CY,^ and y6 to produce a6 and rP . Apopular generalization is 2-point crossover, in whichthe parents a/3r an d 664 produce offspring a c y and15/34. This has been further generalized to Ic-pointcrossover. In uniform crossower (UX) [14], the itcomponent of an offspring is chosen to be the same astha t of one of th e two parents, with equal probability.UX ignores the fact that one parent may have muchbetter genetic material than another, or that one re-gion of the search space is already known to produceindividuals of higher fitness than other regions. UXcan be described in terms of a bit-vector mask, eachbit of which determines the parent from which an off-spring inherits a value for a particular bit-position.Our new Knowledge-based Non- Unaform Crossoveroperator ( K N U X ) generalizes this idea, using a bias

probability vector p = ( P I , . .,pn),where each p , is areal number E [0,1]. The value of each bias prob-ability p , depends on i , the relative fitness of theparent strings, and on problem-specific knowledge.Given p and the two parents, a = ( a l , . . ,a,) andb = ( b l , .. , n ) , the offspring c = (c l , . . ,c,) is ob-tained such that if a , = b,, then C, = a ,, else theprobability tha t cz = a , is p , .For graph partitioning, an initial candidate solutionI is first generated. Let v( i ) be the set of neighborsof node a in the graph under consideration. For anycandidate solution X , et #( i ,X , ) be the number ofnodes in .(a) that are allocated by I to partition X , .If a and b are the two parents, then we define

0.5 if #( i ,a, )+ #(a , b, I ) = 0otherwisei =

2.4 Dynamic KNUX (DKNUX)The quality of solutions obtained by KNUX de-pends on the quality of the heuristic estimate ( Iabove) used to derive bias probabilities. It is therefore

We use a coarse-grained, distributed-population ge-netic algorithm (DPGA), where individuals are dis-tributed into various subpopulations which may bephysically located on different processors configuredin some architecture (e.g., mesh). Crossovers are re-stricted to occur between members of the same sub-population. Each subpopulation periodically commu-nicates copies of its best individuals to its neighboringsubpopulations (situated on neighboring processors inthe parallel archi tecture); this is how genetic informa-tion is exchanged.2.6 Population Initialization

The initial population can be seeded with a pre-estimated heuristic solution such as that obtainedthrough an Index Based Partitioning scheme or theresults of recursive spectral bisection. In th e incremen-tal case, the previous partitioning can itself be used togenerate a good partitioning for the changed graph byrandomly assigning new graph nodes to various nodes,while at the same time ensuring tha t balance is main-tained.2. 7 Hill Climbing

It is possible to perform hill-climbing on offspring,to obtain the nearest local optima of the fitness func-tion. Only the boundary points of each partition(with neighbors in other partitions) are examined tosee if migrating them to t he appropriate neighboringpartition improves fitness.

3 Experimental ResultsIn this section, we compare the results obtained us-ing our approach with those of traditional heuristics(e.g., IBP or RSB) as well as with genetic algorithmstha t invoke traditional crossover operators. The fig-ures are obtained by averaging the results of 5 runs,and the tables represent the best solutions obtainedin these 5 runs. All experiments were done with algo-rithm DPGA set with a total population size of 320.The crossover rate p , = 0.7 and the mutation rate

45 1



4/9

p , = 0.01. Tables 1 , 2 and 3 report C , C ( q ) / 2 val-ues, while Tables 4, 5 and 6 report max, C(q)values,where C(q) is the number of edges that cut acrosspartition q.Experiments were conducted with a single popula-tion aswell as with 16 subpopulations configured as a

four dimensional hypercube. Graphs with unit weightnodes and edges were assumed, although weightededges and nodes can also be handled easily. For clar-ity, the cut-size numbers are given in the tables, in-stead of th e ac tua l fitness function values; for graph-partitioning, smaller cut-size numbers indicate supe-rior performance. Th e results establish very clearlythe excellent performance of KNUX and DKNUX incomparison with two-point crossover and also thatDKNUX is competitive with recursive spectral bisec-tion as a graph partitioning strategy.

Cut Using DKNUXCut Using RSB

3.1 Improving solutions obtained usingother met hods

36 78 13937 88 155

Fast heuristic algorithms can be used to obtain aninitial candidate solution which is then improved byapplying the genetic algorithm. Table 1 comparesthe results of Recursive Spectral Bisection (RSB)[ l l , 12, 131 with the GA initialized by a solutionobtained by the Index-Based Partitioning algorithm(IBP) [lo]described in the Appendix.

Number of Partitions16 7 NodesCut Using DKNUX

2 4 820 63 109

Cut Using RSB14 4 NodesCut Using DKNUXCut Using RSB 11 33 I 65 I 12 0I / 36 I 78 I 119

I20 59 120

Table 1: A Comparison of the Best Solutions foundUsing DKNUX and RSB: starting with a populationinitialized with an IBP solution, using Fitness Func-tion 1. In each case, the tot al number of inter-partitionedges is reported for the best individual explored bythe GA.

3.2 Incremental Graph PartitioningFor this series of experiments, we start with a

graph, part ition it , then modify by adding some num-ber of nodes in a local area chosen randomly withinthe graph. The modified graphs are then partitioned.

Number of Partitions 11 213 9 NodesCut Using RSB21 3 NodesCut Using RSB24 3 NodesCut Using DKNUX 43Cut Using RS B 4727 9 Nodes

vn

Table 2: Improving the Solution found through Recur-sive Spectral Bisection, using Fitness Function 1. Ineach case, the to tal number of inter-partition edges isreported for the best individual explored by the GA.

3.3 Minimizing Worst Case Communica-tion CostUnlike other methods which can be used onlywith a differentiable optimization function, ge-netic algorithms can be used to solve the originalproblem, directly optimizing 01C,(IB(q)I - i)2

P m a , Ciep,jeq e i j - Table 4 exhibits the effect ofpartitioning graphs of 78, 88, 98, 144 and 16 7 nodesinto 4 and 8 partitions. Table 4 shows the best solu-tion found using operator DKNUX is bette r tha n tha tobtained using RSB in most cases. In other cases,improvements can be obtained by seeding the initialpopulation with a heuristically obtained good solutionsuch as the index based partitioner.

4 ConclusionsWe have solved the graph partitioning problem us-ing GAs with new knowledge-based crossover opera-tors; problem-specific knowledge is used t o generatebias probabilities, and t he environment and currentpopulation play roles in controlling genetic expression.The trajectory that the population takes in searchspace is constrained, driving evolution in certain pre-ferred directions.We have introduced novel operators tha t exploit th elocality information inherent in most computational

graphs. We have shown this enhances the speed andperformance of genetic search by orders of magnitude.We have demonstrated t hat genetic algorithms can be

452



5/9

Number of Partitions78 plus 10 nodesWorst Cut Using DKNUX78 plus 20 nodesWorst Cut Using DKNUX11 8 plus 21 NodesWorst Cut Using DKNUXWorst Cut Using RSB11 8 plus 41 NodesWorst Cut Using DKNUXWorst Cut Using RSB18 3 plus 30 NodesWorst Cut Using DKNUXWorst Cut Using RSB18 3 plus 60 NodesWorst Cut Using DKNUXWorst Cut Using RSB24 9 plus 30 NodesWorst Cut Using DKNUXWorst Cut Using RSB24 9 plus 60 NodesWorst Cut Using DKNUXWorst Cut Using RSB

Worst Cut Using RSB

42733293 33834404146465142514646

Number of Partitions78 NodesWorst Cut Using DKNUXWorst Cut Using RSB88 NodesWorst Cut Using DKNUX98 NodesWorst Cut Using DKNUXWorst Cut Using RSB21 3 NodesWorst Cut Using DKNUXWorst Cut Using RSB24 3 NodesWorst Cut Using DKNUXWorst Cut Using RSB27 9 NodesWorst Cut Using DKNUX

Worst Cut Using RSB

a20

42326243324304046455142

252727293435394045454 74 44 75652

-.-.

-.

-

-.

-.

-.-Table 5: A Comparison of the Best Solutions found I

Worst Cut Using RSB30 9 Nodes

46

Using DKNUX: Improving Upon RSB Solutions UsingFitness Function 2. Worst Cut refers to maz,C(q),where C(q) s the number of edges leading out of par-tition q. For the G A , the maximum number of edgesleading out of a partition is reported, for the best in-dividual explored by the G A .

Table 6: A Comparison of the Best Solutions foundUsing DKNUX and RSB: Incremental Partitioningwith Fitness Function 2. Worst Cut refers tomaz,C(q),where C(q) s the number of edges leadingout of partition q. For th e G A , the maximum numberof edges leading out of a partition is reported, for thebest individual explored by th e G A .

Worst Cut Using DKNUXWorst Cut Using RSB

453

4446



6/9

1 Number of Partitions I( 2 I 4 I 8 U118 plus 41 Nodes

L -183 plus 30 NodesCut Using DKNUX 37 72 133Cut Using RSB 41 82 151

Cut Using DKNUXCut Using RSB )I 31 1 66 I 120(1 33 1 75 1 12 8

Table 3: A Comparison of the Best Solutions foundUsing DKNUX and RSB: Incremental Graph Parti-tioning, using Fitness Function 1. In each case, thetotal number of inter-partition edges is reported forthe best individual explored by the GA.

used to greatly refine previously estimated partitionswith the help of KNUX and DKNUX. We show howthe strategies discussed in this paper extend naturallyto incremental graph partitioning. The incrementalpartitioning results obtained using DKNUX could notbe obtained by a simple deterministic algorithm thatassigns new nodes to the partition to which most ofits nearest neighbors belong. Performance can furtherbe improved by incorporating a hill-climbing step.We have presented preliminary results showing thefeasibility of this approach and the gains obtainable byexamining the history of the search process; unfortu-nately, part itioning very large graphs does require high

amounts of computation by the genetic algorithm. Aprior graph contraction step would allow these tech-niques to be applied to graphs much larger than thoseexplored in this paper [13]. Some gains can be ex-pected from executing the GA on parallel comput-ers, since DPGA is an inherently parallel algorithmfrom which we can expect near-linear speedups. Weare currently parallelizing the algorithm to run on dis-tributed memory machines such as the CM-5 and theIntel Paragon.

ReferencesJ. P. Cohoon, W. N. Martin, and D. S.Richards, A multi-population genetic algo-rithm for solving the k-partition problem on

I Number of Partitions

Table 4: A Comparison of the Best Solutions foundUsing DKNUX and RSB: Starting with a RandomlyInitialized Population and Using Fitness Function 2.Worst Cut refers to maz,C(q), where C(q) is thenumber of edges leading out of partition q. For theGA, the maximum number of edges leading out of apartition is reported, for the best individual exploredby the GA.

hypercubes, Proc . 4 th ICGA , 1991, pp. 244-248.R. Collins and D. Jefferson, Selection inmassively parallel genetic algorithms, Proc.4th ICGA , 1991, pp. 249-256.F. Ercal, Heuristic Approaches to TaskAllocation for Parallel Computing, Ph.D.Thesis, Ohio State U niver sity, 1988.J . H. Holland, Adaptation in Natural andArtificial Systems, University of MichiganPress, Ann Arbor, 1975.D. R. Jones and M. A. Beltramo, Solv-ing partitioning problems with genetic algo-rithms, Proc. of the 4th ICGA, 1991, pp.442-450.Gregor von Laszewski, Intelligent structuraloperators for the k-way graph partitioningproblem, Proc. of the 4th ICGA, 1991, pp.45-52.H. S. Maini, Incorporation of Knowledgein Genetic Recombination, Ph.D . Thesis,

454



7/9

School of Computer & Information Science,Syracuse University, August 1994.[8] Nashat Mansour, Physical Optimization Al-

gorithms for Mapping Data to Distributed-Memory Multiprocessors, Ph.D. Thesis,School of Computer and Information Sci-ence, Syracuse University, 1992.

[9] H. Muhlenbein, Parallel genetic algorithms,population genetics and combinatorial opti-mization, Proc. 3rd ICGA, 1989, pp. 416-422.[ lo ] Chao-Wei Ou, Sanjay Ranka, and GeoffreyFox, Fast mapping and remapping algo-rithm for irregular and adaptive problems,

Proc. of the International Conference onParallel and Distributed Computing,Decem-ber 1993.

[ ll]A. Pothen, H. Simon, and K-P. Liou, Par ti-tioning sparse matrices with eigenvectors ofgraphs, SIAM J . Matrix Anal. Appl., 11, 3(July), 1990, pp. 430-452.

[12] H. Simon, Partitioning of unstructuredmesh problems for parallel processing, Proc.Conf. Parallel Methods on Large Scale Struc-tural Analysis and Physics Applications,Pergamon Press, 1991.

[13] S. T. Barnard, H . D. Simon, A fast mul-tilevel implementation of recursive spec-tral bisection for partitioning unstructuredproblems, Technical Report RNR-92-033,November 1992.[14] G . Syswerda, Uniform crossover in geneticalgorithms, Proc. of the 3rd ICGA, 1989,pp. 2-9.[15] R. D. Williams, Performance of dynamicload balancing algorithms for unstructuredmesh calculations, Concurrency: Practice

and Experience, 3(5 ), 1991, pp. 457-481.

Appendix: Index-Based Partition (IBP)Algorithm

Index-based algorithms to partition graphs havebeen described in [lo]. An IBP algorithm in-cludes three phases- indexing, sort ing, and coloring.The indexing scheme is based on converting an N -dimensional co-ordinate into a one-dimensional indexsuch that proximity in the multi-dimensional space ismaintained. Row-major indexing and shuffled row-major indexing are two of the several ways of indexingpixels in a two-dimensional grid. These two indexingschemes are shown in Figure 1 for a graph in whichthe set of vertices are arr anged in a grid of size 8 x 8.

00 01 02 03 04 05 06 07 00 01 04 05 16 17 20 2108 09 10 11 12 13 14 15 02 03 06 07 18 19 22 2316 17 18 19 20 21 22 23 08 09 12 13 24 25 28 2924 25 26 27 28 29 30 31 10 11 14 15 26 27 30 3132 33 34 35 36 37 38 39 32 33 36 37 48 49 52 5340 41 42 43 44 45 46 47 34 35 38 39 50 51 54 5548 49 50 5 1 52 53 54 55 40 41 44 45 56 57 60 6156 57 58 59 60 61 62 63 42 43 46 47 58 59 62 63

Figure 1: (a)Row-Major and (b) Shuffled Row-MajorIndexing for an 8 x 8 image

A simple example of interleaving indices is as fol-lows. Suppose index1 = 001, index2 = 010, andindex3 = 110. Then the interleaved index would be001011100. In the above case the number of bits ineach dimension are equal. This could easily be gen-eralized to cases when the sizes are different. For ex-ample if index1 = 101, index2 = 01, and index3 = 0,then the interleaved index would be 100110. This isdone by choosing bits (right to left) of each of thedimensions one by one, starting from dimension 3.When the bits of a particular dimension are no longeravailable, that dimension is not considered.After indexing is done, an efficient sorting algo-rithm can be applied to sort these vertices accordingto their indices. Finally, this sorted list is divided intoP equal sublists.

455



8/9

Two-Point -155 186

....DKNUXRSB(151) - - -

Two-Point -....- DKNUX..... ... -... ........................... .....140

I 1 I I

300 400 500200213t04

....... ......... .......................... - .....8 1 ...-.I ~76 t74 ' I I 1 t I

0 100 200 300 400 500213t02

I I I I

Two-Point -DKNUX ....

........... -. ......... -. ...................... -. ..-I360 100 200 300 400 500

GENERATIONS

160

155i;j- 150

145

N

6

243to8I I I ITwo-Point -

DKNUX ....RSB (154) - - -

........... ...... ....._...... ....I I I I

100 200 300 400 500140 243to40989694

c1 929088

blv).e

6

I I

I I I I

0 100 200 300 400 500243to2

RSB (47)48- _ _ - _ _ _ - _ _ _ _ - _ _ - - - - - - - - - - - - - - -

.e) 4 7 p , 1' 4 63

'tJ 0 100 200 300 400 500

GENERATIONSFigure 2: Partitioning a 213 node graph and a 24 3 node graph into 2, 4 and8 partitions: The effect of operators Two-Point and DKNUX on improvingsolutions obtained through RSB

456



9/9

280260240220200180160140120

d)G8

1601501 4 0 -130120110

159to8

- ":- :,,-

.... ..................... -.._ ........... .....-.--

Two-Point -RSB 128) - - -. ' '

I D y X .A

4030

I

- I * _ __ - - - - -____.*lr . .................. -. .................- I ,

0 100 200 300 400 500159104I I

Two-Point -00180 , DKNUX .... 1BSB(75) - - -160 1 1

..I I

139t08Two-Point -

180

Two-Point -

...... -I I I I

0 100 200 300 400 500GENERATIONS

Figure 3: Partitioning a 118 node graph incremented by 21 nodes and 41 nodes;into 2, 4 and 8 partitions: A comparison between Two-Point, DKNUX and RSB

451

Documents

Genetic Algorithms for Graph Partitioning and Incremental Graph Partitioning