Nonlinear OptimizationTechniques Applied to
zur Erlangung des akademischen GradesDoktorin der Technischen Wissenschaften
Alpen-Adria- Universi tt Klagenfurt
Fakultt fr Wirtschaftswissenschaften und Informatik
1. Begutachter: Univ.-Prof. Dipl.-Ing. Dr. Franz RendlInstitut fr Mathematik
2. Begutachterin: Ao. Univ.-Prof. Dipl.-Ing. Dr. Christine NowakInstitut fr Mathematik
Ehrenwrtliche ErklrungIch erklre ehrenwrtlich, dass ich die vorliegende Schrift verfasst und die mit ihrunmittelbar verbundenen Arbeiten selbst durchgefhrt habe. Die in der Schriftverwendete Literatur sowie das Ausma der mir im gesamten Arbeitsvorgang ge-whrten Untersttzung sind ausnahmslos angegeben. Die Schrift ist noch keineranderen Prfungsbehrde vorgelegt worden.
Klagenfurt, Oktober 2006
Combinatorial Optimization and Semidefinite Programming are two research top-ics that have attracted the attention of many mathematicians and computer sci-entists during the past two decades. Remarkable results have been achieved inboth fields. This thesis is a further component in exploring the field of Semidefi-nite Programming and investigating Combinatorial Optimization problems.
Due to the various areas of application, one research topic of high interestis the development of algorithms for solving Semidefinite Programs. Althoughreliable methods are already available and widely used these algorithms are ofteninapplicable for large-scale programs, due to the huge memory requirements orthe vast computational effort. The present work proposes methods (and imple-mentations) that are capable of solving Semidefinite Programs of high dimensionsand/or a large number of constraints. These methods are: the Bundle Methodapplied to solve Semidefinite Programs, the Spectral Bundle Method with secondorder information, and the Boundary Point Method.
Exploiting the concept of Bundle Methods allows solving problems, even ifthe number of .constraints is rather large. By the use of Lagrange multipliers, theconstraints (or some of them) are lifted into the objective function and the dualproblem is then solved following the concept of Bundle Methods.
In the Spectral Bundle Method the largest Eigenvalue max of a matrix isminimized. Since the second-order behavior of the max function is well studied,it can be incorporated in this method. Making partial use of this second-orderinformation improves the efficiencyof the Spectral Bundle Method while keepingit computationally practical.
Another new algorithm for solving Semidefinite Programs is the BoundaryPoint Method. This is an augmented Lagrangian algorithm applied to solveSemidefinite Programs. For various problem classes this method is by far superiorto other available algorithms.
Regarding applications, the main focus of this study is on the Max-Cut prob-lem, one of the most challenging Combinatorial Optimization problems. Theapplicability of this problem is even broader than obvious at first sight, since anyunconstrained quadratic (0-1) problem can be transformed to a Max-Cut prob-lem. Apart from recalling the properties of the problem and giving a survey ofrelaxations and known solution methods, new relaxations based on Semidefinite
Programming are introduced. Finally, "Biq Mac" was developed, a solver forbinary quadratic and Max-Cut problems. Biq Mac is an implementation of anexact solution method using a Branch & Bound algorithm with a bounding rou-tine based on Semidefinite Programming. Detailed information on this algorithm,as well as a collection of test problems together with numerical results, can befound in the present thesis. Various test problems that have been considered inthe literature for years, were solved by Biq Mac for the first time. This affirmsthe success of using Semidefinite Programming for Combinatorial Optimizationproblems.
I am grateful to a number of people who have supported me in the developmentof this work and it is my pleasure now to highlight them here.
I want to thank my supervisor Franz Rendl for introducing me into the fieldof Semidefinite Programming, for his enthusiasm about discussing mathematicalissues and for the large amount of time he devoted to my concerns. His ideas andadvice led me into active research and substantiated my thesis.
The Mathematics Department at the Alpen-Adria-Universitt Klagenfurt pro-vided excellent working conditions. I would like to thank my colleagues at thedepartment and especially Christine Nowak. She served as a member of my Ph.D.committee and she was willing to encourage me at any time.
I also want to thank Giovanni Rinaldi for inviting me to IASI-CNR in Rome.Due to him and the people in his group my research stays in Rome were veryfruitful and enjoyable. Furthermore, I gratefully acknowledge financial supportfrom the EU project Algorithmic Discrete Optimization (ADONET), MRTN-CT-2003-504438. Participation at various conferences and workshops, as well asthe research stay at IASI-CNR in Rome were financed by this research trainingnetwork.
Having people around to have fun with, to discuss whatever has to be dis-cussed, to fill my spare time with joyful events, but also to understand thatsometimes 'spare time' is negligibly small, is very important for me. I am grate-ful for being surrounded by such friends and for the many occasions that provedhow precious they are to me.
Above all, my thanks go to my family. Although I was away quite often, Ialways had a hearty welcome when returning home. I want to dedicate this workto them.
1 Semidefinite Programming1.1 The Semidefinite Programming Problem1.2 Duality Theory . . . . .1.3 Eigenvalue Optimization .1.4 On Solving Semidefinite Programming Problems
1.4.1 Interior-Point Methods .1.4.2 Spectral Bundle Method . . . . . . . . .1.4.3 Software for Solving Semidefinite Programs.
2 Combinatorial Optimization2.1 The Max-Cut Problem . . .2.2 The Stable Set Problem ..2.3 The Graph Partitioning Problem2.4 The Max-Sat Problem . . .
3 The Maximum Cut Problem3.1 Properties of the Max-Cut Problem . . . . . . . . .3.2 Quadratic (0-1) Programming and Relation to MC
3.2.1 (QP)~ (MC) .3.2.2 (MC)~ (QP) .3.2.3 (MC) vs. (QP) .
3.3 Relaxations of the Max-Cut Problem3.3.1 Relaxations Based on Linear Programming.3.3.2 A Basic SDP Relaxation . . .3.3.3 Convex Quadratic Relaxations .3.3.4 Second-Order Cone Programming Relaxations
3.3.5 Branch & Bound with Preprocessing 333.4 A Rounding Heuristic Based on SDP . . . . 34
4 SDP Relaxations of the Max-Cut Problem 354.1 The Basic Relaxation. . . . . . . . 354.2 Strengthening the Basic Relaxation . . . . . 364.3 Lift-and-Project Methods 37
4.3.1 The Lifting of Anjos and Wolkowicz . 374.3.2 The Lifting of Lasserre . . . . . . . . 41
4.4 Between the Basic Relaxation and a First Lifting 424.4.1 Exploiting Sparsity . . . . . . . . . . . 434.4.2 Systematically Chosen Submatrices . . 464.4.3 Numerical Results of (MCSPARSE) . 47
5 Algorithms for solving large-scale SDPs 495.1 The Bundle Method in Combinatorial Optimization 50
5.1.1 Solving (MCSPARSE) Using the Bundle Method 515.1.2 Solving (SDPMET) Using the Bundle Method 54
5.2 Spectral Bundle with 2nd Order Information . . . . 585.3 A Boundary Point Method . . . . . . . . . . . . . . 645.4 A Recipe for Choosing the Proper Solution Method 68
6 Biq Mac 716.1 A Branch & Bound Framework for (MC) . 716.2 Branching Rules. . . 73
6.2.1 Easy First . . . . 746.2.2 Difficult First . . 746.2.3 A Variant of R3 . 746.2.4 Strong Branching 74
6.3 Implementation of the Biq Mac Solver 776.4 The Biq Mac Library. . . 80
6.4.1 Max-Cut...... 836.4.2 Instances of (QP) . 84
6.5 Numerical Results. . . . . 856.5.1 Numerical Results of (MC) Instances. 886.5.2 Numerical Results of (QP) Instances 91
6.6 Extensions........... 926.6.1 The Bisection Problem . 926.6.2 Max-2Sat . . . . . . . . 936.6.3 The Stable Set Problem 946.6.4 The Quadratic Knapsack Problem. 95
6.7 Concluding Remarks on Biq Mac . . . . . 95
Summary and Outlook
A Background MaterialA.1 Positive Semidefinite MatricesA.2 Convexity, Minimax Inequality.
B Problem GenerationB.1 rudy Calls . . . . . . . . . . . . . . . . . . .B.2 Pardalos-Rodgers Generator Parameters ..
B.2.1 Parameters for the bqpgka InstancesB.2.2 Parameters for the bqpbe Instances
C Tables with Detailed Numerical Results
Index of Keywords
This is a short description of the symbols used throughout this thesis. Also thenames of the various (semidefinite) programs are given, including the numbers ofthe sections where they appear for the first time.
space of real n-dimensional vectorsspace of n x n symmetric matricesspace of n x n positive semidefinite matricesspace of n x n positive definite matricesspace of n x n negative semidefinite matricesspace of n x n negative definite matricesLwner partial orderminimum, minimizemaximum, maximizeinfimum
sup supremum\7 nabla operatora~J(Xl' ... 1 xn) partial derivativeA( .) linear operatorAT (.) adjoint of the linear operator A( .)trA trace of matrix A(A,B)I
(A, B) := tr(AT B)identity matrix of appropriate dimensionidentity matrix of dimension nmatrix of