Privacy-Preserving Linear Programming Olvi Mangasarian UW Madison & UCSD La Jolla UCSD –...

Privacy-Preserving Linear Programming

Olvi MangasarianUW Madison & UCSD La Jolla

UCSD – Center for Computational Mathematics SeminarJanuary 11, 2011

Problem Statement

• Entities with related data wish to solve a linear program based on all the data

• The entities are unwilling to reveal their data to each other– If each entity holds a different set of variables for all constraints,

then the data is said to be vertically partitioned– If each entity holds a different set of constraints with all variables,

then the data is said to be horizontally partitioned

• Our approach: privacy-preserving linear programming (PPLP) using random matrix transformations– Provides exact solution to the total linear program– Does not reveal any private information

Vertically Partitioned MatrixHorizontally Partitioned Matrix

A

A1

A2

A3

A¢1 A¢2 A¢3

Linear Programming Constraint Matrix

Variables1 2 ..………….…………. n

Constraints

12........m

Outline

• Vertically (horizontally) partitioned linear program

• Secure transformation via a random matrix

• Privacy-preserving linear program solution

• Computational results

• Summary

Vertically Partitioned Data:Each entity holds different variables for the

same constraints

A

¢1

A

¢3

A

¢2

A

¢1

A

¢2

A

¢3

LP with Vertically Partitioned Data

minx2X

c0x where X = fx j Ax ¸ bg

We consider the linear program:

whereA 2 Rm£ n , c 2 Rn

Thematrix£c0A

¤is divided into p vertical blocks

Each block is (m+1) £ nj , j = 1;2;: : : ;p,such that n1+n2+:::+np =n,is \ owned" by a distinct entity unwilling tomake it public.

Objective: Solve this linear programwithout revealing anyprivately held data.

Secure Linear Program Generation

Each of thep entities chooses its own privately heldrandommatrixB¢j 2 Rk£ n j , j = 1;::: ;p,wherek ¸ n. De ne: B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ n.Wenote immediately that the rank of the randommatrixB 2 Rk£ n with k ¸ n is n, which is the reasonfor choosing k ¸ n. Utilizing this fact wede nethe invertible transformation: x = B0u, andits least 2-norm inverse: u = B(B0B)¡ 1x:Wenow transformour original linear program intothe following \ secure" linear program:minu2U

c0B0u where U = fu j AB0u ¸ bg:

Why Secure Linear Program?

Weuse the term \ secure" to describe the transformedlinear programbecause it does not reveal any oftheprivately held data

£c0jA ¢j

¤, j = 1;: : : ;p. This is so

because for each entity di®erent fromentity j , it isimpossible to computeeither cj from the revealed productcj 0B¢j

0, or A¢j from the revealed product A¢j B¢j0

without knowing the randommatrix B¢j chosen byentity j and known to itself only.

Original & Secure LPs Are Equivalent

Let k ¸ n for the randommatrix B 2 Rk£ n .The secure linear programis solvable if and only iftheoriginal linear program is solvable in which casetheextrema of both linear programs areequal.

For onepageproof see:ftp:/ / ftp.cs.wisc.edu/ pub/ dmi/ tech-reports/ 10-01.pdfOptimization Letters, to appear.

PPLP Algorithm

1. All p entities agreeon a k ¸ n, thenumber of rowsof the randommatrix B 2 Rk£ n as de ned earlier.

2. Each entity generates its own privately held randommatrix B¢j 2 Rk£ n j , j = 1;: : :: : : ;p, wherenjis thenumber of features held by entity j . This resultsin B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ n :

PPLP Algorithm (Continued)

3. Each entity j makes public only itsmatrix productA¢j B¢j

0as well as its cost coe±cient product B¢j cj .Theseproducts do not reveal either A¢j or cj butallow thepublic computation of the full constraintmatrix needed for thesecure linear program:AB0=A¢1B¢1

0+A¢2B¢20+:::: : :+A¢pB¢p

0;as well as thecost coe±cient:c0B0= c01B¢1

0+c02B¢20+:::: : :+c0pB¢p

0:

PPLP Algorithm (Continued)

4. A public optimal solution vector u to thesecurelinear programand a public optimal objective functionvaluec0B0u arecomputed. This optimal valueequalstheoptimal objective function of theoriginal linearprogram.

5. Each entity computes its optimal xj component groupas follows:xj =B¢j

0u; j = 1;:: : ;p:

PPLP Algorithm (Continued)

6. Thesolution component vectors xj , j = 1;:: : ;p,are revealed by its owners if a public solution vector totheoriginal linear program is agreed upon. Else,the component vectorsmay bekept private if onlytheminimumvalue is needed, in which case thatminimumvalueequals thepublicly availableminimumvaluemin

u2Uc0B0u of the secure linear program.

Computatinal ResultsExample 1 (k=n=1000)

² Generated a randomsolvable linear programwith m= 100 and n = 1000

² Partitioned thecolumns of A as well as thecost vector cinto threegroups with n1 =500, n2 =300 and n3 =200

² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB¢1 2 Rn£ n1 , B¢2 2 Rn£ n2 andB¢3 2 Rn£ n3

² Solved secure linear programand compared its optimalobjectivevaluewith that of theoriginal linear program.

² The two optimal objectives agreed to 14 signi¯cant¯gures attained at two distinct optimal solution points.

Computatinal ResultsExample 2 (k=n=100)

² Generated a randomsolvable linear programwith m= 1000 and n = 100

² Partitioned thecolumns of A as well as thecost vector cinto threegroups with n1 =50, n2 =30 and n3 =20

² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB¢1 2 Rn£ n1 , B¢2 2 Rn£ n2 andB¢3 2 Rn£ n3

² Solved secure linear programand compared its optimalobjectivevaluewith that of theoriginal linear program.

² The two optimal objectives agreed to 13 signi¯cant¯gures attained at points that wereessentially thesame, that isthe1 -normof their di®erencewas less than 2:3e¡ 13.

Horizontally Partitioned Constraint Matrix:Entities hold different constraints with the same variables

A 1 A 2 A 3

A 3

A 2

A 1

LP with Horizontally Partitioned Data

minx2X

c0x where X = fx j Ax = b; x ¸ 0gWe consider the linear program:

whereA 2 Rm£ n , c 2 Rn

Here, [A b], is divided into p horizontal blocks ofm1;m2; : : : : : : and mp, (n+1)-dimensionalrows with m1+m2+:::+mp =m.Each block of rows of [A b] is \ owned" by a distinctentity that is unwilling tomake its block of data public.Objective: Solve this linear programwithout revealing anyprivately held data.

Secure Linear Program Generation

Each of thep entities chooses its own privately heldrandommatrix B¢i 2 Rk£ mi , i = 1;: : :;p, wherek ¸ m,which de nes: B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ m:Wenote immediately that the rank of the randommatrixB 2 Rk£ m with k ¸ m ism, which is the reasonfor choosing k ¸ m. Utilizing this fact wede nethe transformations: BA and Bb.Wenow transformour original linear program intothe following \ secure" linear program:miny2Y

c0y where Y = fy j BAy= Bb; y ¸ 0g:

Why Secure Linear Program?

Weuse the term \ secure" to describe the transformedlinear programbecause it does not reveal any of theprivatelyheld data: thehorizontal partitions of [A b].This is so because for each entity di®erent fromentity i,it is impossible to computeeither A i from the revealedproduct B¢iA i , or bi from the revealed product B¢ibiwithout knowing the randommatrix B¢i

chosen by entity i and known to itself only.

Original & Secure LPs Are Equivalent

Let k ¸ m for the randommatrix B 2 Rk£ m.The secure linear programis solvable if and only iftheoriginal linear program is solvable in which casetheextrema of both linear programs areequal.

Theproof follows from theobvious fact that:Ax = b ( ) BAx = Bb:For moredetails see:ftp:/ / ftp.cs.wisc.edu/ pub/ dmi/ tech-reports/ 10-02.pdf

PPHPLP Algorithm

1. All p entities agreeon a value for k ¸ m, wherekis thenumber of rows of the randommatrix B 2 Rk£ m

2. Each entity generates its own privately held randommatrix B¢i 2 Rk£ mi , i = 1;:: : : : : ;p,wheremi isthenumber of rows held by entity i which results in:B = [B¢1 B¢2 : : : : : :B¢p] 2 Rk£ m:

PPHPLP Algorithm (Continued)

3. Each entity i makes public only itsmatrix productB¢iA i as well as its right hand sideproduct B¢ibi .Theseproducts do not reveal either A i or bi but allowthepublic computation of the full constraint matrixneeded for thesecure linear program:BA = [B¢1A1+B¢2A2+:::+B¢pAp] 2 Rk£ n ;as well as the right hand side :Bb= [B¢1b1+B¢2b2+:::+B¢pbp] 2 Rk:

4. A public optimal solution vector y to thesecure linearprogram is obtained which also solves theoriginal linearprogram

Computatinal ResultsExample 1 (k=1000)

² Generated a randomsolvable linear programwith m= 600 and n = 1000

² Partitioned the rows of A as well as the right handsidevector b into threegroups with m1 = 100, m2 =200and m3 =300.

² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB1 2 Rk£ m1 , B2 2 Rk£ m2 and B3 2 Rk£ m3 .

² Solved secure linear programand compared its optimalsolution with that of theoriginal linear program.

² The two optimal solutions were identical.

Computatinal ResultsExample 2 (k=1000)

² Generated a randomsolvable linear programwith m= 1000 and n = 1000

² Partitioned the rows of A as well as the right handsidevector b into threegroups with m1 = 200, m2 =300and m3 =500.

² Generated three randommatrices, with coe±cientsuniformly distributed in the interval [0,1] withB1 2 Rk£ m1 , B2 2 Rk£ m2 and B3 2 Rk£ m3 .

² Solved secure linear programand compared its optimalsolution with that of theoriginal linear program.

² The two optimal solutions were identical.

Summary & Outlook

– Based on a transformation using a random matrix B

– Get exact solution to the original linear program without revealing privately held data

Possible extensions to: horizontally partitioned inequality constraints, complementarity problems and nonlinear programs

Privacy preserving linear programmingfor vertically or horizontally partitioned data

References

ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/10-01.pdfftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/10-02.pdf

Optimization Letters, to appear