Usable Synthesis

Sumit Gulwani(sumitg@microsoft.com)

Microsoft Research, Redmond

Usable Verification WorkshopNovember 2010MSR Redmond

• What is Program Synthesis?– Synthesize an executable program from user intent

expressed in form of some constraints.

2

Automated Program Synthesis

• Why today?– Natural goal given that computing has become

accessible, but:• fundamental “how” programming models have not

changed. • most people are not expert programmers.

– Enabling technology is now available• Better search/logical reasoning techniques (SAT/SMT

solvers)• Faster machines (good application for multi-cores)

• State of the art: We can synthesize 10-20 lines of code.

• Technical Dimensions– Search Space– Search Technique

• Usability Dimensions– Impact

• Who will find it useful?• How useful would it be?

– Intent Expression Mechanism• How will they express their intent?

3

Dimensions in Program Synthesis

• Search Space: Straight-line programs that use – Arithmetic Operators: +,-,*,/– Logical Operators: Bitwise and/or/not, Shift left/right

• Search Algorithm: SAT/SMT based techniques

4

Application 1: Bitvector Algorithms

Joint work with: Susmit Jha, Sanjit Seshia (UC-Berkeley), Ashish Tiwari (SRI) and Venkie (MSR Redmond)

Paper: ICSE 2010

1 0 1 0 1 1 0 0

Turn-off rightmost 1-bit

5

Examples of Bitvector Algorithms

1 0 1 0 1 1 0 0

1 0 1 0 1 0 0 0

Z

Z & (Z-1)

1 0 1 0 1 0 1 1

Z

Z-1

1 0 1 0 1 0 0 0

&

Z & (Z-1)

6

Examples of Bitvector Algorithms

Turn-off rightmost contiguous sequence of 1-bits

Z

Z & (1 + (Z | (Z-1)))

1 0 1 0 1 1 0 0 1 0 1 0 0 0 0 0

Ceil of average of two integers without overflowing

(Y|Z) – ((Y©Z) >> 1)

7

Examples of Bitvector Algorithms

P25: Higher order half

of product of x and yo1 := and(x,0xFFFF);o2 := shr(x,16);o3 := and(y,0xFFFF);o4 := shr(y,16);o5 := mul(o1,o3);o6 := mul(o2,o3);o7 := mul(o1,o4);o8 := mul(o2,o4);o9 := shr(o5,16);o10 := add(o6,o9);o11 := and(o10,0xFFFF);o12 := shr(o10,16);o13 := add(o7,o11);o14 := shr(o13,16);o15 := add(o14,o12);res := add(o15,o8);

P24: Round up to next highest power of 2

o1 := sub(x,1);o2 := shr(o1,1);o3 := or(o1,o2);o4 := shr(o3,2);o5 := or(o3,o4);o6 := shr(o5,4);o7 := or(o5,o6);o8 := shr(o7,8);o9 := or(o7,o8);o10 := shr(o9,16);o11 := or(o9,o10);res := add(o10,1);

 

Algorithm Designers

• Impact: Algorithm Designers or Software Developers

• Intent Expression Mechanism: Logical Specifications

Consumers of Program Synthesis Technology

Bitvector Algorithms: Usability Dimensions

8

Software Developers

• Search Space: Programs with conditionals/loops– String operations: Concatenate, Substring– Logical operations: comparison involving # of

occurrences of a regular expression

• Search Algorithm: Combination of – Version Space Algebras – Machine Learning

9

Application 2: String Manipulation Macros

Joint work with: Bill Harris (UW, Madison), Rishabh Singh (MIT)Paper: POPL 2011

 End-Users

Algorithm Designers

Software Developers

Consumers of Program Synthesis Technology

String Manipulation Macros: Usability Dimensions

• Impact: End-users

• Intent Expression Mechanism: Input-output Examples

10

Most Useful Target

11

String Manipulation Macros

Excel Demo

1. Identify tasks that end-users struggle with and identify how they can effectively communicate their intent.– Read help-forums and blogs.– Interview real users.

2. Design a language that satisfies the following trade-off.– Expressive enough to express a lot of tasks.– Small enough to allow efficient learning.

3. Design a learning algorithm with following features.– Interactive with fast convergence (with success or

failure).– Provide feedback.– Noise tolerant.

12

Methodology: Automating end-user programming

• Search Space: Straight-line programs– Operations: Ruler, Compass

• Search Algorithm: Exhaustive Search– Property Testing– Goal-directed search– Commonly used library of constructions

13

Application 3: Geometry Constructions

Joint work with: Kalika Bali, Monojit Chaudhuri (MSR Bangalore) Vijay Korthikanti (UIUC), Ashish Tiwari (SRI)

Ken McMillan (MSR Redmond)

• Programming Language for Geometry– Objects: Point, Line, Circle– Constructors

• Ruler(Point, Point) -> Line • Compass(Point, Point) -> Circle• Intersect(Circle, Circle) -> Pair of Points• Intersect(Line, Circle) -> Pair of Points• Intersect(Line, Line) -> Point

• Logic for Geometry– Inequality predicates over arithmetic expressions

• Distance(Point, Point), Angle(Line, Line), …• Automated Problem Solving

– Given pre/postcondition, synthesize a straight-line program

14

Geometry Constructions: Role of PL/logic/synthesis

Automated Problem Solving• Given pre/postcondition, synthesize a straight-line

program

Example: Draw a line L’ perpendicular to a given line L.Precondition: truePostcondition: Angle(L’,L) = 90

ProgramStep 1: P1, P2 = ChoosePoint(L);Step 2: C1 = Circle(P1,P2);Step 3: C2 = Circle(P2,P1);Step 4: <P3, P4> = Intersect(C1,C2);Step 5: L’ = Line(P3,P4); 15

Geometry Domain: Automated Problem Solving

16

Constructing line L’ perpendicular to given line L

P1 P2

P3

P4

C1C2

L

L’

Step 1: P1, P2 = ChoosePoint(L);Step 2: C1 = Circle(P1,P2);Step 3: C2 = Circle(P2,P1);Step 4: <P3, P4> = Intersect(C1,C2);Step 5: L’ = Line(P3,P4);

• Bisect a given line.• Bisect an angle.• Copy an angle.• Draw a line parallel to a given line.• Draw an equilateral triangle given two points.• Draw a regular hexagon given a side.• Given 4 points, draw a square with each of the

sides passing through a different point.

Other Applications: • New approximate geometric constructions• 2D/3D planning problems

17

Examples of Geometry Constructions

 

Students and Teachers

End-Users

Algorithm Designers

Software Developers

Consumers of Program Synthesis Technology

Geometry Constructions: Usability Dimensions

• Impact: Students and Teachers

• Intent Expression Mechanism: Natural Language

18

Most Trans-formational Target

Most Useful Target

Automating Education

Make education interactive and fun

• Automated problem solving (for students)– Provide hints– Point out mistakes and suggest fixes

• Creation of teaching material (for teachers)– Authoring tools– Problem construction

• Group interaction (for teachers/students)– Ask questions– Share annotations

Domains: Geometry, Algebra, Probability, Mechanics, Electrical Circuits, etc. 19

 

Students and Teachers

End-Users

Algorithm Designers

Software Developers

Most Trans-formational Target

Usability Dimensions

20

Consumers of Program Synthesis Technology

Most Useful Target

Logic

Examples

English

Impact User Interface

• Dimensions in Program Synthesis– Invited Tutorial at FMCAD 2010– Invited paper at PPDP 2010

• Bitvector Algorithms– “Oracle guided component based program synthesis”,

ICSE 2010, Jha/Gulwani/Seshia/Tiwari

• String Manipulation Macros – “Automating String Processing in Spreadsheets using

Input-Output Examples”, POPL 2011, Gulwani

• Geometry Constructions– “Synthesizing Geometry Constructions”,

Techreport 2011, Gulwani/Korthikanti/Tiwari21

References