Using Parallelism to Using Parallelism to Improve Theorem Improve Theorem Prover InteractivityProver Interactivity
David L. RagerDavid L. Rager
May 17, 2010May 17, 2010
Committee Members: Warren A Hunt Jr. (Chair), Matt Kaufmann, J Strother Moore, James C Browne, Emmett Witchel
22
Project GoalProject Goal
Reduce the latency between when a user Reduce the latency between when a user submits a conjecture and when the user submits a conjecture and when the user receives useful feedback concerning that receives useful feedback concerning that conjecture’s provabilityconjecture’s provability
33
OutlineOutline IntroductionIntroduction
The Automated Proof Process – the The Automated Proof Process – the WaterfallWaterfall The ways ACL2 currently uses parallelismThe ways ACL2 currently uses parallelism
Research ContributionsResearch Contributions Key StepsKey Steps
Removing sequential dependenciesRemoving sequential dependencies Introducing Parallelism into the Proof ProcessIntroducing Parallelism into the Proof Process
• Adding futures to the underlying implementation languageAdding futures to the underlying implementation language• Adding parallelism abstractions to the logic Adding parallelism abstractions to the logic • Using parallelism in the waterfallUsing parallelism in the waterfall
Managing outputManaging output Managing user interruptsManaging user interrupts
Evaluating the Soundness and Performance of our Evaluating the Soundness and Performance of our ApproachApproach
44
The Proof ProcessThe Proof Process Named “the waterfall”Named “the waterfall”
We hope to parallelize the application of all but the We hope to parallelize the application of all but the induction heuristicinduction heuristic
55
The Proof ProcessThe Proof Process
Each proof obligation that is not the original goal that Each proof obligation that is not the original goal that needs to go through the waterfall is called a subgoalneeds to go through the waterfall is called a subgoal
We parallelize at the subgoal level because:We parallelize at the subgoal level because: Highest level of granularity available without considering Highest level of granularity available without considering
parallelizing the proofs of theorems themselvesparallelizing the proofs of theorems themselves• Parallelizing the proofs of goals is thought to be much less usefulParallelizing the proofs of goals is thought to be much less useful
Already attempted parallelizing the rewriter without much Already attempted parallelizing the rewriter without much practical gainspractical gains
The waterfall is mostly functional The waterfall is mostly functional in nature, whereas the code in nature, whereas the code above the waterfall (which includes above the waterfall (which includes the induction code) tends to havethe induction code) tends to havemore side-effectsmore side-effects
66
Current Use of Parallelism in ACL2Current Use of Parallelism in ACL2
Process-level parallelismProcess-level parallelism GNU’s “make –j#” and cert.plGNU’s “make –j#” and cert.pl Certifies the regression suite in parallelCertifies the regression suite in parallel
Plet/pargs/pand/porPlet/pargs/pand/por User level parallelismUser level parallelism Useful for improving performance of proofs by Useful for improving performance of proofs by
simulationsimulation
77
Research ContributionsResearch Contributions
Maintain InteractivityMaintain Interactivity Continue to support ACL2 users’ ability to use Continue to support ACL2 users’ ability to use
the prover despite parallel execution in its the prover despite parallel execution in its proof processproof process
Mechanisms for Early FeedbackMechanisms for Early Feedback Provide feedback to the theorem prover user Provide feedback to the theorem prover user
asapasap Could result in super-linear speedupCould result in super-linear speedup
88
Research ContributionsResearch Contributions
Improve Support for Lisp-Level ProgrammingImprove Support for Lisp-Level Programming Interface that unifies multi-threading librariesInterface that unifies multi-threading libraries
Provide Parallelism AbstractionsProvide Parallelism Abstractions Enrich the built-in theory with primitives that allow Enrich the built-in theory with primitives that allow
parallel executionparallel execution• spec-mv-let, a better plet, etc.spec-mv-let, a better plet, etc.
Evaluate our ApproachEvaluate our Approach Determine the usefulness of parallelizing a modern Determine the usefulness of parallelizing a modern
semi-automatic theorem prover at the subgoal levelsemi-automatic theorem prover at the subgoal level• SpeedupSpeedup• How we present non-deterministic outputHow we present non-deterministic output
99
Key StepsKey Steps
Create version of ACL2 without sequential Create version of ACL2 without sequential dependencies (e.g., the modification of state and dependencies (e.g., the modification of state and pspv) in the main proof process (the waterfall)pspv) in the main proof process (the waterfall)
Introduce raw Lisp primitives and ACL2 Introduce raw Lisp primitives and ACL2 abstractions necessary to evaluate the waterfall abstractions necessary to evaluate the waterfall in parallelin parallel
Reincorporate the interactive portion of the Reincorporate the interactive portion of the waterfall (output and interrupts)waterfall (output and interrupts)
Evaluate the soundness and performance of our Evaluate the soundness and performance of our approachapproach
1010
Removing Sequential Removing Sequential Dependencies from the WaterfallDependencies from the Waterfall
StateState-based Challenges-based Challenges StateState is a special type of variable in ACL2 is a special type of variable in ACL2
• Used when performing I/O, when performing system calls, etc.Used when performing I/O, when performing system calls, etc.• Contains the “logical story” for these side-effectsContains the “logical story” for these side-effects
ACL2 restricts the use of ACL2 restricts the use of statestate in the following two ways in the following two ways• ACL2 restricts the name “state” from being used a variable ACL2 restricts the name “state” from being used a variable
anywhere that it doesn’t represent this one particular instanceanywhere that it doesn’t represent this one particular instance• ACL2 requires that if ACL2 requires that if statestate is modified, that the modified is modified, that the modified statestate be be
returned as part of the return valuereturned as part of the return value So, if we remove the modification of So, if we remove the modification of statestate from the waterfall, we from the waterfall, we
will know that the waterfall is (mostly) side-effect free.will know that the waterfall is (mostly) side-effect free.• Allows us to more easily find the big “gotchas” (e.g., I/O)Allows us to more easily find the big “gotchas” (e.g., I/O)
1111
Removing Sequential Removing Sequential Dependencies from the WaterfallDependencies from the Waterfall
StateState-based Solutions-based Solutions Preliminary step: remove I/O from the Preliminary step: remove I/O from the
waterfallwaterfall Disable proof techniques that require Disable proof techniques that require
modifying state (e.g., clause processors and modifying state (e.g., clause processors and computed hints)computed hints)• Skip the proofs of libraries that require those Skip the proofs of libraries that require those
techniquestechniques Results in skipping about 7% of the regression suiteResults in skipping about 7% of the regression suite Our current thoughts are that we can reinstate most of Our current thoughts are that we can reinstate most of
these techniques as demand occursthese techniques as demand occurs
1212
Removing Sequential Removing Sequential Dependencies from the WaterfallDependencies from the Waterfall
PSPVPSPV-based Challenges-based Challenges The Prover SPecial Variables (The Prover SPecial Variables (PSPVPSPV) data ) data
container acts as an accumulator for changes container acts as an accumulator for changes to variables that would be global, if ACL2 to variables that would be global, if ACL2 were written in a non-functional manner.were written in a non-functional manner.
Need to find a way to combine these changes Need to find a way to combine these changes that does not negatively affect the soundness that does not negatively affect the soundness or performance of ACL2or performance of ACL2
1313
Removing Sequential Removing Sequential Dependencies from the WaterfallDependencies from the Waterfall
PSPVPSPV-based Solutions-based Solutions If we know how to combine the changes If we know how to combine the changes
between the two proof steps, do so.between the two proof steps, do so. Otherwise, terminate the latter proof step and Otherwise, terminate the latter proof step and
restart it with the intermediate restart it with the intermediate PSPVPSPV value value Results in a computation exactly the same as Results in a computation exactly the same as
the serial computation, without a need to the serial computation, without a need to combine combine PSPVsPSPVs
1414
Component 2:Component 2:Introducing ParallelismIntroducing Parallelism
Adding futures to the underlying Adding futures to the underlying implementation language (Lisp)implementation language (Lisp)
Adding parallelism abstractions to the logic Adding parallelism abstractions to the logic (ACL2’s logic)(ACL2’s logic)
Managing outputManaging output Managing user interruptsManaging user interrupts
1515
Adding Futures to the Underlying Adding Futures to the Underlying Implementation LanguageImplementation Language
Create functions for spawning, reading, Create functions for spawning, reading, and terminating the evaluation of futuresand terminating the evaluation of futures (future x) :: X -> Future-structure(future x) :: X -> Future-structure (future-read x) :: Future-structure -> X(future-read x) :: Future-structure -> X (future-abort x) :: Future-structure -> C(future-abort x) :: Future-structure -> C
Relies upon our multi-threading interface Relies upon our multi-threading interface that unifies CCL and SBCL featuresthat unifies CCL and SBCL features
1616
Adding Parallelism Abstractions to Adding Parallelism Abstractions to the Logic the Logic
Modify Modify pletplet to support multiple values and speculative evaluation to support multiple values and speculative evaluation Give the ACL2 user a means to specify that a variable is unused in a Give the ACL2 user a means to specify that a variable is unused in a
particular expressionparticular expression Therefore, once evaluation enters a branch with such a specified Therefore, once evaluation enters a branch with such a specified
branch, branch, pletplet can terminate the evaluation of the unnecessary variable can terminate the evaluation of the unnecessary variable valuesvalues
Already defined Already defined pletplet differently in raw Lisp and the ACL2 logic differently in raw Lisp and the ACL2 logic• This is an enhancement of what we already created [Rager 2008, Rager and This is an enhancement of what we already created [Rager 2008, Rager and
Hunt 2009]Hunt 2009] Example usage:Example usage:
(plet (((x y) (mv 3 4))(plet (((x y) (mv 3 4)) ((q r) (mv 8 9)))((q r) (mv 8 9))) (if (equal q 8)(if (equal q 8) (check-vars-unused-and-kill (x y)(check-vars-unused-and-kill (x y) (+ q r))(+ q r)) (+ x y q r)))(+ x y q r)))
1717
Adding Parallelism Abstractions to Adding Parallelism Abstractions to the Logicthe Logic
By using this enhanced version of plet, we can create new abstractions, e.g. By using this enhanced version of plet, we can create new abstractions, e.g. spec-mv-letspec-mv-let
Spec-mv-let automatically performs the check that certain variables are Spec-mv-let automatically performs the check that certain variables are unused and automatically terminates unnecessary computationsunused and automatically terminates unnecessary computations
Example annotated usage:Example annotated usage:
(spec-mv-let (x y)(spec-mv-let (x y) ;; speculatively evaluate (mv 3 4);; speculatively evaluate (mv 3 4) (mv 3 4)(mv 3 4) (mv-let (q r)(mv-let (q r) (mv 8 9)(mv 8 9) (if (equal q 8)(if (equal q 8) ;; the speculative evaluation is irrelevant, return;; the speculative evaluation is irrelevant, return ;; a value that doesn’t use those results;; a value that doesn’t use those results (+ q r)(+ q r) ;; the speculative evaluation is useful, return a ;; the speculative evaluation is useful, return a ;; value that uses those results;; value that uses those results (+ x y q r))))(+ x y q r))))
1818
Component 3:Component 3:Managing InteractivityManaging Interactivity
Original Goal: maintain output consistent Original Goal: maintain output consistent between serial and parallel proofs of between serial and parallel proofs of subgoalssubgoals Store the output in a data structure and signal Store the output in a data structure and signal
when the next piece of output is ready for when the next piece of output is ready for printingprinting
I believe this to be feasible, but we would like I believe this to be feasible, but we would like to do moreto do more
1919
Managing OutputManaging Output
New Goal: print meaningful output as it becomes New Goal: print meaningful output as it becomes availableavailable Print the proof checkpoints as soon as they are Print the proof checkpoints as soon as they are
computedcomputed• There is usually a very minimal amount of checkpoint output There is usually a very minimal amount of checkpoint output
compared to the amount of output that occurs with a full compared to the amount of output that occurs with a full proof attempt’s narrativeproof attempt’s narrative
Users suggest that in the relatively rare case that a Users suggest that in the relatively rare case that a user wants to see the full narrative, that they would user wants to see the full narrative, that they would likely be satisfied with replaying the proof in a serial likely be satisfied with replaying the proof in a serial manner.manner.
2020
Managing OutputManaging Output
It is unclear what type of interface changes It is unclear what type of interface changes will be necessary to accommodate non-will be necessary to accommodate non-deterministic outputdeterministic output
An investigation into such An investigation into such accommodations is a planned part of this accommodations is a planned part of this workwork
An example of one potential interface An example of one potential interface followsfollows
21 / 29
2222
Managing User InterruptsManaging User Interrupts
When a user aborts a proof, all subgoal When a user aborts a proof, all subgoal computations will be gracefully terminated and computations will be gracefully terminated and the parallelism state of ACL2 will be reset to its the parallelism state of ACL2 will be reset to its initial stateinitial state
When a user attempts to debug a proof:When a user attempts to debug a proof: Automatically interrupt and pause other threadsAutomatically interrupt and pause other threads Automatically order those threads to resume Automatically order those threads to resume
evaluationevaluation Users also have access to these functions that pause Users also have access to these functions that pause
and resume subgoal computationsand resume subgoal computations
2323
Component 4:Component 4:Evaluating our ApproachEvaluating our Approach
SoundnessSoundness PerformancePerformance
2424
Evaluating SoundnessEvaluating Soundness
A large subset of the regression suite (93%) will A large subset of the regression suite (93%) will be able to pass with parallelism enabledbe able to pass with parallelism enabled
We rely on our strategic development and use of We rely on our strategic development and use of macros as our main source of credibilitymacros as our main source of credibility
Insert assertions that check for unexpected Insert assertions that check for unexpected changes in the program’s state.changes in the program’s state.
This branch of the build will be tuned for This branch of the build will be tuned for interactivity. Those desiring the assurance level interactivity. Those desiring the assurance level of non-parallel ACL2 can run the serial version.of non-parallel ACL2 can run the serial version.
2525
Evaluating PerformanceEvaluating Performance
Goal: Reduce the latency between when a Goal: Reduce the latency between when a user submits a conjecture and when the user submits a conjecture and when the user receives useful feedback concerning user receives useful feedback concerning that conjecture’s provabilitythat conjecture’s provability
We do not try to improve the performance We do not try to improve the performance of proof attempts that take < 1 secondof proof attempts that take < 1 second
2626
Evaluating PerformanceEvaluating Performance
We are not concerned with proof attempts that take less We are not concerned with proof attempts that take less than a second to compute, because anything between than a second to compute, because anything between 0.10 seconds and 1.0 seconds feels the same to a user.0.10 seconds and 1.0 seconds feels the same to a user.
Example performance result for certifying one of the Example performance result for certifying one of the ACL2 libraries:ACL2 libraries:
Summary for making books/ordinals/ordinal-addition.cert:Summary for making books/ordinals/ordinal-addition.cert:Average sequential time was: 6.22sAverage sequential time was: 6.22sAverage parallel time was: 5.09sAverage parallel time was: 5.09sSequential minimum was: 6.12sSequential minimum was: 6.12sParallel minimum was: 4.87sParallel minimum was: 4.87sOf 10 iterations, the parallel version was faster thanOf 10 iterations, the parallel version was faster thanthe sequential version 10 times.the sequential version 10 times.
2727
What We’ve Done so FarWhat We’ve Done so Far
Removed the modification of state and Removed the modification of state and output from the waterfalloutput from the waterfall
Modified Lisp library and ACL2 to permit Modified Lisp library and ACL2 to permit parallel evaluation of the waterfallparallel evaluation of the waterfall
Run some preliminary performance Run some preliminary performance benchmarksbenchmarks
2828
What’s LeftWhat’s Left
Reintroducing output in a helpful wayReintroducing output in a helpful way Improving performanceImproving performance Evaluating our solutionEvaluating our solution
2929
OutlineOutline IntroductionIntroduction
The Automated Proof Process – the The Automated Proof Process – the WaterfallWaterfall The ways ACL2 currently uses parallelismThe ways ACL2 currently uses parallelism
Research ContributionsResearch Contributions Key StepsKey Steps
Removing sequential dependenciesRemoving sequential dependencies Introducing Parallelism into the Proof ProcessIntroducing Parallelism into the Proof Process
• Adding futures to the underlying implementation languageAdding futures to the underlying implementation language• Adding parallelism abstractions to the logic Adding parallelism abstractions to the logic • Using parallelism in the waterfallUsing parallelism in the waterfall
Managing outputManaging output Managing user interruptsManaging user interrupts
Evaluating the Soundness and Performance of our Evaluating the Soundness and Performance of our ApproachApproach