Towards Higher-Order Superposition and SMT
Jasmin Blanchette
Outline
Counterexample Generation 1. Nitpick 2. Nunchaku
Proof Search 3. Sledgehammer 4. Matryoshka
1. Nitpick 1. A (countermodel) finder1. for Isabelle/HOL
Joint work withAlexander Krauss and Tobias Nipkow
Architecture
HOLFORL
SAT
Isabelle Nitpick .Kodkod.. .SAT solver
Translation
fixed finite cardinalities:try all cards. ≤ K for base types
τ1 ! ⋅ ⋅ ⋅ ! τn ! bool A1 × ⋅ ⋅ ⋅ × An⟼
τ1 ! ⋅ ⋅ ⋅ ! τn ! τ A1 × ⋅ ⋅ ⋅ × An × A+constraint
⟼
first-order
σ ! τ A × ⋅ ⋅ ⋅ × A
|σ|times{⟼higher-order
??
datatypes codatatypes
inductive preds. coinductive preds.
Con
3
Nil
Con
0
Con
2
Con
3
Nil
Con
0
Con
2
p = F p
p0 = (λx. False) pi+1 = F pi
p = F p
p0 = (λx. True) pi+1 = F pi
Translation
2. Nunchaku 2. A modular model finder2. for higher-order logic
Ongoing joint work withSimon Cruanes, Andrew Reynolds, and Cesare Tinelli
multiple frontendsIsabelle/HOL, TLAPS, Coq, Lean, …
multiple backendsCVC4, Kodkod, Paradox, SMBC, Leon, Vampire, …
more precisionby better approximations
more efficiencyby using better backends andby letting them enumerate cardinalities
Simplified Translation Pipeline
1. Monomorphize 2. Specialize 3. Polarize 4. Encode (co)inductive predicates 5. Encode (co)recursive functions 6. Encode higher-order functions
Actual Translation Pipeline
$ nunchaku --print-pipeline Pipeline: | ty_infer ➜ convert ➜ skolem ➜ | fork { | | mono ➜ elim_infinite ➜ elim_copy ➜ elim_multi_eqns ➜ specialize ➜ elim_match ➜ elim_codata ➜ | | polarize ➜ unroll ➜ skolem ➜ elim_ind_pred ➜ elim_quant ➜ lift_undefined ➜ model_clean ➜| | close {smbc ➜ id} | | mono ➜ elim_infinite ➜ elim_copy ➜ elim_multi_eqns ➜ specialize ➜ elim_match ➜ | | fork { | | | elim_codata ➜ polarize ➜ unroll ➜ skolem ➜ elim_ind_pred ➜ elim_data ➜ lambda_lift ➜ elim_hof ➜ | | | elim_rec ➜ intro_guards ➜ elim_prop_args ➜ | | | fork { | | | | elim_types ➜ model_clean ➜ close {to_fo ➜ elim_ite ➜ conv_tptp ➜ paradox ➜ id} | | | | model_clean ➜ close {to_fo ➜ fo_to_rel ➜ kodkod ➜ id} | | | } | | | polarize ➜ unroll ➜ skolem ➜ elim_ind_pred ➜ lambda_lift ➜ elim_hof ➜ | | | elim_rec ➜ intro_guards ➜ model_clean ➜ close {to_fo ➜ flatten {cvc4 ➜ id}} | | } | }
OCaml for Translation Pipeline
. . .
Encoding of higher-order functionsA simple approach
Replace HO σ → τ by hσ,τ
Add extensionality axiom
Add guards whenever app is used
Introduce appσ,τ : hσ,τ → σ → τand ασ,τ
Encoding of higher-order functions
∀x. f x = g x ?
f = g
rec fact x = (if x > 0 then x· fact (x – 1) else 1)
rec bad x = (if x = 666 then x else fact x)
fact = bad ?
An annoying imprecision
Encoding of higher-order functionsA more precise approach
Also introduce protoσ,τ : hσ,τ → σ
If (h, protoσ,τ h) in ασ,τ,then protoσ,τ h gives default value
f = g ∧ HAS_PROTO f
Encoding of higher-order functionsA more precise approach
HAS_PROTO h := ∃a : α. γ(a) = (h, proto h)
APP h s := if ∃a : α. γ(a) = (h, s) then app h s else app h (proto h) asserting HAS_PROTO h
3. Sledgehammer 2. Automatic proof search 2. for Isabelle/HOL
Joint work withSascha Böhme, Jia Meng, Tobias Nipkow,Larry Paulson, Makarius Wenzel, and many others
Does there exist a function f from reals to reals such that for all x and y, f(x + y2) − f(x) ≥ y?
let lemma = prove(`!f:real->real. ~(!x y. f(x + y * y) - f(x) >= y)`, REWRITE_TAC[real_ge] THEN REPEAT STRIP_TAC THEN SUBGOAL_THEN `!n x y. &n * y <= f(x + &n * y * y) - f(x)` MP_TAC THENL [MATCH_MP_TAC num_INDUCTION THEN SIMP_TAC[REAL_MUL_LZERO; REAL_ADD_RID] THEN REWRITE_TAC[REAL_SUB_REFL; REAL_LE_REFL; GSYM REAL_OF_NUM_SUC] THEN GEN_TAC THEN REPEAT(MATCH_MP_TAC MONO_FORALL THEN GEN_TAC) THEN FIRST_X_ASSUM(MP_TAC o SPECL [`x + &n * y * y`; `y:real`]) THEN SIMP_TAC[REAL_ADD_ASSOC; REAL_ADD_RDISTRIB; REAL_MUL_LID] THEN REAL_ARITH_TAC; X_CHOOSE_TAC `m:num` (SPEC `f(&1) - f(&0):real` REAL_ARCH_SIMPLE) THEN DISCH_THEN(MP_TAC o SPECL [`SUC m EXP 2`; `&0`; `inv(&(SUC m))`]) THEN REWRITE_TAC[REAL_ADD_LID; GSYM REAL_OF_NUM_SUC; GSYM REAL_OF_NUM_POW] THEN REWRITE_TAC[REAL_FIELD `(&m + &1) pow 2 * inv(&m + &1) = &m + &1`; REAL_FIELD `(&m + &1) pow 2 * inv(&m + &1) * inv(&m + &1) = &1`] THEN ASM_REAL_ARITH_TAC]);;
John Harrison
Does there exist a function f from reals to reals such that for all x and y, f(x + y2) − f(x) ≥ y?
[1] f(x + y2) − f(x) ≥ y for any x and y (given)
[2] f(x + n y2) − f(x) ≥ n y for any x, y, and natural number n (by an easy induction using [1] for the step case)
[3] f(1) − f(0) ≥ m + 1 for any natural number m (set n = (m + 1)2, x = 0, y = 1/(m + 1) in [2])
[4] Contradiction of [3] and the Archimedean property of the reals
John Harrison
intermediateproperties
generated automatically
manual
vs.
well suited for large formalizations
but require intensive manual labor
fully automatic
but no proof
management
Sledge- hammer
Proof assistants Automatic provers
h A
�=Isa
belle
`
_
Isabe
lle Vampire
� �
�=Isa
belle
�
�
HOL
select lemmas + translate to FOL
reconstruct proof
superposition
SMT
superposition SMT
refutational
resolution rule
term ordering
equality reasoning
E, SPASS, Vampire, …
redundancy criterion
refutational
SAT solver
+ congruence closure
+ quantifier instantiation
CVC4, veriT, Yices, Z3, …
+ other theories (e.g. LIA, LRA)
Other hammers in proof assistants
pre-SledgehammerOtter in ACL2 Bliksem in Coq Gandalf in HOL98 DISCOUNT, SPASS, etc., in ILF Otter, SPASS, etc., in KIV LEO, SPASS, etc., in ΩMEGA E, Vampire, etc., in Naproche ...
post-Sledgehammer
HOLyHammer for HOLs MizAR for Mizar SMTCoq/CVC4Coq for Coq SMT integration in TLAPS ...
HOL FOL
⟼ t(app(t(suc, fun(nat, nat)), t(n, nat)), nat)
⟼ app(suc, n)Suc n
higher-order features (e.g., currying)
⟼ t(suc(t(n, nat)), nat)Suc n
types (possibly polymorphic)xTraditional encodings of HOL in FOL
are inefficient
More efficient encodings ofhigher-order features are possible
FOL
Suc x
map f [x] = [ f x]
λ x y. y + x
HOL
p (x = x)
⟼ suc(x)
⟼ map(f, [x]) = [app(f, x)]
⟼ c(plus)
⟼ p(eq(x, x))
More efficient encodings oftypes are possible
Suc x
Nil ≠ Cons x xs
x ≠ y
HOL FOL
x = off
⟼ suc(x)
⟼ nil ≠ cons(A, x, xs)
⟼ x ≠ y
⟼ t(x, state) = off
Upon success,proofs are translated to Isabelle
one-line detailed (Isar)
lemma "length (tl xs) ≤ length xs" by (metis diff_le_self length_tl)
⊕ usually fast and reliable
⊕ lightweight
⊖ cryptic
⊖ sometimes slow (several seconds)
⊖ often cannot deal with theories
proof method lemmas
One-line proofs
lemma "length (tl xs) ≤ length xs" proof - have "⋀x1 x2. (x1∷nat) - x2 - x1 = 0 - x2" by (metis comm_monoid_diff_class.diff_cancel diff_right_commute) hence "length xs - 1 - length xs = 0" by (metis zero_diff) hence "length xs - 1 ≤ length xs" by (metis diff_is_0_eq) thus "length (tl xs) ≤ length xs" by (metis length_tl) qed
⊕ faster than one-liners
⊕ higher reconstruction success rate
⊕ self-explanatory
⊖ technically more challenging⊖ not always so self-explanatory
Detailed (Isar) proofs
I have recently been working on a new development. Sledgehammer has found some simply incredible proofs. I would estimate the improvement in productivity as a factor of at least three, maybe five.
Sledgehammers … have led to visible success. Fully automated procedures can prove … 47% of the HOL
Light/Flyspeck libraries, with comparable rates in Isabelle. These automation rates represent an
enormous saving in human labor.
Developing proofs without Sledgehammer is like walking as opposed to running.
Sledgehammer really works
Larry Paulson
Thomas Hales
Tobias Nipkow
Isabelle’s pros and cons,according to my students
11.5 Sledgehammer 4 Nitpick 4 Isar 2.5 automation 2 IDE 1 Quickcheck 1 set theory 1 schematic variables 1 structural induction 1 classical logic 1 function induction 1 infix operators 1 "qed auto"
⊕ 5 goal/assumption handling 4 weak logic (props as types, types as terms) 3 Sledgehammer on lists, HO goals, or induction 1 automatic induction 1 Sledgehammer-generated Isar 1 arithmetic 1 Isar 1 opaque proofs 1 double quotes around inner syntax 1 underdeveloped "fset" 1 proof reuse 1 no hnf for statements, not even definitions 1 guaranteed computability 1 forward "apply" in assumptions (drule?) 1 error messages in inner syntax 1 ltac (Eisbach?) 1 cannot click on fun to see definition (?) 1 tooltips for built-in functions etc.
⊖
Sledgehammer's main 3 weaknesses
⊖ Higher-order "lost in translation"
⊖ No induction
⊖ Explosive search space
4. Matryoshka 2. Higher-order automatic 2. provers for proof assistants
Ongoing joint work withAlex Bentkamp, Pascal Fontaine,Johannes Hölzl, Rob Lewis, Stephan Schulz,Uwe Waldmann, and many others
Vision: Take the Hard Labor out ofVision: Interactive Verification
Push button automation for proof assistants (e.g. Coq) based on efficient higher-order (HO) provers
{}
Σλ∫
Π
λ∫ {}
ΣΠ
4
super-position
prover
HOSMT
solver
Discover ProofUsing HO Provers
HO
First-Order Provers via SLEDGEHAMMER
Induction Rule
Simplifier
Arithmetic Procedure
General Reasoner
“PC members cannot review papers if they have a conflict of interest”
Proof today:using assms proof induction case (Step s a) thus ?case proof (cases a) case (Cact ca) show ?thesis using Step pref_Conflict_isRev reach.Step by simp next case (Uact ua) show ?thesis proof (cases ua) case (uPref confID uID p paperID pref) thus ?thesis using Step unfolding Uact uPref isRev_def2 by (blast dest: pref_Conflict_isRevNth reach.Step) qed (insert Step, simp add: Uact isRev_def2 u_defs pref_Conflict_isRevNth_def)+ next case (UUact uua) show ?thesis using Step unfolding UUact isRev_def2 by (meson IO_Automaton.reach.Step pref_Conflict_isRevNth) qed simp+ qed (simp add: istate_def)
fully automatic5
Application: A Verified “EasyChair”
First-Order Provers via SLEDGEHAMMER
Induction Rule
Simplifier
Arithmetic Procedure
General Reasoner
Proof today:using assms proof induction case (Step s a) thus ?case proof (cases a) case (Cact ca) show ?thesis using Step pref_Conflict_isRev reach.Step by simp next case (Uact ua) show ?thesis proof (cases ua) case (uPref confID uID p paperID pref) thus ?thesis using Step unfolding Uact uPref isRev_def2 by (blast dest: pref_Conflict_isRevNth reach.Step) qed (insert Step, simp add: Uact isRev_def2 u_defs pref_Conflict_isRevNth_def)+ next case (UUact uua) show ?thesis using Step unfolding UUact isRev_def2 by (meson IO_Automaton.reach.Step pref_Conflict_isRevNth) qed simp+ qed (simp add: istate_def)
fully automatic5
Application: A Verified “EasyChair”
“PC members cannot review papers if they have a conflict of interest”
First-Order Provers via SLEDGEHAMMER
Induction Rule
Simplifier
Arithmetic Procedure
General Reasoner
Proof today:
fully automatic
manual hints
6
using assms proof induction case (Step s a) thus ?case proof (cases a) case (Cact ca) show ?thesis using Step pref_Conflict_isRev reach.Step by simp next case (Uact ua) show ?thesis proof (cases ua) case (uPref confID uID p paperID pref) thus ?thesis using Step unfolding Uact uPref isRev_def2 by (blast dest: pref_Conflict_isRevNth reach.Step) qed (insert Step, simp add: Uact isRev_def2 u_defs pref_Conflict_isRevNth_def)+ next case (UUact uua) show ?thesis using Step unfolding UUact isRev_def2 by (meson IO_Automaton.reach.Step pref_Conflict_isRevNth) qed simp+ qed (simp add: istate_def)
First-Order Provers via SLEDGEHAMMER
Application: A Verified “EasyChair”
“PC members cannot review papers if they have a conflict of interest”
boilerplate
Discover ProofUsing HO Provers
using assms proof induction case (Step s a) thus ?case proof (cases a) case (Cact ca) show ?thesis using Step pref_Conflict_isRev reach.Step by simp next case (Uact ua) show ?thesis proof (cases ua) case (uPref confID uID p paperID pref) thus ?thesis using Step unfolding Uact uPref isRev_def2 by (blast dest: pref_Conflict_isRevNth reach.Step) qed (insert Step, simp add: Uact isRev_def2 u_defs pref_Conflict_isRevNth_def)+ next case (UUact uua) show ?thesis using Step unfolding UUact isRev_def2 by (meson IO_Automaton.reach.Step pref_Conflict_isRevNth) qed simp+ qed (simp add: istate_def)
Proof after Matryoshka:
fully automatic
Discover ProofUsing HO Provers
7
missing proof
Application: A Verified “EasyChair”
{}Σ λ∫Π
λ ∫ {}Σ Π
“PC members cannot review papers if they have a conflict of interest”
Extend superposition and SMT to higher-order logicSO1.
Design practical methods and heuristics based on benchmarksSO2.
Conceive stratified architectures to build higher-order proversSO3.
Integrate our provers into proof assistants (Coq, Isabelle, TLA+)SO4.
My Grand Challenge
Create efficient proof calculi and higher-order proverstargeting proof assistants and their applications tosoftware and hardware development
Scientific Objectives
by fusing and extending two lines of research:automatic proving & interactive proving
‣
8
SO1—Higher-Order Superposition (λSUP)
9
A "counterexample": a =β (λx. a) (f a) > f a > a
The problem is not so much λ as β.
SO1—Higher-Order Superposition (λSUP)
9
σ = mgu(t, u) u is not a variable tσ ≰ t'σ sσ ≰ s'σ ≰ (t ≈ t' )σ is strictly maximal in (D' ⋁ t ≈ t' )σ and no selection ≰ (s ≉ s' )σ is maximal in (C' ⋁ s ≉ s' )σ or selected ≰
where
First-order rule:
D' ⋁ t ≈ t'
(D' ⋁ C' ⋁ s[t'] ≉ s')σSUP-Left
C' ⋁ s[u] ≉ s'
9
σ = mgu(t, u) u is not a variable tσ ≰ t'σ sσ ≰ s'σ ≰ (t ≈ t' )σ is strictly maximal in (D' ⋁ t ≈ t' )σ and no selection ≰ (s ≉ s' )σ is maximal in (C' ⋁ s ≉ s' )σ or selected ≰
where σ = mgu(t, u)
We need sequences of unifiers ‣
SO1—Higher-Order Superposition (λSUP)
First-order rule:
D' ⋁ t ≈ t'
(D' ⋁ C' ⋁ s[t'] ≉ s')σSUP-Left
C' ⋁ s[u] ≉ s'
9
D' ⋁ t ≈ t'
(D' ⋁ C' ⋁ s[t'] ≉ s')σSUP-Left
C' ⋁ s[u] ≉ s'
σ = mgu(t, u) u is not a variable tσ ≰ t'σ sσ ≰ s'σ ≰ (t ≈ t' )σ is strictly maximal in (D' ⋁ t ≈ t' )σ and no selection ≰ (s ≉ s' )σ is maximal in (C' ⋁ s ≉ s' )σ or selected≰ ≰
where
We need sequences of unifiers ‣
σ = mgu(t, u)
We need higher-order term ordering ‣
tσ ≰ t'σ sσ ≰ s'σ
We also want proof-assistant-style HO rewriting ‣
SO1—Higher-Order Superposition (λSUP)
First-order rule:
10
SO1—Higher-Order Term Orderings
Well-foundedness
Transitivity
Stability under substitution
FO subterm property
Totality for ground terms
KBO
✔
✔
✔
✔
✔
✔
LPO
✔
✔
✔
✔
✔
✔
Compat. with FO contexts
First-order
10
SO1—Higher-Order Term Orderings
Well-foundedness
Transitivity
HO subterm property
Compat. with HO contexts
Stability under substitution
FO subterm property
Totality for ground terms
KBO
✔
✔
✔
✔
✔
✔
LPO
✔
✔
✔
✔
✔
✔
Compat. with FO contexts
CPO/ HORPO
✔
✘
✔
✔
✘
?
✘
?
λfKBO
✔
✔
✔
✔
✔
✔
(✔)
λfLPO
✔
✔
✔
(✔)
✔
✔
✔
✔ ✔
First-order Higher-order
SO3—Stratified Architecture
11
rulesFO
formulasFO
rulesHO
formulasHO
main loop
Inspired by Nelson–Oppen (SMT)
Base FO provers: E & veriT
Some scientific challenges:
How to exploit derived FO formulas and/or candidate models to guideHO quantifier instantiation?
‣
How to generate certificates forreconstruction in proof assistants?
‣
Matryoshka Prover (e.g. veriHOT)First-Order Prover (e.g. veriT)
SO3—Higher-Order SMT
12
First-order quantifier instantiation (QI):
E-matching (triggers) ‣Model-based QI ‣
Congruence closure with free variables ‣Conflict-guided instantiation ‣
We need to extend these strategiesto higher-order logic
‣
FOQI
HO QI
loopmain
Matryoshka Prover (e.g. veriHOT)First-Order Prover (e.g. veriT)
SAT Solver (e.g. MiniSat)
Proof Assistant (e.g. Coq) 13
Matryoshka Prover (e.g. veriHOT)First-Order Prover (e.g. veriT)
SAT Solver (e.g. MiniSat)
SO4—Connection with Proof Assistants
SO4—Connection with Proof Assistants
14
DependentType Theory
ClassicalHigher-Order Logic
Set Theory
Coq Isabelle/HOL TLA+
… … …
Agda HOL4 Isabelle/ZFLean HOL Light Mizar
Matita PVS Rodin (Event-B)
veriHOT
HOE
veriHOT
HOE
veriHOT
HOE
The Team
15
Associated Members:Other Collaborators:
Scientific Leader:Senior Collaborator:
Postdoctoral Researchers:
Ph.D. Students:
Jasmin BlanchettePascal FontaineJohannes HölzlRob LewisAlex BentkampDaniel El Ouraoui Hans-Jörg Schurr Petar Vukmirović
Stephan SchulzUwe WaldmannHaniel BarbosaSimon CruanesSimon Robillard& more
Adam NcyAdam Adam Adam Ncy Ncy Adam
Stgt SBNcy Ncy Gbg
m a t r y o s h k a
y
λm a t r y o s h k a
Matryoshka in one Slide
Grand Challenge & Outcome
Create efficient proof calculi (λSUP & λSMT) andstratified higher-order provers (HO E & veriHOT)to dramatically improve automation in proof assistants
RisksEfficient HO automation is a long-standing open problem ‣Proposed stratified architecture has never been tried ‣
ImpactThe project will recast the methods of automatic provingto reach the goals of interactive provingInteractive verification will become a cost-effective optionfor building software & hardware of the highest quality
‣
‣
16
m a t r y o s h k a
y
λm a t r y o s h k a