Diagnosing Abstraction Failure in Separation Logic-based Analyses

DIAGNOSINGABSTRACTION FAILUREIN SEPARATION LOGIC-

BASED ANALYSES

Arlen CoxJosh BerdineSamin Ishtiaq

Christoph Wintersteiger

The Abstraction Refinement Dream

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

Failure

Diagnose FailureFailure

State of the ArtSeparation Logic Analysis

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

Failure

Diagnose FailureFailure

Previously Unexplored1

2

Traditional Abstraction Refinement

Not Our Contributio

n

Pick Abstract/Attempt Proof

Proof Fails

Diagnosing Abstraction Failure

WeakestPrecondition

1. An Abstract State2. Concrete State

• Unreachable• Reaches Error• Contained in

Abstract State

Partition the Abstract State

No WP() in Separation Logic

WeakestPrecondition

No WP() in Separation Logic

int* p;

…

*p = 17;PSPACE-

complete*due to aliasing

* Calcagno, C., Yang, H., O’Hearn, P.W.: Computability and complexity results for a spatial assertion language for data structures. In: FSTTCS (2001)

Separation Logic-based Analyses

Restricted Logic• Does not support separating implication ( ),

general negation ( ), general conjunction ( ) Do not support backward reasoning• No weakest precondition

Contribution: A method to use forward analysis to diagnose failures

Contribution: A method for efficiently performing forward counterexample searches

…l

l

Examplel = new ListNode(new Obj(), NULL);while(*) { l = new ListNode(new Obj(), l);}while(l != NULL) { n = l->next; free(l->data); free(l); l = n;}

NULL

Background: Pick Abstraction

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

FailureDiagnose Failure

Failure

Pattern-Based Abstraction…

l

NULL

Pattern-Based Abstraction…

l

NULL

Pattern-Based Abstraction…

l

NULL

Background: Proof Attempt

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

FailureDiagnose Failure

Failure

Proof Search (SLL)

l = new ListNode(new Obj(), NULL);while(*) { l = new ListNode(new Obj(), l);}while(l != NULL) { n = l->next; free(l->data); free(l); l = n;}

Proof Search (SLL)

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), NULL);while(*) { l = new ListNode(new Obj(), l);}while(l != NULL) { n = l->next; free(l->data); free(l); l = n;}

Proof Search (SLL)

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), NULL);while(*) { l = new ListNode(new Obj(), l);}while(l != NULL) { n = l->next; free(l->data); free(l); l = n;}

l = new ListNode(new Obj(), l);

Proof Search (SLL)

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

assume(l != NULL)n = l->next;free(l->data);

l = new ListNode(new Obj(), NULL);while(*) { l = new ListNode(new Obj(), l);}while(l != NULL) { n = l->next; free(l->data); free(l); l = n;}

Counterexamples

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

FailureDiagnose Failure

Failure

Traditional Bounded Model Checking

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

assume(l != NULL)n = l->next;free(l->data);free(l);l = n;

assume(l == NULL)

1.Unroll Transition System

2.Check Property

3.Repeat- Can explode for deep properties

- Doesn’t help proof process

Not Our Contributio

n

Contribution: BMC Over Abstract Transition System

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

assume(l != NULL)n = l->next;free(l->data);

1.Unroll Abstract Transition System

2.Check Property3.Repeat+ Restricted

search space+ Finds counter-

examples that caused this proof failure

Contribution: BMC Over Abstract Transition System

Contribution: BMC Over Abstract Transition System

Contribution: BMC Over Abstract Transition System

Must End in Error

Contribution: BMC Over Abstract Transition System

Contribution: BMC Over Abstract Transition System

Unroll up to a bound

Contribution: BMC Over Abstract Transition System

Stay in Error

Contribution: BMC Over Abstract Transition System

Otherwise Transition

According to Program

Contribution: BMC Over Abstract Transition System

Send to SMT solver; quantifiers and all.

Contribution: BMC Over Abstract Transition System

Send to SMT solver; quantifiers and all.

Encoding ofData

Allocated

Size

Address

p = malloc(size);

Data

Allocated

Size

Address

p = malloc(size);q = malloc(size);

Encoding of

Data

Allocated

Size

Address

p = malloc(size);q = malloc(size);

Encoding of

Data

Allocated

Size

Address

p = malloc(size);q = malloc(size);r = p + size;*r = 3; //(no error)

Encoding of

Data

Allocated

Size

Address

p = malloc(size);q = malloc(size);r = p + size;*r = 3; //(error)

Encoding of

Counterexample Search

l = new ListNode(new Obj(), NULL);

assume(l != NULL)n = l->next;free(l->data);

l = new ListNode(new Obj(), l);

Counterexample Search

l = new ListNode(new Obj(), NULL);

assume(l != NULL)n = l->next;free(l->data);

l = new ListNode(new Obj(), l);Just need structure.

Don’t need separation logic

formulas

Counterexample Search

No Error

Counterexample Search

l = new ListNode(new Obj(), NULL);

No Error

l

NULL

Counterexample Search

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

Error Unreachabl

e

assume(l != NULL)n = l->next;free(l->data);

No Error

NULL

l

Counterexample Search

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

Error Unreachabl

e

assume(l != NULL)n = l->next;free(l->data);

No Error

NULL

l

Counterexample Search

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

Error Unreachabl

e

assume(l != NULL)n = l->next;free(l->data);

No Error

Error Unreachabl

e

assume(l != NULL)n = l->next;free(l->data);

NULL

l

l = new ListNode(new Obj(), l);

Counterexample Search Produces concrete counterexamples Contribution: Only explores failed proof• Finds counterexamples that would cause

this particular proof failure Contribution: Relies on SMT solver for

unrolling• Property-guided, intelligent backtracking

Bit-precise memory model

Contribution: Diagnosing Failure

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

FailureDiagnose Failure

Failure

Diagnosing the Failure

l = new ListNode(new Obj(), NULL);

assume(l != NULL)n = l->next;free(l->data);

Was the abstraction here responsible for

failure?

l = new ListNode(new Obj(), l);

Diagnosing the Failure

assume(l != NULL)n = l->next;free(l->data);

Delete Program

Before Join Point

l = new ListNode(new Obj(), l);

Diagnosing the Failurel = NULL

assume(l != NULL)n = l->next;free(l->data);

l = new ListNode(*, l);

Synthesize Program Prefix that Creates

Abstract State Precisely

Error Found!

l = new ListNode(new Obj(), l);

Re-run Counterexam

ple SearchNon-

deterministic data field

Diagnosing the Failure

Diagnosing the Failure

Diagnosing the Failure

Diagnosing the Failure

Diagnosing the Failure

for p in Join_Points(ATS) { ATS’ = Synthesize_Prefix(p, ATS) CEx = Find_Counterexample(ATS’) if(exists CEx) { ATS = Refine(ATS, p, CEx); }}

Picking New Abstraction

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

FailureDiagnose Failure

Failure

Picking New Abstraction Partial order of abstractions Pick next best abstraction

Proof Search with SLL_OBJ

l = new ListNode(new Obj(), NULL);

l = new ListNode(new Obj(), l);

assume(l != NULL)n = l->next;free(l->data);free(l);l = n;

assume(l == NULL)

Conclusions

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

Failure

Diagnose FailureFailure

Conclusions

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

Failure

Diagnose FailureFailure ✔

New BMC Approach• Search abstract

transition system instead of program• Only finds causes for

proof failure• Use monolithic

encoding• Take advantage of

intelligent backtracking

Conclusions

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

Failure

Diagnose FailureFailure ✔

✔

New Approach to Diagnosis• Synthesize program

prefix• Use guided

counterexample search to diagnose• Find failing

abstraction• Find failing concrete

value contained by abstraction

Conclusions

StartVerification

Pick Abstraction

AttemptProof

Pick New Abstraction

Success

Fix Bug

Success

FindCounterexampl

e

Failure

Diagnose FailureFailure ✔

✔

-

Questions?