Technion

Haifa Research Labs Israel Institute of Technology

Underapproximation for Model-Checking Based on Random Cryptographic Constructions

Arie Matsliah (presenting)and Ofer Strichman

IBM / Technion2

Introduction

Motivation: Efficient “bug-hunters” for heavy verification instances

Underapproximation: M, M’ – Kripke structures M’ underapproximates M if for every LTL formula φ: M φ → M’ φ

M’ has a subset of the behaviors of M

Our goal: Automatic and efficient underapproximation-based model checking

IBM / Technion3

Model-checking with underapproximation

Potentially good for falsification, not verification.

M’Model-

checker

M’ φ

M’ φ

?

failφ

M

Refine: add behaviors

IBM / Technion4

The time complexity of model checking depends exponentially on the number of inputs

Natural approach for Underapproximation: reduce # of inputs.

What makes Model Checking hard?

M’

inputs

outp

uts

M

inputs

outp

uts

IBM / Technion5

Reducing the number of inputs

An underlying assumption:

“The values of some of the inputs are immaterial for exposing the bug”

A simple technique for underapproximation: fixing inputs. Pick those inputs manually (using high-level information). Fix their value.

A similar process which is automatic and complete is ineffective.

Our method: reduce # inputs without fixing any.

IBM / Technion6

Our contribution

Underapproximation which: Reduces the number of inputs Maintains a measurable and uniform degree of freedom to the original

inputs Based on adding circuitry to the model.

Can be applied to any form of verification

Moriginal inputs

outputs

M’C

new inputs

inputs

outputs

IBM / Technion7

Main idea - Universality

A (combinatorial) circuit C is k-universal if any valuation of at most k of its outputs ... ...can be reached under some assignment to its inputs.

Example: 2-universal circuit

inputs outputs

00 0 0 0

10 1 0 1

01 0 1 1

11 1 1 0

Why universality? if #(important inputs) ≤ k, then k-universal circuit is enough

inputs

outputs

C

IBM / Technion8

Universality of some naïve methods

Fixing some of the inputs to constants

0-universalM’

Minputs

outp

uts

0

1

1

0

Merge groups of inputs together

1-universal

M’

M

inputs

outp

uts

C

C

IBM / Technion9

Inspiration - Pseudo Random Generators (PRGs)

Generator

random string

pseudorandom string

looks random for any

poly-time algorithm

f f f f f f f

PRG construction [NW 94]:- the circuit has certain properties- f is “hard to invert”

Our construction:- the circuit is random- f is a XOR function

IBM / Technion10

Using universal circuits

M

original inputs

outputs

M’

C

new inputs

IBM / Technion11

Constructing universal circuits

1

1

1

1

1

1

1

1

1

1

1 1

1 1

1 1

1 1

1 1outputs (inputs of M)

inputs (inputs of M’)

o1 o2 o3 o4 o5 o6 o7

i1 i2 i3 i4 i5 i6

i1 i2 i3 i4 i5 i6

o1

o2

o3

o4

o5

o6

o7

C

iiio 6311

j

jjh,h i)(Ao

j

jj1, i)(A

m

2

1

n

2

1

i

...

i

i

A

o

...

o

o

Arandom matrix

mod 2

IBM / Technion12

How universal is C?

Lemma: if every k rows in A are linearly independent – C is k-universal Proof (for k=3, n=7, m=6):

1

1

1

1

1

1

1

1

1

1

1 1

1 1

1 1

1 1

1 1

i1 i2 i3 i4 i5 i6

o1

o2

o3

o4

o5

o6

o7

A

1

1

1

1

1

1

1

1 1

i1 i2 i3 i4 i5 i6

o2

o4

o7

A’

6

2

1

7

4

2

i...ii

ooo

A’

A’ has full rank all 23 values covered

IBM / Technion13

How universal is C?

Lemma: for k=O(m/log n), with high probability,

every k rows in A are linearly independent Proof (for k=3, n=7, m=6):

1

1

1

1

1

1

1

1

1

1

1 1

1 1

1 1

1 1

1 1

i1 i2 i3 i4 i5 i6

o1

o2

o3

o4

o5

o6

o7

A

1

1

1

1

1

1

1

1 1

i1 i2 i3 i4 i5 i6

o1

o4

o6

A’

Pr[A1 is in span(A4,A6)] ≤ 22/26

for general k,m,n: Pr[ … ] ≤ 2-m+k-1

Apply Union Bound

A1

A4

A6

IBM / Technion14

How universal is C?

Lemma: for k=O(m/log n), with high probability,

every k rows in A are linearly independent Lemma: if every k rows in A are linearly independent – C is k-universal Corollary: for k=O(m/log n), with high probability, C is k-universal

Sample values:

IBM / Technion15

Better bounds for k

What if we relax the requirement?

Lemma: for any ε > 0 and k ≤ m - log m – log (1/ε),

each subset of k outputs is covered with probability 1-ε

for any k ≤ m - log m – 7,

each subset of k outputs is covered with probability ~0.99

Sample values:

k cannot be larger than m

m 20 30 40 50 70 100 200 500 800 1000

k 7 18 28 37 57 86 185 484 783 983

IBM / Technion17

What now?...

The main contribution of the work is theoretical: Showing relevance of universality to model-checking. Proving universality properties of PRG-like circuits.

Experiments show that indeed universality matters.

The challenge: from theory to practice.

IBM / Technion18

Experiments

Implemented in IBM RuleBase PE

17 BMC instances with known bugs

For each design with n inputs, we generated a new design with m inputs, for m = n/2, n/3, n/5, n/10

We compared the following methods: Our: Our circuit with m inputs. Orig: No underapproximation Fix: Fixing n-m inputs to some constant. Set: Partitioning the inputs to m sets. All inputs in the same set are

mapped to a single input.

IBM / Technion19

Orig Our FixDesign inputs (n) n n/2 n/3 n/5 n/10 n/2 n/3 n/5 n/10

IBM#1 45 96 66 63 66 63 246 - - -

IBM#2 76 173 149 76 72 68 - - - -

IBM#3 76 191 127 77 79 - 373 - - -

IBM#4 85 211 170 121 105 140 191 317 - -

IBM#5 68 61 65 20 592 - - - - -

IBM#6 68 73 59 14 661 - - - - -

IBM#7 68 482 308 46 52 - - - - -

IBM#8 68 122 152 16 90 - - - - -

IBM#9 64 2101 1915 1966 1654 1208 1693 - - -

IBM#10 80 1270 1392 1830 1137 - - - - -

IBM#11 83 2640 2364 2254 1845 - - - - -

IBM#12 6 8201 7191 - - - - - - -

IBM#13 60 942 453 432 351 - 1206 - - -

IBM#14 218 965 735 778 510 396 - - - -

IBM#15 52 1206 - - - - - - - -

IBM#16 157 953 - - - - - - - -

IBM#17 68 21503 TO TO TO TO - - - -

Run-times

-13.6% -17.5% -22.7% -47.1% 4.7% 50.2%

IBM / Technion20

Orig Our SetDesign inputs (n) n n/2 n/3 n/5 n/10 n/2 n/3 n/5 n/10

IBM#1 45 96 66 63 66 63 223 229 227 231

IBM#2 76 173 149 76 72 68 361 446 - -

IBM#3 76 191 127 77 79 - 168 317 - -

IBM#4 85 211 170 121 105 140 306 289 405 -

IBM#5 68 61 65 20 592 - 410 - - -

IBM#6 68 73 59 14 661 - - - - -

IBM#7 68 482 308 46 52 - 561 491 - -

IBM#8 68 122 152 16 90 - 113 - - -

IBM#9 64 2101 1915 1966 1654 1208 2150 - - -

IBM#10 80 1270 1392 1830 1137 - - - - -

IBM#11 83 2640 2364 2254 1845 - - - - -

IBM#12 6 8201 7191 - - - - - - -

IBM#13 60 942 453 432 351 - 413 407 - -

IBM#14 218 965 735 778 510 396 969 1102 - -

IBM#15 52 1206 - - - - - - - -

IBM#16 157 953 - - - - - - - -

IBM#17 68 21503 TO TO TO TO TO - - -

Run-times

-13.6% -17.5% -22.7% -47.1% 6.2% 7.2% 105.9% 140.6%

IBM / Technion21

The effect of m and p

Tested 4 heaviest designs with various m and p’s

Depth in which bug was found, was increased in this many designs:

1/2 1/3 1/5 1/10

n/2 0 0 0 0

n/3 0 0 0 0

n/5 0 0 0 0

n/10 0 0 0 1

m

p

inputs

probability of each input to be included in

the fanin

IBM / Technion22

Future work

1. Attach the circuit C to the unrolled model

2. Refinement strategies

3. Construct universal circuits without XORs

4. Construct universal circuits deterministically

5. Experiments with (unbounded) model-checking + simulation

M0C

M1

M2

Mk

IBM / Technion23

Thank you!