Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
IBM Labs in Haifa © 2006 IBM Corporation
Formal Verification of Business Continuity Solutions
Emmanuel Zarpas, Sivan TalIBM Research Lab in Haifa, May 2006
IBM Labs in Haifa
© 2006 IBM Corporation2
Agenda
Rationale
What is Formal Verification / Model Checking
How can FV be applied to BCS
Case study
Conclusion
Acronyms:FV = Formal Verification
BCS = Business Continuity Solutions (or Systems)
DR = Disaster Recovery
IBM Labs in Haifa
© 2006 IBM Corporation3
Why is FV useful for storage/systems solutions
Human-built BCSs often fail to meet the expected levels of robustness and business continuanceWhy?
They involve sophisticated control functions of coordination, interactions and synchronization between components.Understanding the specifications of the components is difficult;misinterpretations are frequent.The system is expensive to implement for testing purposes, and testing coverage for business scenarios in test labs is usually poor.
This also applies to automated systemsOur goal is to cope with these difficulties using FV techniquesWe are doing research on applicability of FV for various systems
IBM Labs in Haifa
© 2006 IBM Corporation4
What is Formal Verification ?
Prove mathematically that a design obeys its specification
Methods: theorem proving, model checkingContrast to simulation testing:
Formal Verification Simulation Testing
all legal input sequences (large) set of particular cases
correctness expressed as set of general properties
correctness usually expressed per run (expected results)
IBM Labs in Haifa
© 2006 IBM Corporation5
Model Checking - Our model from 50,000 feet
Our verification system mathematically proves properties of the model’s behavior
Requirements(Properties/Assertions)
Req Holds?
[Yes] / [No, with failure scenario]
Exhaustive analysis
Covering all possible scenarios
Model
IBM Labs in Haifa
© 2006 IBM Corporation6
Model Checking (cont.)
Given a model M and a temporal logic formula f, decide if M |= f
A model M = (S, S0, R, P, L):
A PSL (IEEE standard) formula f = always (request -> Next[1..3] process)(meaning: if a request is received, it must be processed within 3 cycles)
Model is non-deterministic (checks all input sequences simultaneously)
0
1
2 5
6
34
process
request
request
S : states
S0 : initial states
R : transition relations
P : atomic propositions
L : labels (s,p)
IBM Labs in Haifa
© 2006 IBM Corporation7
Model Checking (cont.)
A model is not usually described explicitly (too many states)
A bit-by-bit description describes a model implicitly
Example in the PSL formal language:
var a, b, c : boolean;
assign next(a) := b & c;
assign next(b) := if a then c else {0,1};
assign next(c) := if b then !a else c;
n bits describe a model of size 2n011
110
000
001
100
010
101
111
IBM Labs in Haifa
© 2006 IBM Corporation8
Formal Modeling and Verification of BCS
FV is a proven technology in hardware designIn hardware, used for verification of both a design and its implementationAlso used for verification of protocolsMany millions of development expense savings in chip development
Advantages of formal modeling for BCSmakes requirements explicit and unambiguousimproves documentation and understanding of the designprovides a firmer foundation for enhancements and maintenance
Advantages of formal verification for BCSDesign quality assuranceCan be used to certify systems and solutions
For BCS, the design is verified - not the implementation
IBM Labs in Haifa
© 2006 IBM Corporation9
Case Study: BCS Developed by Customer
Synchronous mirroring of both data and log volumes – performance impactAsynchronous mirroring with consistent snapshot once a day
JFS data
log
data
sync mirror
sync mirror
PiT Copy
PiT Copya
IBM Labs in Haifa
© 2006 IBM Corporation10
Case Study: Async mirroring with PiT CopyThe async mirroring function does not retain order of writes at the remote mirror
Iterates on all blocks in an infinite loop and copies blocks that were updated since last visited
After first pass, transition to full-sync synchronous mirror can be requested and would complete in short time
p.i.t. consistent copy
async mirror inconsistent image
Momentarily quiescing the application and taking snapshot of the mirror generates a consistent PiT copy and then async mirroring can be resumed
Quiesce I/O async mirrorSyncfull sync PiT Copy
IBM Labs in Haifa
© 2006 IBM Corporation11
Case Study: The Requirement on the BCS
This BCS must guarantee that the content of volume C is always identical to the content of A at some point in time no more than 1 day ago.
But... there are some subtle issues:
Order and timing of quiesce I/O, move to full sync, PiT Copy
Want to optimize for performance and meet BC requirement above.
When the system gets more complicated, the requirements become more complicated to define and prove, and more issues arise…
A B C
IBM Labs in Haifa
© 2006 IBM Corporation12
Case Study: Where this BCS Might Fail
1st problem: A wrong assumption about the time it takes to reach full sync mode.
A B C
1 13 3
2
13
sync-pending PiT Copy
2nd problem: There were actually two volumes involved: data and log. They depend on one another, and the snapshots must be coordinated so as to create a consistent copy of the application data image.
async mirror
sync mirror
PiT Copy
PiT Copy
IBM Labs in Haifa
© 2006 IBM Corporation13
Formal Modeling – an Example
Consider the 3 volumes in the case study example
Assume each volume contains a single block that can have either of two values, say 0 and 1Define four operations/events:
Write 0 to AWrite 1 to ACopy A to BCopy B to C
A B C
IBM Labs in Haifa
© 2006 IBM Corporation14
Finite State Machine Diagramwrite 0 to A
copy A to BCopy B to C
0 | 0 | 0write 1 to A
0 | 0 | 1 0 | 1 | 0 0 | 1 | 1
1 | 0 | 0 1 | 0 | 1 1 | 1 | 0 1 | 1 | 1
copy A to B
copy A to B
copy A to B
copy A to B
write 1 to A
copy B to C
copy A to BCopy B to C
Copy B to C
write 0 to A
Write 1 to A
Write 0 to A
write 1 to A
write 0 to A
Copy B to C
write 1 to A
copy B to C
write 1 to A
copy A to B
write 0 to A
write 1 to A
write 1 to A
write 0 to A
write 0 to Acopy B to C
Each state is represented by the values of the 3 data items
IBM Labs in Haifa
© 2006 IBM Corporation15
PSL specification of the model
Model representation:var a, b, c : boolean;
var update_mirror, update_snapshot : boolean;
assign init(a) := {0,1};
assign next(a) := {0,1};
assign init(b) := {0,1};
assign next(b) := if update_mirror then a else b;
and so on...
Properties:assert always( (a = 1) -> next[1..8](c = 1) );
and so on...
0 | 0 | 0 0 | 0 | 1 0 | 1 | 0 0 | 1 | 1
1 | 0 | 0 1 | 0 | 1 1 | 1 | 0 1 | 1 | 1
IBM Labs in Haifa
© 2006 IBM Corporation16
The Basic Paradigm
Develop an abstraction of the BC/DR solutions domain
Formalize the abstraction in a formal language, creating a model which represents:
1. The components of the solution and their behaviors
2. Set of assertions about the requirements from the solution
Using the RuleBase PE™ verification platform (developed in our IBM lab), verify the specific models against specific assertions, or find scenarios where an assertion fails
We have initial results: Modeled an automated DR system design and caught a subtle design bug that would have led to a recovery failure and extended downtime.
IBM Labs in Haifa
© 2006 IBM Corporation17
Conclusion
Formal Verification / Model Checking successfully applied in HW design
Can be leveraged for compound storage systems and solutions
Formalization helps create better design
Formalization helps understanding specifications and relationships
Formal verification assures bullet-proof design
Formal verification may be used as certification tool
Research work in progress, current plans are:
Develop a human-friendly tool to verify/certify “user” BCS
Develop framework for FV of system design