Upload
antonia-hancock
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
Verification Techniques for Macro Blocks (IP)
OverviewInspection as VerificationAdversarial TestingTestbench DesignTiming Verification
Overview of Macro Verification
Overall verification planStrategiesSub-block simulationMacro simulationPrototypingLimited production
Overall Verification Plan
Functional verification based on Requirements Specifications Behavioral models Extended functions (for generalization
and re-use)
Engineering review of verification planDocumentation of verification steps
Strategy: Bottom Up Testing
Verify individual sublocks Focus on coverage of data values and internal
states
Verify complete macro (block) Focus on interfaces and timing among sub-blocks Test I/O timing and “corner cases”
Perform prototyping Proto-board PC board FPGA Platform based design*
Verification vs. Validation
In the hardware world: Verification means testing Validation means “formal proof”
In the software world: Verification means “formal proof” Validation means testing
Don’t be confused…
Verification vs. Manufacturing Testing
Verification is to confirm that the design is correct It meets the requirements,
specifications It meets timing, power, and area goals It meets the re-use goals
It assumes a debugging cycleManufacturing Test Fixed set of tests (vectors from above) Go/NoGo tests
Fundamental IdeasControllability Can you wiggle every wire? Can you set every bit?
Observability Can you see the state of every bit? Can you see the value on every wire?
During validation, we can do this in the simulator – especially in a bottom-up mode
(In manufacturing test this is not so easy)
Types of Verification TestsCompliance Testing Against specifications and/or standards
Corner Case Testing Try to break the design with complex scenarios “Answer Yes or No [Y]?” > fred
Random Testing Use statistical models to generate data patterns
Real Code Testing Work with the software team, use real data
Regression Testing Keep all tests in a suite and add to them
Testing in the Product Development Cycle
More complete tests find more subtle bugsFixes often make design more complexTest time goes upNew bugs are introduced by fixesRepeat…
time
bu
gs
fou
nd
Verification Tools Simulation functional, cycle, RTL, gate level, circuit level
Testbench automation tools Verisity (and others) testing language
Code coverage tools Measurements of what got wiggled
Hardware Modeling Comparison against / with real hardware
Emulation High speed hardware boxes interfaced to
simulator
Prototyping
“Static” Tools Static timing analysis tools“Lint-like” toolsFormal verification tools Model checking
Build abstract model of design and environment Uses “smart” exhaustive methods for safety and
liveness tests Liveness: something good will eventually happen Safety: nothing bad will ever happen
Theorem proving Uses automatic theorem provers to verify assertions
about the design
Inspection as Verification“The fastest, cheapest, and most effective way to
detect and remove bugs is by careful inspection of the design and the code”Design reviews Large group review requirements, specifications,
assumptions and overall design
Code reviews Smaller groups perform line-by-line review of
implementation
Reviews must be done in a collegial atmosphere with the goal of enhanced quality, not to asses designer performance
Prototyping
Use Prototyping when: You can afford it
$cost $time
You can not wait for the time it takes to find bugs using available simulation techniques The prototype is just a fast simulator for
the device itself
Adversarial TestingThe designer’s tests have the goal to
show that the system will work…Adversarial Testing uses a verification team of experts in testing all kinds of designsGoal: develop tests to show that the system will failReport failures, and test cases to design team
Sub-Block Simulation1. Use automated response checking
Not appropriate for designer to check waveforms “by eye”
How many times will it be done? Use of automated test to build macro
block testbench Use assertions in code to test assumptions
2. Test sub-blocks for 100% statement and path coverage
Testbench
The testbench For each component Is the model of its external world Must generate (all) legal input sequences Must check output sequences Must alert the designer and log
discrepancies between expected and actual results
The testbench is often more complex than component itself
Sub-block Testbench Design
• Input transaction data types, sequences
• Output transaction data types, sequences
InputTransactionGenerator
OutputTransaction
Checker
Device UnderTest (DUT)
Assume simple I/O relationships
Input Stimulus
Generate “all” legal sequences of inputsLook at functionality of DUTCheck corner casesAchieve 100% coverage
Output Checking
Check for legal sequences of outputsLook at functionality of DUTCheck corner casesCheck input/output relationships Ideally each sub-block has
Simple Functionality Simple I/O
Getting ready for integration
Timing checks Timing plan for each sub-block Clocking, handshaking Timing budgets, per-cycle latency
budget
Area and power checks Are the sub-blocks within their bugets
Concurrent Design, Synthesis & Testbench Development
Block Integration
Functional Specifications, Documentation, Behavioral Model
Validate against Specification and Test Plan / Productization
DevelopMacro-
Test Bench,Test Cases
Coordinate TimingSpecify Blocks
Code and Test Synthesize
Test Full Block Synthesize Full Block
Macro Specifications / Behavioral
Model
Macro Partitioning and Block
Specifications
Block Design
Test Team Design Team Synthesis Team
Integration Steps
Integrating the sub-blocks Top level netlistFinal synthesisFinal testingManufacturing testabilityScan insertionATPGTest coverage
Timing Verification
Post synthesis timing “more real” delays
Delay models based on statistical models No layout Not real delays
Check for clock/data skew Signal loading, critical path analysis
Post Synthesis Test Bench Development
Refine the Behavioral Model for testing and verification Wrappers for each sub-block as
needed (for signal compatibility before/after synthesis) After synthesis every wire is a “bit”
Document test plan and test vectors
Macro Testbench
Design is more complex Functional complexity I/O Complexity
More people involvedTestbench will be delivered with product Testbench will be used to develop
scan path and manufacturing tests
Generating Testbenches
Not simple “do scripts” Complex state machines: Loop
Generate stimulus Get output Check results Log results Change stimulus parameters
Simple ExampleSimulate Busses
Check bus timing Check data ranges
Simulate ROM Also, supply data
values Check address ranges
Register File Also, r/w timing Overwriting data
I/O Also, response to
asynchronous events Response to out of
range data
DUT
ROMMemory
RegisterFile
I/OA
ddre
ss B
us
Data
Bus
I/O
Bus
Testbench
Clo
ck/T
imin
g G
enera
tor
Help!Use existing models for bench components Have a test team, with their own test library Take library components and enhance them with
checking/logging functions
Use behavioral models for bench Bus functional models
Need to have statistical bounds on input/output timing Use random generators for stimulus sequences
Use C/C++ models and co-simulation for benchNeed to have a “master bench controller” to coordinate I/O behaviors on all components
Use the Behavioral Model
InputTransactionGenerator
OutputTransaction
Checker
SynthesizedBlock
BehavioralModel
Use Commercial Tools
Vera - Synopsis Specman Elite – VerisityBoth tools use abstract data types to
allow user to define data-ranges, states, and scenarios.
Tools support statistical models, for stimulus, checking, and logging
(beyond the scope of this class)
Test, Test, Test
Back to Requirements and SpecificationsCorner CasesRandom CasesRegression TestingCode Coverage
Types of CoverageStatement Coverage
# times each statement executed
Branch Coverage Each side of each if/then else executed
Condition Coverage Checks each boolean clause in branch condition
Path Coverage Paths between blocks (if – if, if – else, else – if)
Trigger Coverage Each signal on sensitivity list
Toggle Coverage 1/0 transition on every signal
Still Not Enough
Does not verify that code works right, just that it was runDoes not verify that design meets specifications or requirementsDoes not check results of optimization and synthesis tools