Staffan Berg - T&VS · iTBA Verification Results •Reached coverage in a single iTBA sim vs multiple CRT sims/seeds •Minimum 10x coverage closure advantage for simple CRT cases

Staffan Berg

European Applications Engineer

Graph-Based Verification in a UVM Environment

July 2012

© 2010 Mentor Graphics Corp. Company Confidential

www.mentor.com

Graph-Based Intelligent Testbench Automation (iTBA)

Welcome DVClub — Attendees — Organizers — Presenters

Verification Challenges — Time To Market — Quality Products

Step Function Gains — Higher Coverage Faster — Testbench Automation

at SoC Level — Testbench Re-Use — Verification Productivity

Questa - April 2011 2


www.mentor.com

AGENDA

Graphs, Rules & Intelligent Testbench Automation

Why Graph-Based Verification is important

Fundamentals of iTBA

The Role of Coverage

Advanced Concepts

Graph based stimulus in the Context of OVM/UVM/*VM

Applications and Results – what to expect

SB , Graph Based Verification June 2012 3


www.mentor.com

Intelligent Testbench Automation (iTBA) From a Graph-Based Stimulus Description Model

4

Gary Smith -

“ . . .The [automatic] generation of a testbench from a system-level design description. . . “

Start = init repeat ( wait_rdy Rw_opts Rw_size ack ) ;

Rw_opts = setup_rd | setup_wr ;

Rw_size = rw_1 | rw_2 | rw_4 ;

Stop

init

Rw_opts

Rw_size

setup_rd setup_wr

rw_4 rw_2 rw_1

ack

Start

wait_rdy

SB , Graph Based Verification June 2012

Mentor Graphics -

“ . . . testbench automation that is aware

of the valid test space, the engineer’s

verification targets within that space,

and the current state of the design - that

uses automation to efficiently achieve

the verification goals . . . “

iTBA Definitions

The graph defines the ―valid test space‖


www.mentor.com

Benefits of Graph-based Verification

Predictable Coverage Closure — Achieve coverage goals in a fraction of the time — Graph techology enables N vs N*ln(N) advantage

Facilitate Re-use — Stimulus model is independent of testbench environment

– Same model can be re-used for SystemC, RTL, Emulation… – Same model can be re-used between projects

Brings iTBA to SoC level — Automatically generate stimulus at RTL block level — Automatically generate stimulus at SoC level (s/w & h/w)

Productivity improvements — Generation of testbench code — Graphical analysis / debug — Visualizing test scenario correlation to design spec



www.mentor.com

Defining the valid test space - Rules

Rules have two sections: — Declarations — Grammar

Declarations include: — Graph nodes linked to tasks/functions — Variables linked to tasks/functions — Rule building blocks

Grammar defines abstract behavior — Replaces directed test procedural code — Replaces CRT constraint code — Often declarative decision tree of choices — May also use algebraic constraints

Rules are implementation-independent

iTBA Rule

User-Created Rule Text

1

6 SB , Graph Based Verification June 2012


www.mentor.com

Visualizing the test space - Graphs

compile 2

• Rules compiled into graphs

• Graphs visually depict stimulus

• Protocol behavior

• Packet construction options..

• Stimulus sizing

• Stimulus sizing useful to assess

• What to target for coverage

• What to generate randomly

• Simulation time needed to reach coverage goals

• For this example

• 864 total test combinations

• 108 tests in CovParams sub-graph

Rules



www.mentor.com

Defining the Verification Goals - Coverage

• Coverage Strategy defines Goals & Priorities

• Specify regions to cover graphically

• Generate stimulus coverage code

• Two types of stimulus coverage

• Path coverage – traversal paths / cross coverage

• Node coverage – like a system verilog coverpoint

• Path coverage atomic_tb_bl_combos covers

• 108 combinations of am, bt, bl

• Node coverage bsz[] covers

• 8 values of bsz

Path Coverage

Node Coverage

User Added Stimulus Coverage Directives 3



www.mentor.com

Graph-based iTBA in Simulation

tb.sv top.sv iTBA_comp*.sv

Compiled Graph(s)

compile

iTBA runtime

Simulator

Runtime Graph

Debugging

• Integration/compilation

• Plug-in to existing methodologies

• Runtime code

• Graphs loaded into simulator

• Algorithms manage graphs during simulation

• Runtime graph debug

• Debug TB or DUT problems

• Set graph breakpoints

• Single-step or run to breakpoint



www.mentor.com

The Role of Coverage

The rule graphs define the entire stimulus space

Without coverage goals, traversal will be purely random

Graph coverage strategy defines goals and priorities

iTBA algorithms prioritize generation — Can still generate random vectors

outside the coverage space



www.mentor.com

Graph-based Verification Advanced Concepts Reactive / adaptive graphs

Some fields in a stimulus item may be a property of the testbench or DUT state, not randomly selected — And may still be included in a cross cover goal

These fields become inputs (imports) to a stimulus graph — Enables opportunistic targeting of random-resistant scenarios


Graph behaviour adapts to TB or DUT state

DUT state can be used to change constraints


www.mentor.com

Integrating Graph Based Verification Generating Stimulus in an OVM/UVM Environment

Stimulus is generated by sequences

A sequence produces sequence items

Driver applies stimulus to DUT



www.mentor.com

Generating Graph-Based Stimulus

Graph integrates within a sequence — Graph execution produces sequence items

Maximizes reuse of infrastructure — No need to change existing drivers, monitors, etc

iTBA Sequence



www.mentor.com

iTBA OVM/UVM Integration Process

Identify the target sequence Item

Describe stimulus domain with a graph • Declare a graph variable for each sequence-item field • Define relationships between graph variables

Define stimulus-coverage goals • Often corresponds to existing functional coverage goals

Run the iTBA sequence via an OVM/UVM test



www.mentor.com

Sequence Item

Verifying a simple bus protocol — 32-bit address — 64-bit data bus — Supports burst lengths up to 16 beats

Existing UVM sequence Item — Contains fields that describe a transaction



www.mentor.com

Defining Graph Variables

Declare a graph variable for each item field

Declare the valid domain of each variable

Sequence Item Rules



www.mentor.com

Declare Variable Relationship

Simple bus protocol constraints — Address must be aligned to the transfer size — Burst-transfer beats may only be 32 or 64-bit width

Graph permits flexible description of relationships — Branches — Algebraic constraints

Reuse existing constraints — Can be imported automatically



www.mentor.com

Simple Bus Protocol Graph

Graphical view of the stimulus space — Automatically created from rule description — Intuitive way to visualize choice tree

Graph branches restrict burst_len/size — Enables visual approach to code review



www.mentor.com

Graph Integration into Sequence

Graph nodes link to tasks in the sequence class

Tasks set the value of sequence-item fields

Integration code is automatically-created from graph



www.mentor.com

Define Stimuli-Coverage Goals

Simple protocol coverage goals — Cover 64 address ranges — Cover all valid combinations of transaction parameters

Describe address value binning

Define graph-coverage strategy — Node Coverage for address — Path Coverage for transaction parameters

Pre-simulation coverage space analysis



www.mentor.com

Testbench Environment Integration

iTBA sequence is just like any other sequence — Use standard approaches to select and execute

Select the iTBA sequence via a type override

Explicitly create and run iTBA sequence



www.mentor.com

Example iTBA Applications

Architectural exploration — Performance characterization — Behavioral model verification

IP Block level verification — Bus (AXI, AHB, PCI*, …) — Memory (DDR,SDDR, nand flash,…) — Peripheral (i2c, dma,…)

SOC integration — Firmware — Subsystem interactions

Processor verification — L2 cache — Instruction set verification



www.mentor.com

iTBA Verification Results

• Reached coverage in a single iTBA sim vs multiple CRT sims/seeds

• Minimum 10x coverage closure advantage for simple CRT cases

• Often see >100x coverage closure advantage for common CRT cases



www.mentor.com

Graph-Based iTBA Gaining Acceptance Across The Globe

Office

Products

Printer Image

Processor

Questa

SystemVerilog

3

Days

36 hours on 6 CPUs 8 weeks on 6 CPUs

100% coverage 60% coverage

37 X faster

+ 40% coverage =

Wireless

Networking

Ethernet 802.11

Device

VCS

NTB 35

Hours

48 CPU hours 3175 CPU hours


66 X faster

+ 2 % coverage =

Consumer

Electronics

Error Checking and

Correcting Module

NC Sim

Specman e

1

Day

118 minutes >18 hours


9.5 X faster

Equal coverage =

Storage &

Networking AXI Bus Bridge

VCS

SystemVerilog

1

Week

170 X faster 196,000 tests 26,315,000 tests

100% coverage 79% coverage + 21% coverage =

Basestation

Telecom

Proprietary Interface

Module for a Router

NC Sim

Specman e

1

Week

75,000 vectors 825,000 vectors


10 X faster

Equal coverage =

Industry Design Verification Time Current Results Ultra Results Benefits

Switching

Subsystems

Multiple Master

AXI Bus Fabric

Questa

Directed Tests

2

Days

400,000 tests 10,000 tests 40 X more tests =

Wireless

Telecom Interrupt Controller

VCS

Vera and SV

6

Days

45 minutes 3 days


27 X faster

Equal coverage =

Processors Multi-Core Memory

Sub-system

Questa

SystemVerilog

1

Day

30 minutes 5 hours


10 X faster

Equal coverage =

170 X faster

37 X faster

9.5 X faster

10 X faster

66 X faster

+ 40% coverage

+ 21% coverage

40 X more tests

27 X faster

10 X faster

iTBA - Accelerating Time to Coverage Closure 24


www.mentor.com

Conclusions….

Graph algorithms enable fast, efficient — Coverage closure — Test scenario generation

Graph-based stimulus description emphasizes reuse — Vertically from block to SoC level

– And across multiple testbench environments / languages

— Horizontally from project to project

Easily integrate into existing environments, e.g. OVM/UVM



www.mentor.com SB , Graph Based Verification June 2012 26

Documents

Staffan Berg - T&VS · iTBA Verification Results •Reached coverage in a single iTBA sim vs multiple CRT sims/seeds •Minimum 10x coverage closure advantage for simple CRT cases